BGP Dataset-Based Malicious User Activity Detection Using Machine Learning

Round 1

Reviewer 1 Report

The paper is well structured with clearly defined topics. It addresses an important issue of cyberattacks analysed with machine learning approach for detection of anomalies from BGP data. Nevertheless, I have a few suggestions that need to be taken into account before the publication. There is very little description of experiments and collected data sets used in training and evaluation of ML algorithms presented in two figures and tables, they should be better described in the text, checked and reviewed regarding the following comments below:

1) Introduction

Ref [1] was accessed in 2023, but the information is not exactly summarized as it is available on the website.

2) Related work

Check the label of Table 2 - ?

Figure 1 is taken from reference [3] and should therefore be cited accordingly. The right side of the picture has only 3 trees - why?

Lines 182-183: In training of One SVM is specified to be used only normal data – this is unusual for any classification problem - justify this claim.

Figure 2, is not significant for the description by showing only general scheme of AE- it can be deleted. AE is then presented in context with the detection model in Figure 5.

 3) In section 3 »BGP Data description and Anomaly detection«, the description of experiments in the proposed environment with access to BGP data, in real time, should be expanded and described in more detail also where and how long did one or several experiments last.

It is also misleading in some cases to say abnormal data (e.g.: L236, …) and in another anomalous data (L239, L244, …) - this should be unified.

 4) In Experiments (did you perform one or several experiments for data collection) is missing the detailed description of datasets used in supervised ML algorithms (number of samples, selection of training and test subsets, …). It needs to be defined either the two datasets in Figure 7 are used for training or evaluation. In case of evaluation in real time which is based on new data, normally you would use the received imbalanced set – highlight both cases.

In 4.2 is very extensive description of evaluation metrics, but is missing the important information of ML algorithm's parameters (only parameters for AE are listed).

As AE is unsupervised and other algorithms are supervised there should be also mentioned the comparison of their training and performance evaluation of anomalies detection in real time.

It is also important to highlight the experimental results based on upper comments. Why AE results outperform the others, can you add any explanation.

 5) In Conclusion is written »Finally, the superiority of our research is emphasized by providing better detection  capabilities and various auxiliary indicators than other studies that use BGP data to detect cyber anomalies.« How can you justify this claim when you use your own BGP dataset and not the same as the one evaluated in other references.

 6) Additional comments:

- Check the text and style of the manuscript

- arrange properly:  No equation with label (1) found; Use of One-SVM and 1-SVM, SNE and t-SNE, …) and check for duplication in Abbreviations.

Author Response

Reviewer#1, Concern # 1: Ref [1] was accessed in 2023, but the information is not exactly summarized as it is available on the website.

Author response and action: Thank you for thoroughly reviewing the paper.

This part has been organized and rewritten.

Reviewer#1, Concern # 2: 2) Related work

 

Check the label of Table 2 - ?

 

Figure 1 is taken from reference [3] and should therefore be cited accordingly. The right side of the picture has only 3 trees - why?

 

Lines 182-183: In training of One SVM is specified to be used only normal data – this is unusual for any classification problem - justify this claim.

 

Figure 2, is not significant for the description by showing only general scheme of AE- it can be deleted. AE is then presented in context with the detection model in Figure 5..

Author response and action: Thank you for your careful review of my paper.

Table 2 has been updated to Table 1.

Figure 1 is intended to depict the structure of RF. We merely referred to the structure of Figure RF in the references but did not alter the numbering or structure itself.

We apologize for not being able to thoroughly review the One-SVM section. Furthermore, if any relevant parts were modified upon request, additional information has been included in the paper.

Upon reviewing your feedback, we have recognized that Figure 2 and Figure 5 are identical images. Consequently, Figure 2 now displays the autoencoder structure we employed, while Figure 5 (referred to as Figure 6 after modification) illustrates the reconstruction error values.

 

Reviewer#1, Concern # 3:  In section 3 »BGP Data description and Anomaly detection«, the description of experiments in the proposed environment with access to BGP data, in real time, should be expanded and described in more detail also where and how long did one or several experiments last.

It is also misleading in some cases to say abnormal data (e.g.: L236, …) and in another anomalous data (L239, L244, …) - this should be unified.

 

Author response and action: Thank you for your feedback.

A typo was identified during the review. In the conclusion, this paper suggests the potential for real-time detection but does not actually implement real-time detection.

The requested experimental environment has been documented in Table 6. Furthermore, all instances of 'anomalous data' have been replaced with 'abnormal data.'

Edits to the paper have been finalized.

 

Reviewer#1, Concern # 4:  In Experiments (did you perform one or several experiments for data collection) is missing the detailed description of datasets used in supervised ML algorithms (number of samples, selection of training and test subsets, …). It needs to be defined either the two datasets in Figure 7 are used for training or evaluation. In case of evaluation in real time, which is based on new data, normally you would use the received imbalanced set – highlight both cases.

 

Author response and action: Thank you for your feedback.

The information you requested is provided in Section 4.2. The details of the experimental environment you requested are presented in Table 6. Additionally, all instances of 'anomalous data' have been corrected to 'abnormal data.'

Regarding Figure 7, there was a typographical error, and the same dataset as Figure 8 was mistakenly used. Additionally, Tables 4, 5, 6, and 7 have been added to report the parameter information of all models in the paper. As previously mentioned, the term 'real-time' was an error that occurred during the translation from Korean to English. We apologize for this oversight.

To highlight the effectiveness of AE, a confusion matrix is presented in Figure 11 in Section 4.3, along with a note indicating that it achieved a detection accuracy of 0.99 without overfitting.

 

Reviewer#1, Concern # 5:   In Conclusion is written »Finally, the superiority of our research is emphasized by providing better detection capabilities and various auxiliary indicators than other studies that use BGP data to detect cyber anomalies. « How can you justify this claim when you use your own BGP dataset and not the same as the one evaluated in other references.

 

Author response and action: Thank you for your careful review of my paper.

The part you mentioned is "Finally, to detect cyber anomalies, we used BGP data to detect anomalies, and the detection abilities of Accuracy 0.99, Precision, Recall, f1-score, and AUROC curve were overloaded to 0.98, 0.99, 0.99, and 0.96, respectively. It was modified to “It was shown that summation does not occur.”

 

Reviewer#1, Concern # 6:   Additional comments:

- Check the text and style of the manuscript

- arrange properly:  No equation with label (1) found; Use of One-SVM and 1-SVM, SNE and t-SNE, …) and check for duplication in Abbreviations.

Author response and action: Thank you for your careful review of my paper.

Test style and vocabulary have been unified

Reviewer 2 Report

I have some comments.

(1) In section one, research motivation is not clear. What is the shortcomings of existing researches related to detect and classify anomalous behavior through BGP using machine learning ? 

(2) In table 2, there are four anomaly detection algorithm using BGP data. Why do the authors not perform a comparative analysis between these four algorithms and the proposed approach, based on the same architecture mentioned in this paper?

(3) During the processes of proposed approach, the authors use several methods, such as tokenizer and SMOTE. What kinds of cost should the system pay? 

(4) Lines 503 and 504 should be deleted.

Author Response

Reviewer#2, Concern # 1:    In section one, research motivation is not clear. What is the shortcomings of existing researches related to detect and classify anomalous behavior through BGP using machine learning ? 

Author response and action: Thank you for your careful review of my paper.

The BGP data presents a challenge for machine learning models due to its combination of textual and numerical data, which makes direct model training challenging. Furthermore, the limited availability of abnormal data samples for anomaly detection exacerbates the difficulties encountered during both model training and the anomaly detection process.This content has been added to Section 1

 

Reviewer#2, Concern # 2:    In table 2, there are four anomaly detection algorithm using BGP data. Why do the authors not perform a comparative analysis between these four algorithms and the proposed approach, based on the same architecture mentioned in this paper? 

Author response and action: Thank you for pointing out the shortcomings in my paper.

Regarding Table 2, we conducted machine learning analysis using BGP data only for the Autoencoder. As a result, performing a comparative analysis with RF, One-SVM, and CNN-LSTM is challenging. Furthermore, our paper stands out from existing research as it analyzes the autoencoder after preprocessing data using tokenizer and SMOTE techniques, highlighting a significant distinction.

 

Reviewer#2, Concern # 3:  During the processes of proposed approach, the authors use several methods, such as tokenizer and SMOTE. What kinds of cost should the system pay? 

Author response and action: Thank you for reading my paper

CPU operations are essential for tasks such as data processing with Tokenizer and various preprocessing tasks. The time required for CPU usage depends on factors like the complexity of the preprocessing procedures, data volume, and CPU performance. For instance, handling extensive BGP datasets demands a high-performance CPU, potentially increasing CPU usage time. GPUs, on the other hand, excel in parallel computing and are well-suited for processing large-scale data intricacies. For instance, when employing the SMOTE algorithm, multiple GPU cores can process data and generate samples concurrently. GPU usage time depends on GPU performance, parallel processing capabilities, and algorithm efficiency.

 

Reviewer#2, Concern # 4:  Lines 503 and 504 should be deleted.

Author response and action: Thank you for pointing out the shortcomings in my paper.

The part you requested has been deleted.

Reviewer 3 Report

The authors introduced an anomaly detection method for BGP data. The reviewer considers that the paper cannot be accepted unless the methods introduced in the manuscript can be reproduced. According to the reviewer, there is a lack of transparency with regards to the methods implemented. For instance, which are the architectures of the AE and CNN-LSTM? Which are the hyperparameters of the models and how they have been selected? Furthermore, the novelty is unclear. The analysis consists of implementing four widely known methods, which have been analysed extensively for anomaly detection in academia. Furthermore, the novelty introduced with regards to the data pre-processing is presented in the experimental section in tandem with the case study and results. By introducing the novelty in such a way, when the novelty is one of the most important parts of a research paper, may be a symptom of the lack of contribution of the paper. Also, the following points should be considered: 

1. 1) More keywords should be considered. 

2. 2) Please revise style (font style differences between computer and network and the remain text in line 31 of the introduction section). Analogously, the font style in the studies column of Table 2 differs from row to row (e.g., please see Copstein and Choudhay, for instance).  

3. 3) Analogously, please revise reference style (e.g., [8 line 89 page 2). 

4. 4) The reviewer suggests replacing etc. of page 3 line 122 with all models identified. 

5. 5) Table 2 states that data mining was implemented. Data mining is a process not a technique. Which was the technique employed in the modelling stage to extract patterns from data? 

6. 6) Please revise the numbering of images and tables. The first table introduce in the paper is Table 2, and not Table 1. 

7. 7) The machine learning methods introduced in section 2.3. are widely known (RF, combination of CNN and LSTM, One-SVM, and AE). Thus, the information provided can be summarised. Furthermore, there is plenty of information that can be implemented in this sense to add value. For instance, which kernel has been utilised when considering One-SVM? How was this determined? Which is the architecture of the proposed autoencoder? Why were VAE or other types of deep learning approaches not considered? Regarding CNN-LSTM: did the authors considered average or max pooling layers? How many convolutional layers were added? These are just examples of information expected in this section. As the authors did not provide the code, all information with regards to the architecture, selection of the hyperparameters and training process should be provided. 

8. 8) Please provide the libraries utilised to develop the proposed code (scikit-learn, TensorFlow, etc.). 

9. 9) The models introduced are very generic. Information should be provided in a more specific manner based on the application domain, which relates to cybersecurity in this instance. For example, why CNN-LSTM should be applied in cybersecurity for anomaly detection? Which are the temporal and spatial patterns expected to be retrieved from BGP data? 

10. 10) Figure 2 should be adapted to present the architecture of the proposed AE. 

11. 11) Analogously, a diagram with the architecture of the CNN-LSTM should be provided in section 2.3.2. 

12. 12) Figure 5 is not relevant, as it presents the same information as Fig. 2. Furthermore, as the authors stated, the output of the AE is the reconstruction of the input data, and not whether the input data is normal or abnormal. Thus, one can say the figure is inaccurate. The reviewer considers either removing the figure from the manuscript or include the method that determines the dynamic threshold to differentiate between normal and abnormal data based on the reconstruction error. 

13. 13) Based on the preceding point, please provide information about how the anomaly score was obtained and how this has been considered to determine if the data is normal or abnormal. 

14. 14) t-SNE and PCA are widely known. Thus, the information included in the experimental section should be summarised. Including a sentence indicating why the authors implemented t-SNE instead of PCA is more than enough. 

15. 15) Please indicate why RF, one-SVM, CNN-LSTM, and AE were considered, as there are other state-of-the-art methods that outperform them. 

16. 16) Analogous to the preceding points, the metrics introduced in page 11 are widely known. Thus, this text should not be included in the manuscript. The reason of estimating these metrics in the introduced case study should be given instead. 

17. 17) Did the authors implement any type of cross-validation to ensure the generalisation capabilities of the classifiers? If not, please do so. 

18. 18) In the abstract F1-Score is introduced to highlight the performance of the models. However, the data is clearly imbalanced. Please provide a more adequate metric result. 

19. 19) Please provide the confusion matrix for BGP Data and BGP Data (Tokenizer).

20. 20) The authors indicated that SMOTE technique was applied to address the issue of data imbalance. However, the ratio of abnormal data differs significantly from the ratio of normal data, even after of applying SMOTE. 

20. 20) How SMOTE compares with other data augmentation methods? Please provide a comparative study with baseline models. 

21. 21) Figure 7 and 8 differs conceptually. Are the labels of Figure 7 incorrect? Please revise. 

22. 22) The reviewer considers that the statements provided by the authors (e.g., “superiority of our research”) is not objective, as, for instance, one may say that the statement is too optimistic when a comparative study of four widely known classifiers have been provided and the “novelty” of the study is introduced in the experimental section in tandem with the case study and results. Please rephrase such statements in a more objective manner.

Minor english errors were perceived.

Author Response

Reviewer#3, Concern # 1: More Keywords should be considered.

Author response and action: Thank you for thoroughly reviewing the paper. We sincerely appreciate your feedback. As you pointed out, the keywords were sorely lacking. We have added the following keywords to address the above issue. 'Anomaly detection', 'Cyber security, 'Autoencoder'

Reviewer#3, Concern # 2:  Please revise style (font style differences between computer and network and the remain text in line 31 of the introduction section). Analogously, the font style in the studies column of Table 2 differs from row to row (e.g., please see Copstein and Choudhay, for instance).  

Author response and action: We followed your advice and unified all font styles with Tables 2, 3, and 4

Reviewer#3, Concern # 3:  Analogously, please revise reference style (e.g., [8 line 89 page 2). 

Author response and action: Thank you for your careful review of my paper. We've also changed the font style in the sections you requested

Reviewer#3, Concern # 4: The reviewer suggests replacing etc. of page 3 line 122 with all models identified.

Author response and action: Thank you for your suggestion. We've added the names of all models as you requested

Reviewer#3, Concern # 5: Table 2 states that data mining was implemented. Data mining is a process not a technique. Which was the technique employed in the modelling stage to extract patterns from data?

Author response and action: Thank you for the update. We have made changes to the background color of Figure 6 to improve the visibility of the colors.

Reviewer#3, Concern # 6: Please revise the numbering of images and tables. The first table introduce in the paper is Table 2, and not Table 1.

Author response and action: Thank you for taking the time to review my paper. We have corrected Table 2 to Table 1 as you pointed out.

Reviewer#3, Concern # 7: The machine learning methods introduced in section 2.3. are widely known (RF, combination of CNN and LSTM, One-SVM, and AE). Thus, the information provided can be summarised. Furthermore, there is plenty of information that can be implemented in this sense to add value. For instance, which kernel has been utilised when considering One-SVM? How was this determined? Which is the architecture of the proposed autoencoder? Why were VAE or other types of deep learning approaches not considered? Regarding CNN-LSTM: did the authors considered average or max pooling layers? How many convolutional layers were added? These are just examples of information expected in this section. As the authors did not provide the code, all information with regards to the architecture, selection of the hyperparameters and training process should be provided.

 

Author response and action: Thank you for taking the time to review my paper.

As requested, we have added a description of the kernel and an explanation of why we used it to the paper. In addition, we have added a description of the calculation process in Hyperplane in One-SVM.

Generative adversarial network (GAN) or variational autoencoder (VAE) can be considered effective methods for dealing with imbalance issues, but their models focus on mimicking the distribution of the data. This means that they generate less anomalous data than normal data. Furthermore, these models are not suitable as anomaly detection models because their main purpose is to generate data according to the data distribution, not to detect anomalies. AE is a more powerful model for dimensionality reduction and feature extraction than other deep learning models. In anomaly detection tasks, it is important to detect and extract specificity, and AE is very good at doing this. It also has a relatively simple structure and can avoid overfitting and improve generalization performance, so in this study, we used AE to perform anomaly detection. This is also described in the paper.

Thank you for pointing out the lack of inquiries regarding CNN-LSTM, but since the code and technology were written while performing defense tasks for the Republic of Korea, we are unable to disclose information in that section. Instead, we have added more information to the CNN-LSTM section in Section 2.3.

Reviewer#3, Concern # 8: Please provide the libraries utilised to develop the proposed code (scikit-learn, TensorFlow, etc.). 

Author response and action: Thank you for taking the time to review our paper. We have added the libraries we used.

Reviewer#3, Concern # 9: The models introduced are very generic. Information should be provided in a more specific manner based on the application domain, which relates to cybersecurity in this instance. For example, why CNN-LSTM should be applied in cybersecurity for anomaly detection? Which are the temporal and spatial patterns expected to be retrieved from BGP data?

Author response and action: Thanks for pointing out the shortcomings of my paper.

the analysis of AS path changes in a temporal pattern enables the detection of diverse anomalies. These temporal patterns play a crucial role in recognizing alterations in network conditions and can be effectively employed for the identification of anomalies or potential security breaches. Spatial patterns pertain to the interplay between Autonomous Systems AS, subnet configurations, and network topologies within BGP data. CNN-LSTM is tasked with tracing AS relationships and overseeing shifts in network topologies, thus enhancing its ability to detect anomalies.

This is described in the paper

Reviewer#3, Concern # 10: Figure 2 should be adapted to present the architecture of the proposed AE. 

Author response and action: Thank you for pointing out the shortcomings of our paper. We have revised Figure 3. as you requested

 

Reviewer#3, Concern # 11: Analogously, a diagram with the architecture of the CNN-LSTM should be provided in section 2.3.2. 

Author response and action: We've added a picture. 2 as you requested.

Reviewer#3, Concern # 12: Figure 5 is not relevant, as it presents the same information as Fig. 2. Furthermore, as the authors stated, the output of the AE is the reconstruction of the input data, and not whether the input data is normal or abnormal. Thus, one can say the figure is inaccurate. The reviewer considers either removing the figure from the manuscript or include the method that determines the dynamic threshold to differentiate between normal and abnormal data based on the reconstruction error.  

Author response and action: Thank you for complementing my thesis.

The reconstruction error values are visually shown in Figure 6, and the process was described in detail in the paper.

Reviewer#3, Concern # 13: Based on the preceding point, please provide information about how the anomaly score was obtained and how this has been considered to determine if the data is normal or abnormal. 

Author response and action: First, thank you for your question and we will explain why my paper's explanation was insufficient.

This paper analyzed BGP data from countries that continuously attempt cyber-attacks (North Korea, Russia). In the case of North Korea, their network is largely disconnected from the global network and records of external cyber activity are very limited. we was afraid that mentioning the country directly would cause problems, so we could not indirectly mention the content in the paper. This inevitably made it difficult for you to understand.

When countries with low network activity, such as North Korea, develop a new pattern or suddenly change the route rather than a specific route, the existing autoencoder identifies it as abnormal data because it is not a pattern learned from normal data. This can be confirmed through Figure 6.the paper.

Reviewer#3, Concern # 14: t-SNE and PCA are widely known. Thus, the information included in the experimental section should be summarised. Including a sentence indicating why the authors implemented t-SNE instead of PCA is more than enough.

Author response and action: Thank you for complementing my paper.

As requested, we have briefly summarized the differences between PCA and TSNE and the reasons for using TSNE.

Reviewer#3, Concern # 15: Please indicate why RF, one-SVM, CNN-LSTM, and AE were considered, as there are other state-of-the-art methods that outperform them. 

Author response and action: First, the reason we considered various models such as RF, One-SVM, CNN-LSTM, and AE is because we wanted to find the most effective anomaly detection model by exploring both traditional methods that are widely used to date and the latest deep learning techniques. These models have been applied to a variety of data types and problems and are basically proven techniques.

Second, these traditional models are simple, easy to interpret, and provide a strong foundation for application to a variety of anomaly detection problems.

finally, different state-of-the-art models may not be suitable for all data and problems. In fact, CNN-LSTM is the latest anomaly detection technique often used in image and time series analysis, but it is not suitable for BGP data and has lower anomaly detection ability than autoencoder.

Reviewer#3, Concern # 16: Analogous to the preceding points, the metrics introduced in page 11 are widely known. Thus, this text should not be included in the manuscript. The reason of estimating these metrics in the introduced case study should be given instead. 

Author response and action: Thank you for your good comments.

First, we don't think We can leave out the explanation on page 11. The reason is that people who are encountering anomaly detection experiments for the first time will read my paper. At first, we felt a lot of confusion when we came across a paper that did not explain the auxiliary indicators. To avoid this situation, we recommend that you enter the relevant description as is.

They also asked me to write about the reasons for using the auxiliary indicators, and we wrote that in the paper.

Reviewer#3, Concern # 17: Did the authors implement any type of cross-validation to ensure the generalisation capabilities of the classifiers? If not, please do so.

Author response and action: Thank you for your good question.

Cross-validation is an important technique for evaluating the performance of machine learning models and improving their generalization ability. That is, cross-validation is used to assess how well a model generalizes to unseen data and to alleviate problems such as overfitting.

Precision, recall, F1 score and AUROC curves are used for cross-validation for anomaly detection to rigorously evaluate and fine-tune the model to achieve good performance on different data samples and maintain a balance between accurate anomaly detection and prevention.

This information suggests the generalized classification ability of the model.

Reviewer#3, Concern # 18: In the abstract F1-Score is introduced to highlight the performance of the models. However, the data is clearly imbalanced. Please provide a more adequate metric result.

Author response and action: Thank you for your valuable feedback.

We think the problem may have been caused by insufficient explanation in my paper.

increasing the number of abnormal data to achieve a 5:5 ratio of abnormal to normal data may blur the boundaries between normal and abnormal classes. This can cause problems with models that classify normal data as abnormal.

To prove that the model was not overfitted, we showed F1-score, Precision, Recall, and AUROC Curve, which show generalization ability.

Reviewer#3, Concern # 19: Please provide the confusion matrix for BGP Data and BGP Data (Tokenizer).

Author response and action: Thank you for your good comments.

If the requested confusion matrix is provided in Figure 11, you can see that both show 0.99 accuracy through the results, but anomaly detection is possible with BGP data without tokenizer through F1-score, precision, recall, AUROC curve, etc. This paper states that overfitting occurs when.

Reviewer#3, Concern # 20-1: The authors indicated that SMOTE technique was applied to address the issue of data imbalance. However, the ratio of abnormal data differs significantly from the ratio of normal data, even after of applying SMOTE.

Author response and action: Thank you for your valuable feedback.

We think the problem may have been caused by insufficient explanation in our paper.

First, increasing the number of abnormal data to achieve a 5:5 ratio of abnormal to normal data may blur the boundaries between normal and abnormal classes. This can cause problems with models that classify normal data as abnormal.

In addition, overusing SMOTE can cause the model to learn patterns from the synthetic data instead of capturing patterns in the actual abnormal data, which can degrade the performance of the model.

Due to the above issues, we have adjusted the ratio of normal to abnormal data to 8:2 to mitigate the above issues by properly adjusting the quantity and quality of synthetic data.

Reviewer#3, Concern # 20-2: How SMOTE compares with other data augmentation methods? Please provide a comparative study with baseline models.

Author response and action: Thank you for pointing out the deficiencies in our research.

We believe the issues you mention are due to insufficient explanation.

We have explained in the paper why We chose SMOTE over other sampling techniques.

Undersampling involves removing many classes to achieve proportionality, which can result in the loss of potentially valuable information and patterns.

Resampling increases the amount of data in the minority class to make it similar to the amount of data in the majority class but has the disadvantage that overfitting can occur in the minority class.

ADASYN requires additional resources and careful parameter tuning due to its increased complexity, and has the potential to generate misleading information by oversampling mislabeled data points.

For the above reasons, in this thesis, we use SMOTE

.

Reviewer#3, Concern # 21: Figure 7 and 8 differs conceptually. Are the labels of Figure 7 incorrect? Please revise. 

Author response and action: Thank you for catching my mistake

This problem occurred because We did not read the paper carefully while writing it.

We are very grateful to you for catching our mistake, and after completing the correction, We have re-posted it in the paper. We really appreciate you catching our mistake

Reviewer#3, Concern # 22: The reviewer considers that the statements provided by the authors (e.g., “superiority of our research”) is not objective, as, for instance, one may say that the statement is too optimistic when a comparative study of four widely known classifiers have been provided and the “novelty” of the study is introduced in the experimental section in tandem with the case study and results. Please rephrase such statements in a more objective manner.

Author response and action: Thank you for your valuable feedback.

This problem occurred because We did not read the paper carefully while writing it.

We are very grateful to you for catching our mistake, and after completing the correction, We have re-posted it in the paper. We really appreciate you catching our mistake.

Round 2

Reviewer 2 Report

The revised version has fully addressed my concerns.

Author Response

Author response and action: Thank you for thoroughly reviewing the paper.

Reviewer 3 Report

The reviewer would like to thank the authors for addressing all the concerns. Thus, the reviewer considers the paper can be accepted after the following point is addressed:

 

- Figure 7 is still unclear. Please consider revising. I would suggest utilising the same color for the same concept (yellow for abnormal and green for normal, for instance). The current version of the figure is unclear, as the yellow color is used for both abnormal and normal, which may suggest there is a mistake.

Minor mistakes can be perceived.

Author Response

Review3, Concern # 1:  Figure 7 is still unclear. Please consider revising. I would suggest utilising the same color for the same concept (yellow for abnormal and green for normal, for instance). The current version of the figure is unclear, as the yellow color is used for both abnormal and normal, which may suggest there is a mistake.

Author response and action: Thank you for thoroughly reviewing the paper.

As per your request, the color of Figure 7 has been changed to blue for 'normal data' and orange for 'abnormal data' to clearly highlight the differences.