Anomaly Detection and Classification in Predictive Maintenance Tasks with Zero Initial Training
Round 1
Reviewer 1 Report
The authors propose a framework for detecting anomalies in the industry domain. The paper is well presented, and the ideas are well elucidated. There are a number of grammatical issues throughout the paper, and I would encourage the authors to thoroughly proofread the paper.
- The framework and methodology described at the start of the paper refers to the system employing an anomaly detector and a classifier. In literature, since anomaly detectors and one-class classifiers tend to refer to the same class of algorithms, it implicitly implies that the classifier in the proposed framework is not an anomaly detector (or a one-class classifier) but a binary/multi-class classifier. However, upon reading section 3.3, it becomes apparent that the classifiers are also in fact one-class classifiers. Indeed, all the models used are one-class SVMs. I would suggest that the authors make this clear at the start, otherwise it can be a bit misleading to the reader, as one can erroneously assume that the classifiers are binary/multi-class classifiers. For example, the authors can make it clear what the type of classifiers are and their role in the introduction itself.
- One-class SVMs are heavily parameterized algorithms. Depending on the number of unique anomalies that will be found, the framework will end up with a multitude of OCSVMs, each with a unique set of hyperparameters...this could end up being time consuming and costly, trying to find the right set of hyperparameters. How hard was this process during the evaluation?
- Did the authors consider using multi-class classifiers instead? For instance, once a sufficient number of issues have been collected, a MCC could be trained over them. So instead of having n OCSVMs, you could have just a single MCC, and this would also significantly reduce the amount of hyperparameters that would need to be set. What was the motivation for just using only one-class classifiers for the entire framework?
- How much data (the value of K in section 3.3) per issue was needed during the evaluation process? It would be useful to have these numbers, as that would illustrate the practicality of the framework with respect to the amount of data needed to train each OCSVM for each issue.
- Figure 2, in the caption the base state is referred to as Sb, however in the text and graphics, it is Ss.
- In page, 7, the text "Whenever an anomaly is detected by A, we enter the Sa state depicted in Figure 3" appears to refer to an incorrect Figure; I believe the correct figure is Figure 2.
Author Response
We thank the reviewer for the positive evaluation of our work, and for the detailed comments which helped us to improve the paper.
We have gone through the paper and made the necessary grammatical edits.
Response to Issue 1:
We agree with the reviewer that there may be a misunderstanding regarding the specific model used. Both the anomaly detection and the classification use one class SVM models as the reviewer pointed out. Therefore, we have added an appropriate sentence in introduction to clarify this aspect.
Response to Issue 2:
The reviewer is right, since One Class SVM can be heavily parametrized. With respect to our work, however, it was not hard to find a suitable configuration, mainly because the set of anomalies to be recognized is not large, and also because the signals sensed for any issue present clear differences with respect to the steady case. However, we agree that a discussion was missing, therefore we have added it in Section 3.3 to clarify this aspect.
Response to Issue 3:
We have considered multi-class classifiers, but we ended up using only one-class SVM due to their simplicity, and due to the fact that they suit our problem well. Moreover, issues may arise in different parts of a machine, therefore with several models we have also the possibility to distribute them on different computational modules, and only consider those which pertain to a specific part of the industrial machine. We note however that for our planned future works we will certainly leverage on multi-class classifiers. We have added this in the Conclusion section.
Response to Issue 4:
The value of K depends on the time needed by the operator to causalize the anomaly. Let t0 be the time at which our system starts to experience the anomaly, and t1 the time in which the operator causalizes it. Then t1-t0 is the time in which the anomaly was experienced on the industrial machine, hence the data can be used to train the model. In our scenario, t1-t0 was in the [30:90] range, depending on how fast the operator was to causalize the anomaly experience. We have added this discussion in the appropriate section.
Response to Issue 5:
We thank the reviewer for spotting this, and we have corrected the issue in the updated version of the manuscript.
Response to Issue 6:
The reviewer is right, we have corrected the typo.
Reviewer 2 Report
This paper considers the problem of the Anomaly Detection and Classification in Predictive Maintenance tasks with Zero Initial Training. This problem is very investigated in the literature and the authors risk to propose yet another paper about this topic.
This risk is reinforced by considering that the related literature and, especially, the references are quite poor.
Therefore, in my opinion, the authors should enhance the innovativity of their approach by adding a section "Discussion" and telling how their approach behaves with more innovative IoT scenario. In particular, they should consider at least two scenarios, namely SIoT and MIoT.
Author Response
We wish to thank the reviewer for the valuable comments provided, which gives us the opportunity to improve our manuscript.
Related work:
We have improved the related work section, by adding relevant papers from literature on the topic we studied. We have kept the same structure as before, because the scenario considered has different challenges, and we have improved the description of the contributions from literature. We have also added other models and contributions with respect to the data used.
Discussion:
We agree that the technical novelty of our contribution may not be clearly stated, and we thank the reviewer for raising this point which gives us the possibility to improve the paper and clearly state the improvements of our platform. Therefore, we have changed the last section of the related work section into "Challenges and Discussion", and we have stated there the novelties and the improvements of our solution compared to the state of the art. In general, our solution tackles three main challenges, which are the heterogeneity of the scenario, the data availability, and the causal effect of components. We have described these challenges in detail, and explained how our platform addresses them. We believe that now the discussion is much more complete.
Round 2
Reviewer 2 Report
Since the novelty of the issue address by this paper is low I had asked the authors only ONE THING, i.e., adding a new section, called Discussion, where they would compare their approach with some most recent advanced IoT architectures, i.e. SIoT (Social Internet of Things) and MIoT (Mutli-IoT). Unfortunately, the authors did not perform any effort to consider these two paradigms and to say how their approach could be applied to them. They performed some light changes without even adding a section discussion. This is not serious and it is against the spirit of research and the peer review approach.
When a reviewer asks me for something, I do everything I can to satisfy her/him, and usually the reviewer compliments me on how I made an effort to satisfy them. Here unfortunately we are at the reverse way of proceeding.
The authors have added a few lines without taking into consideration the ideas given by the reviewer. Any other reviewer would have proposed a strong reject in the face of such a review.
However, I would like to give the authors an appeal and one LAST chance. Therefore my suggestion, if possible, is to make a serious major revision with the addition of a whole Discussion section where the authors say how their approach could be integrated with the latest ones in the IoT context, primarily SIoT and MIoT but if possible also others. To be clear, I expect at least one page of discussion on this topic with the consideration of several papers on SIoT and MIoT that have been published in the recent literature.
Author Response
We apologize with the anonymous reviewer, as we misunderstood the correct way of answering the comment in the first round of revision. In this updated version of the manuscript, we have removed the "Discussion" from the "Challenges" section, and we have written a completely new section just before the "Conclusion", named "Discussion", in which we discuss three popular IoT scenarios and how our system can be adapted and integrated into them. We now believe that the "Discussion" section better complements the paper, and that the comparison and details with different IoT scenario provide a better overview on the future of our system.
Round 3
Reviewer 2 Report
In mylast review, I asked the authors to add a Discussion section where they could compare their approach with the most innovative IoT architectures, and I had pointed to SIoT and MIoT (Multi-IoT or Multiple IoT) as examples. The authors added this section but they did everything in a hurry and, in doing so, they confused MIoT meant as (Multi-IoT), which is a generic approach directly derived from SIoT, with MIoT meant as Medical IoT, which has absolutely nothing to do with their approach and with what I requested. Therefore, I ask for the last time to the authors to do a serious review (this is the third time I ask for it) comparing their approach with MIoT intended as a general architecture derived from SIoT. Some authors who have discussed MIoT intended as Multi-IoT are Fortino, Terracina, Virgili, Baldassarre and various others. Please consider that this is the last time I am willing to ask for major revision on this paper. If the authors do not operate in a serious manner this time, I will ask for Strong Reject at the next revision.
Author Response
In our first review we were thinking that MIoT was referring to Medical IoT, and we weren't sure about the potential match about our work and Medical IoT. Now that we have understood that MIoT was intended for Multi IoT it makes much more sense, therefore we have updated our manuscript following this advice. We thank the reviewer for giving us this opportunity.
