Fabric Defect Detection System Using Stacked Convolutional Denoising Auto-Encoders Trained with Synthetic Defect Data

Round 1

Reviewer 1 Report

The manuscript “Fabric Defect Detection System Using Stacked Convolutional Denoising Auto-Encoders Trained with Synthetic Defect Data» by Young-Joo Han and Ha-Jin Yu, presents a new approach for identifying defects on fabrics using unsupervised learning techniques without having actual defect data. The authors propose a method that generates artificial defect data using the types and characteristics of defects known by experienced workers. Then, they use the generated data to train a set of stacked convolutional autoencoders and identify the defects. This work is motivated by the fact that the amount of the available labeled actual defect data is small and time-consuming to obtain and thus they are not enough to train any advanced network using supervised learning.

The authors use a clear and straight forward method to generate artificial data to train their model using unsupervised learning methods. Using both artificial training data and unsupervised learning, they address both of the problems one faces in such kind of problems, i.e. not having enough training data and consuming a lot of human time to label them. In my understanding, the authors use the well know performance of autoencoders in removing noise from images and take it one step ahead to identify the “noise” in the images as a defect.

The weak point of this manuscript is that it needs moderated editing of the English language. The authors should read again their manuscript and fix several language mistakes to improve the reading flow of it.

In general, the methodology is clear, and I think it can constitute a step ahead in the corresponding field.

For these reasons, I strongly support the publication of the manuscript.

General comments:

1) Could the authors suggest any other applications that their method would be useful?

2) Could the authors make an additional comment on how they believe they will address the problem of low recall and precision when the difference between the input image and the restored image is very small?

3) I would suggest the use of "supervised/unsupervised learning" instead of "supervised/unsupervised method".

4) Number "3" in-line 50 should be in bold.

5) In the paragraph starting in-line 53 the authors use Latin numbers for describing the Sections but in the text, they use Arabic numbering for each Section.

6) In-line 115, the authors state that: "However, in our system, real-world-shaped defects are unnecessary. Our system needs only simple form of defects." This statement sounds like a limitation of their method. It gives the impression that, if the defect is a complicated one like the real-worlds-shaped defects, their method will not work well.

7) In-line 129. In Section 3.3 "Pre-processing Unit" it seems that the authors use the standard-scaler method and they prepare their data to have zero mean value and standard deviation equal to one. I would suggest them to clarify this.

8) In-line 175. I would suggest the use of the word "real" after "images of" to emphasize that the tests were performed in real defect-data.

9) I would suggest the authors of moving the Figures closer to the references of them in text.

10) Table 1. It can be compressed! I would suggest adding one row after the "Recall        Precision           F1 Score" line and for each one of the three metrics to create two columns "Our Work, U-Net [20]" and put their results and the results using U-Net.

11) Figure 5. Is it the "Dataset 1/2/3" or "Sample image in Dataset 1/2/3"? Also, they should label the columns as (a), (b), (c) and (d).

Author Response

Response to Reviewer 1 Comments

Point 1) Could the authors suggest any other applications that their method would be useful?

Author response:

According to the reviewer's comment, we revised the conclusion by adding the discussion on the topic.

“This method can be applied to many industrial and medical applications such as cancer detection using the knowledge of experienced doctors.”

(page 8, line 218)

Point 2) Could the authors make an additional comment on how they believe they will address the problem of low recall and precision when the difference between the input image and the restored image is very small?

Author response:

According to the reviewer's comment, we revised the conclusion by adding the discussion on the topic.

“We can also improve the performance by solving the problem of low recall and precision when the difference between the input image and the restored image is very small. As one of the solutions, we may set a threshold in the tolerance according to the applications such that we can decide the limit of small defects that is allowed in a specific application.”

(page 9, line 228)

Point 3) I would suggest the use of "supervised/unsupervised learning" instead of "supervised/unsupervised method".

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the manuscript.

(page 1-2)

Point 4) Number "3" in-line 50 should be in bold.

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the manuscript.

(page 2, line 49)

Point 5) In the paragraph starting in-line 53 the authors use Latin numbers for describing the Sections but in the text, they use Arabic numbering for each Section.

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the manuscript.

(page 2, line 53)

Point 6) In-line 115, the authors state that: "However, in our system, real-world-shaped defects are unnecessary. Our system needs only simple form of defects." This statement sounds like a limitation of their method. It gives the impression that, if the defect is a complicated one like the real-worlds-shaped defects, their method will not work well.

Author response:

According to the reviewer's comment, we modified the manuscript:

“The human experts have knowledge of the various defects that occur frequently and also the defects that are very rare or hard to detect but critical for the quality of the products.”

(page 3, line 114)

Point 7) In-line 129. In Section 3.3 "Pre-processing Unit" it seems that the authors use the standard-scaler method and they prepare their data to have zero mean value and standard deviation equal to one. I would suggest them to clarify this.

Author response:

According to the reviewer's comment, we modified the manuscript:

“The pre-processing unit normalizes the images to have zero mean value and standard deviation equal to one.”

(page 4, line 130)

Point 8) In-line 175. I would suggest the use of the word "real" after "images of" to emphasize that the tests were performed in real defect-data.

Author response:

According to the reviewer's comment, we modified the manuscript.

(page 5, line 176)

Point 9) I would suggest the authors of moving the Figures closer to the references of them in text.

Author response:

According to the reviewer's comment, we modified the manuscript.

Point 10) Table 1. It can be compressed! I would suggest adding one row after the "Recall        Precision           F1 Score" line and for each one of the three metrics to create two columns "Our Work, U-Net [20]" and put their results and the results using U-Net.

Author response:

We modified Table 1 according to the reviewer's comment.

(page 8)

Point 11) Figure 5. Is it the "Dataset 1/2/3" or "Sample image in Dataset 1/2/3"? Also, they should label the columns as (a), (b), (c) and (d).

Author response:

We modified Figure 5 according to the reviewer's comment.

Reviewer 2 Report

The manuscript describes a machine learning approach to detect fabric defects. The approach is based on the use of stacked convolutional denoising autoencoders. Training data are obtained by using actual fabric images, which contain few or no defects, enriched with synthetic defect images, obtained from expert workers and possibly replicated on several fabrics.
Results on real-world data are compared to those of a published supervised learning method (U-net).
The unsupervised approach proposed in this work appears comparable (although slightly inferior) to the reference method and exhibits some practical advantages.

In my opinion, the manuscript is interesting, sound and reasonable. The presentation is quite clear. The English language is fair.
I have the following suggestions for the authors.

1) The use of human expertise to obtain scarce data (the defective fabrics in this case) may be two-fold: one the one hand, it gives the advantage of integrating the available data using reduced effort; on the other hand, relying on such human integration makes the approach less robust if the human experts are not able to do a complete job. For example, they may describe very well several types of defects, but if they forget to mention one, the system will be unable to deal with that one... Some discussion on this topic should be added to the manuscript.

2) In the Introduction, the authors describe two main paradigms of machine learning, namely unsupervised learning and supervised learning. They classify their approach as unsupervised, as opposed to several other supervised approaches.
However, there is a third paradigm which is recently receiving much attention: semisupervised learning. It is based on the use of a large amount of unlabeled data but also employs a small amount of labeled data which may be available, see for instance:
Bruni, Bianchi, Effective Classification using a Small Training Set based on Discretization and Statistical Analysis, IEEE Transactions on Knowledge and Data Engineering 27, 2015.
In the case of the synthetic fabric defects, the label of these data is clearly known. Does the proposed approach use this knowledge in some way, for example during the training phase? In this case, it should be more precisely described as a semisupervised learning approach. Or, if it doesn't do so yet, it could take into account the label of the labeled training data to increase the performance.

3) The English language is fair but there are some imprecisions on the choice of the words, for example:
Abstract: "to compensate for this" should be more precisely "to overcome this problem".
Page 2 line 1 "several-fold": actually, point 2 and 3 of the contributions are equivalent, so I suggest to use only 2 points and rephrase the sentence accordingly. Also, in point 1, "for environments that are hard to obtain" should be "for environments where it is hard to obtain".
There are many other cases, I recommend to check again the whole manuscript for similar problems.

4) In section 4.1, the authors say that they capture 3 types of fabrics. If those are the 3 datasets of Table 1, please specify so. Otherwise, it is not clear what are the datasets 1, 2 and 3 of Table 1.

5) Page 7 after the definition of F1 score: "the portion of the pixels that ..." should probably be "the portion of the cases that ...", and the same applies to the subsequent sentences.

6) Conclusions, line -7: "To verify the performance of our method, we show that We compared" should be "To verify the performance of our method, we compare".

Author Response

Response to Reviewer 2 Comments

Point 1) The use of human expertise to obtain scarce data (the defective fabrics in this case) may be two-fold: one the one hand, it gives the advantage of integrating the available data using reduced effort; on the other hand, relying on such human integration makes the approach less robust if the human experts are not able to do a complete job. For example, they may describe very well several types of defects, but if they forget to mention one, the system will be unable to deal with that one... Some discussion on this topic should be added to the manuscript.

Author response:

We revised the conclusion by adding the discussion on the topic.

“The system can be improved by iteration of the whole process. If the human expert forgets to mention a type of defect and the system makes some errors, the expert can add the type and some related types.”

(page 8, line 223)

Point 2) In the Introduction, the authors describe two main paradigms of machine learning, namely unsupervised learning and supervised learning. They classify their approach as unsupervised, as opposed to several other supervised approaches.

However, there is a third paradigm which is recently receiving much attention: semisupervised learning. It is based on the use of a large amount of unlabeled data but also employs a small amount of labeled data which may be available, see for instance:

Bruni, Bianchi, Effective Classification using a Small Training Set based on Discretization and Statistical Analysis, IEEE Transactions on Knowledge and Data Engineering 27, 2015.

In the case of the synthetic fabric defects, the label of these data is clearly known. Does the proposed approach use this knowledge in some way, for example during the training phase? In this case, it should be more precisely described as a semisupervised learning approach. Or, if it doesn't do so yet, it could take into account the label of the labeled training data to increase the performance.

Author response:

Thank you for the comment. As the reviewer pointed out, the proposed approach use the knowledge in training, therefore we can say that it is a supervised method, and we modified the manuscript accordingly. Moreover, he semisupervised learning can further improve the system, but since we have only a short period of time for the revision, the recommended approach can be tried in the future works. We revised the conclusion by adding the discussion on the topic.

(page 9, line 232)

Point 3) The English language is fair but there are some imprecisions on the choice of the words, for example:

Abstract: "to compensate for this" should be more precisely "to overcome this problem".

Page 2 line 1 "several-fold": actually, point 2 and 3 of the contributions are equivalent, so I suggest to use only 2 points and rephrase the sentence accordingly. Also, in point 1, "for environments that are hard to obtain" should be "for environments where it is hard to obtain".

There are many other cases, I recommend to check again the whole manuscript for similar problems.

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the abstract and the introduction section. The contributions are merged into two points. This paper is being checked currently by a professional English editing service. Since we are not allowed to have enough time for revision, we submit the manuscript modified according to the reviewers’ comments, while it is being checked for the language.

(page 2, line 46)

Point 4) In section 4.1, the authors say that they capture 3 types of fabrics. If those are the 3 datasets of Table 1, please specify so. Otherwise, it is not clear what are the datasets 1, 2 and 3 of Table 1.

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the section 4.1.

(page 5, line 173)

Point 5) Page 7 after the definition of F1 score: "the portion of the pixels that ..." should probably be "the portion of the cases that ...", and the same applies to the subsequent sentences.

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the manuscript.

(page 7)

Point 6) Conclusions, line -7: "To verify the performance of our method, we show that We compared" should be "To verify the performance of our method, we compare".

Author response:

Thank you for the comment. According to the reviewer's comment, we modified the manuscript.

(page 8, line 215)