A Study on Sample Size Sensitivity of Factory Manufacturing Dataset for CNN-Based Defective Product Classification
Round 1
Reviewer 1 Report
1. Line 46: Correct the sentence. 'distinguishing'
2. There is a lot of theory that is obvious related to machine learning and I think that should be taken off and only the contributions of this paper are to be written in this paper.
3. Looks like the paper is trying to apply the problem to the manufacturing domain, however, there is nothing specific to manufacturing that is relevant to the sample size problem paper is trying to solve. If the paper is generic, please mention so.
4. General typo, and sentence corrections to be looked at.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The revised version could be accepted now.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 3 Report
The paper „A Study on Sample Size Sensitivity of Factory Manufacturing Dataset for CNN-based Defective Product Classification” by Kim et al. is not written according to the standards of scientific articles. First of all, the manuscript presents well-known issues, the article does not contain any scientific novelty. The research methodology has not been adequately described. In addition, the language is incorrect and does not conform to scientific standards. The formatting does not fully comply with the guidelines imposed by the journal.
In conclusion, I do not recommend the article for publication in Computation journal in its current form.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
In this manuscript, the authors attempted to determine sample size required to classify defective products in manufacturing data by evaluating the model performance according to the sample size change. They used a simple Sequential deep learning model and identified how many samples were needed to stabilize the model performance. They compared several statistical indicators like accuracy, recall, precision, and F1-score for different sample sizes and concluded that more that 500 cases were required to achieve stability.
While the research objective is intriguing and worth investigating, the research and results presented were very much inadequate to provide any worthwhile resolution or insight. The dataset and model used are too trivial and specific, while the claims made were rather generalized. The conclusions drawn at the end, such as less samples led to overfitting and accuracy increased with sample size, are pretty much common knowledge; there is no merit in doing additional research to state the obvious! The thresholds measured for minimum sample size for accuracy and overfitting are only applicable to the dataset and model used, so those are of no use for a real-world SME, which was the original objective of this research. Only outcome of this research seems to be, as stated by the authors at the end, "find a criterion of sample size required for the deep learning-based classification of defective products in manufacturing data to apply in practical factories, especially SMEs". Again, their research didn't have any contribution to provide any resolution; it pretty much obvious for both researchers and developers that there exists a minimum sample size for any deep learning model to provide acceptable accuracy.
To summarize, this paper does not provide enough novel research contribution to be considered for a journal publication.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Overall improvement after the revision
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Although the authors put a lot of effort into supplementing the manuscript, which should be appreciated, I do not change my opinion that the work presents known issues, personally I do not see any scientific novelty here. The authors should motivate and define the originality of the presented method more clearly, extending the introduction with a broader reference review.
Figure 3 is not fully clear to the reader, the captions are illegible.
Figs 4 and 5 should have both axes properly signed.
The axes should also be signed in Fig. 2.
In Fig. 1, the defects should be marked with, for example, arrows and the types of defects should be signed.
The conclusions should be a separate, last point of the article.
The language in which the article was written requires correction, and the language typical of scientific publications was not used.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 3
Reviewer 3 Report
Earlier comments were taken into account, the article has been significantly supplemented, in its current form it can be considered for publication.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Regarding the subject of the paper and the solution presented, they are very interesting and topical. There are a few comments that I hope may be helpful to the paper:
1. The topic is interesting, referring to real problems in the field of production, problems faced by companies in fierce competition, in which the problem of quality becomes very important.
2. From the beginning, the abstract and the keywords capture the reader's interest. The introduction provides a database that includes a large part of the relevant references for the researched field. Indicate in a comprehensive way what the paper will consist of. The issue of quality inspection / classification of defective products as part of the manufacturing process is well analyzed in the context of Industry 4.0. The motivation, necessity and opportunity of this study result from the description.
3. The research is well structured, so that the reader can understand the stages of analysis of the studied problems. But it was even better if a separate section was developed, called "Literature Review", in which to insist more on the solutions of other authors in the research field. This would have resulted in an even more relevant comparison with the materials published by other authors.
4. The research methodology is adequately presented and is well applied to achieve the research objectives. It was useful to point out exactly why this methodology was chosen over others.
5. The research results are well presented, the content presented being scientifically relevant. I also note the inclusion of suggestive figures for the analyzes performed.
6. The conclusions are relevant in terms of content. It would be great if they were included in a separate section for conclusions. It would have highlighted much better the contributions and originality of this study.
7. It is very good that the authors have highlighted the limitations of this study, especially regarding the use of methods in SMEs, which face specific problems and often do not have access to such methods.
8. The references are representative, however, as we pointed out in point 2, a more detailed review of these papers was useful.
9. In conclusion, we can appreciate that the paper is original, and through its significant and relevant content, it brings elements of novelty in the field, thus presenting interest for readers and for the studied field.
10. I mention that no significant corrections are needed regarding the English text.
Reviewer 2 Report
The introduction clearly defines the problem. The introduction must intensely discuss the importance of the sample size sensitivity problem, the CNN methods, and the sample size sensitivity problem solutions.
The research methodology is not deep enough. There is no formalization for the datasets and the CNN either. I would like to suggest moving the CNN overview to the introduction and defining the exact algorithms and definition in Section 2 only.
The Results section is also not deep enough.
Reviewer 3 Report
Comments for author File: Comments.pdf
