A Generative Adversarial Network Structure for Learning with Small Numerical Data Sets

Round 1

Reviewer 1 Report

The paper deals with a generation of virtual samples when the original real data is insufficient or small. The method is a combination of two known methods au WGAN and the MTD. The paper is very hard to understand. It could be interesting if we han one example of application instead of giving only tibles and numbers. The flowchart of figure 2, supposed to explain the proposed metod is very hard to understand. Authors proposed a method composed of two known methods but there is no study about the convergence of the whole framework and about its limitation. The point that intrigued me is that there is no comparison either with MTD nor with WGAN allone to evaluate the benifits of the combination. The comparison of the results is done with old methods and sometimes these old methods outperform the proposed method without any explanation in the text.  The references are really old and there is no citing of new litterature on the question (ex. MCF-GAN). Besides, the english is sometimes hard to understand. To summarize, the studied subject is of high interest but the reading of this paper shoul be entirely reconsidered for such a journal. I advice the authors to take more time to explain things and carry out more experiments, It's worth it.

Author Response

The authors thank the reviewer for the fruitful advice and helpful comments. We have prepared the responses and updates as follows.

To more clearly deliver the concept of WGAN_MTD, we have redrawn the flow chart of Figure 2, in the revision.

After generating virtual sample, we applied common learning tools, including SVM, Decision Tree and Naïve Bayes Classifier, to conduct the classification tasks with these generated datasets. We believe that the generated sample could contain sufficient information of its population and be easily processed by common learning tools if the proposed method performs well enough. For a more clear demonstration, we compared the results from WGA_MTD, WGAN, and MTD and the results of the experiments are listed in Table 6. It shows that the learning results based on generated data from integrated WGAN_MTD could be better than those from WGAN and MTD. 

The recommended article has been cited by the revision and numbered [1]. We also think this article did demonstrate an application of GAN in the medical field. Besides, we have considered several more recent articles related to the topics, and list them in References, such as [5], [21] and [22].

 

We conducted more experiments with another dataset, Cervical Cancer, containing healthcare survey data and coming from UCI Machine Learning Repository as well. The experiments results are listed in Table 5.

Despite the theoretical support still taking time to explore, including mathematical proof of framework convergence, a threshold can make the usage of our proposed WGAN_MTD more systematically. Even we can find the critical sample size for deciding to adopt or not, it could probably vary by case. Based on our experience, the dependence of virtual sample generation (VSG) affects the model performance indeed. And VSG is also dependent on the original data collected. As for the VSG dependence, we currently work on this topic in another project and expect acceptable results. For more focusing on the operation and mechanism of WGAN_MTD, we might leave the issue to address in future studies.

 

 

 

Reviewer 2 Report

The paper introduces a study of the state of the art of GANs in general and more specifically of how to approach data processing when applied to databases with few records.

The article includes alternatives that generally appear in the bibliography and that are applied to process empty or null data in the database. It should be more specifically described what to do in the GAN in case null or empty data appears in a database with little data. The author is recommended to read the article https://www.mdpi.com/2079-9292/10/18/2220 and include it, since he proposes a way of working with empty data. Similarly, this article can serve as an example of the use of GAN in cancer or healthcare data.

In the proposal, there is a difference between the way of generating data for databases with many records versus those with few. Authors must indicate when one or the other applies, that is, if there is a threshold beyond which it is better. Try to justify the answer.

The examples that have been put as practical cases, the one with the least number of records is a hundred. It would be interesting to know how robust the proposal is if the number is less than this.

GAN training is quite dependent on the initial randomly generated virtual samples. It is interesting to carry out a study on the dependence of this seed in the generation of data related to the proposal.

Self-citations may be reviewed and reconsidered.

Author Response

The authors thank the reviewer for the fruitful advice and helpful comments. We have prepared the responses as follows.

GAN is capable of mimicking the present data and generates another set of analogous ones, so-called the imitating ability. The basic logic is that a sufficient sample offering enough information about the population can enable GAN's imitating ability to generate another similar one. When facing a little database, with null or empty data, it's not unexpected to see a worse performance from running a GAN. That's why we proposed the WGAN_MTD, tied-in-used of GAN and MTD. MTD aims to exhaust and extract all the underlying information behind the small data, while GAN can adopt the acquired information for better performance of imitating.

The recommended article has been cited by the revision and numbered [5]. We also think this article did demonstrate an application of GAN in the medical field.

Despite the theoretical support still taking time to explore, including mathematical proof, a threshold can make the usage of our proposed WGAN_MTD more systematically. Even we can find the critical sample size for deciding to adopt or not, it could probably vary by case. Based on our experience, the dependence of virtual sample generation (VSG) affects the model performance indeed. And VSG is also dependant on the original data collected. As for the VSG dependence, we currently work on this topic in another project and expect acceptable results. For more focusing on the operation and mechanism of WGAN_MTD, we might leave the issue to address in future studies.

To make the citation and reference list more concise, we have removed several articles, which were cited but not directly related, from the revision.

 

Round 2

Reviewer 1 Report

The paper is well revisited and most of the required information are required. However, as we know that the number of the data is important to feed a WGAN, the augmented data by MTD still not sufficient. It could be interesting to apply the proposed method on medical datasets where the problem of small datasets is an important issue and validate with an extra task like segmentation. The comparison will be fair as there is many attempts to segment for instance medical imaging with GAN augmentation and U-Net network.