Ensemble One-Class Support Vector Machine for Sea Surface Target Detection Based on k-Means Clustering
Round 1
Reviewer 1 Report (Previous Reviewer 1)
Comments and Suggestions for AuthorsThe authors have successfully addressed the comments .
Comments on the Quality of English LanguageGive a thorough language check.
Reviewer 2 Report (Previous Reviewer 2)
Comments and Suggestions for AuthorsThis article has been resubmitted, i.e. in a new round. That's why this is the first review in this round, but my third overall, regarding this paper. In the previous one, I suggested that this article might be suitable for acceptance for publication, provided that some editing errors were corrected. The authors fulfilled this task and corrected all the errors and editing glitches that I pointed out.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors have proposed "Ensemble One-class Support Vector Machine for Sea Surface 2 Target Detection Based on K-means Clustering". The results of the proposed approach is promising. However, there substantial factors which the authors must take into consideration for the improvization of this manuscript. The points are highlighted as:
1. Rename Section 1 as "Introduction and Background"
2. The similarity index is too high. The authors need to ensure the incorporation of references at the required places, and reduce the similarity index.
3. In the methodology section, the authors must elaborate more on joint optmization i.e., clustering in the kernel space and constructing hyperplane in each cluster.
4. The authors should include a section on "Dataset Description".
5. Give a thorough english check.
Comments on the Quality of English LanguageGive a thorough english check.
Author Response
We have answered each question in order, please refer to the attachment for specific answers.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper proposes a method for sear surface detection which is based on an ensemble of OC-SVMs. These are obtained by application of the k-means clustering method to split the feature space for consecutive OC-SVM classifiers. To some extend this idea is original but not that much as presented in the paper. This is because the authors did not do their homework of thoroughly reviewing prior work in this area. These are as follows:
Tax, D.M.J.: One-class classification. PhD thesis, TU Delft University (2001)
Tax, D., Duin, R.: Support vector domain description, Pattern Recognition Letters, Vol. 20, pp.1191-1199 (1999)
Tax, D., Duin, R.: Support Vector Data Description, Machine Learning 54, pp.45-66 (2004)
On the other hand, the very similar concept of applying first k-means to the data, then an ensemble of OC-SVM is presented in the paper
One-class support vector ensembles for image segmentation and classification, B. Cyganek, Journal of Mathematical Imaging and Vision, (2012) vol. 42, no. 2-3.
The authors should refer to this. It would be also useful to compare these two approaches. In the presented paper the authors use two sets of kernels – that is, one for kernel k-means, and the second kernel for OC-SVMs. How these two sets interact? Are these necessary? I mean – can these be substituted with one kernel, either for k-means or for OC-SVM?
Next important things that are missing:
- How the value of ‘k’ is chosen?
- How are chosen parameters of the kernels?
The authors just set them in a rather arbitrary fashion. However, in practice this can pose a serious problem.
I have also some doubts about the experiments. The authors compare En-OCSVM with OCSVM (Table 1). However, the more natural for the two class problem would be to use a binary SVM and compare it as well.
Experiments with IPIX dataset, described in Section 4.3, are not clear. Especially the compared to methods should be described, such as PCA – how was it used to obtain 92% accuracy? And so on.
Minor remarks:
Ref. [23] is used in wrong context.
In this context it is not sure what are the derivations in Section 2 taken from – they lack proper citations.
Section 3.1 – needs some editorial polishing.
Algorithm 1 - needs some editorial work, as well since it is not clear.
Section 3.3 – needs some editorial polishing.
Comments on the Quality of English Language
English needs to be significantly improved.
Author Response
We have answered each question in order, please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsI want to thank the authors for the work submitted to this journal, that I have the opportunity to review now. The authors identified a caveat in the use of unsupervised classification systems, and in particular, in the use of SVM when the transformed data cannot be separated linearly in the kernel space. The idea of combining clustering with hyperplane identification is actually quite smart, which in basic terms means finding local solutions for the hyperplane computation instead of aiming for a single but complex-shaped hyperplane for the overall dataset.
The results presented by the authors prove the validity of the approach, and show how the implemented novel method performs overall better than other methodologies.
However, there are several limitations to the work that are worth discussing:
1. Authors propose a Gaussian kernel for their model, and then they tested it with synthetic data generated by a Gaussian Mixture Model (section 4.1). Not surprisingly, their method outperforms OCSVM. This, more than a validation, is a verification or confirmation; logically, their method should work better if you generate a dataset with a proxy of the same kernel you use to classify. In addition to this verification, a proper validation should be carried out with synthetic data generated using different kernels and given to different types of datasets. In general, I miss a discussion on the limitations of the method in itself, which probably is not in the manuscript because of a limited effort has been made in terms of validation and testing. Further work to offer readers an idea on the limitations and range of validity of their proposed method is pertinent.
2. There is no discussion on why using the Gaussian kernel is preferable to other types of kernels, which could be de facto be implemented with the same idea, like sigmoidal or polynomical. This is probably motivated by the later exercise done using the IPIX dataset, which is likely to have Gaussian noise as the main source of error. However, a discussion on why using a Gaussian kernel instead of others could be pertinent.
3. The use of the benchmark datasets (Table 2) looks strange; these datasets have a considerably high dimensionality with respect to the number of points used. It is hard to tell until what point it will be really significant. Additional metrics to these classification exercises should be considered, in order to properly evaluate their range of validity.
4. The results over IPIX dataset ("real fire" exercise) look promising, but improvements vs the KOCSVM method look quite marginal, and within the realm of being inconclusive with the real data exercise. Again, an extended validation with many different datasets is required, which makes sense to properly frame the proposed method in a wider frame for the audience. For instance, a reader could want to know if this method may serve him/her for his/her problem in particular; however, not having tested it against several different datasets of different characteristics and with respect to the set of methodologies proposed as benchmarks is a shortcoming in this regard.
5. Authors do use of many common classification methods, but they do not explore the use of one of the most common tools for this type of cases: extreme gradient boosting. It could be nice to know why.
6. The Conclusions section is really too short, shorter even than the abstract. This section should be further developed including:
- Summary of main results.
- Strong points and weaknesses of the proposed method.
- Range of applications to which the method is worthwhile (and which ones to avoid)
- Future work, including things that shall be improved.
In addition to these points, there are some other issues with the manuscript:
1. Formatting errors: I suspect the authors used Latex to write the manuscript. There are however, several strange behaviours of math and tables, like, for instance, the alpha parameter in line 106 and the text overflow in the "Algorithm Description" (page 5). It could be a good idea to proofread the manuscript before submitting it, especially if using Latex, which is prone to issues of this type.
2. Mistypes: All figures refer to "demension1" and "demension2". I guess author refers to "dimension" instead. It could also be good to use something nicer for labelling the axis. Similarly, the Figure 8 x-axis label should be rewritten simply (maybe by simply calling it n. clusters and then adding to what they refer to in the caption).
4. Tables: Please, make sure the tables do not extend from one page to another. It makes it difficult to evaluate them.
5. Tables 3, 5 and 6 have the same list of methods, but a) they are not always referred to by the same name (please correct it), and b) they are also presented in a different order. It would be much better to put them always in the same order, and if not, explain if there is any logic to such an order so the reader can follow.
6. The Data description section is missing: While authors describe very briefly the data they use throughout the text, it would be much better to provide a self-contained section for that. The section should be very clear and provide readers with the following information:
- Description of the various datasets and their purpose in the study.
- Sources (URL, repositories...) from where they can be obtained, so readers could try to replicate their work.
- Any type of pre-processing they may have applied to them.
7. Missing code: This is a methodological paper where the authors propose a novel method to the community. Strangely, they do not provide the code. Typically, one wants to ensure that both the code and the data used are available fr independent testing and replication of the work. I believe the manuscript should be compliant with such good practices (e.g., available through a GitHub repository).
As a final comment, I want to insist on the importance of facilitating to the readers access to the code and datasets used, especially in a methodological paper as this one. Any new method must be independently confirmed by others before it can really be taken up by the community. To that end, the provision of code and data is essential.
I want to thank the authors again for their submission and their contribution to further improving tools of relevance.
Comments on the Quality of English LanguageWhilst the abstract is very well written, the main text lacks of English skills. In particular, there is a lack of use of both articles and prepositions that makes the text look "funny" when read. There are also some errors when using the right verbal tense (is vs. are, for example), and, sometimes there is also incoherence in the use of present and past tenses. I recommend a revision of English by a more experienced writer, a native person,
Author Response
We have answered each question in order, please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsThe authors did a lot of work and significantly improved and expanded their article. I think that now it is a valuable work that, once published, will be useful to a significant group of researchers. But for this to happen, it needs to be improved a bit because there are still places that are simply illegible due to poor editing. This is mainly about the layout of algorithm 2. It must be improved. Also, please better indicate that the support vectors are just selected points from the set of x_i. There is also a minor typo in line 164 on pg. 5 – should be “… as Eq. (8)”, or just “… as follows”.
Comments on the Quality of English LanguageThe paper should be precisely checked to improve its editorial aspects.
