Classification of the Complex Agricultural Planting Structure with a Semi-Supervised Extreme Learning Machine Framework
Round 1
Reviewer 1 Report
Authors have paid much attention to address the comments raised by this reviewer and the editor, thus the revised version has been significantly improved, however, there are some minor comments that can also be incorporated as the authors claims that their method is iterative thus serval active/self/interactive learning frameworks has been proposed in the literature which I believe deserve to be discussed either way in SOA section or results and discussion. These methods may include "https://www.mdpi.com/2072-4292/11/9/1136", "https://ieeexplore.ieee.org/document/8075429", "https://www.mdpi.com/2072-4292/10/7/1070", "https://docs.lib.purdue.edu/dissertations/AAI3507324/", "https://www.tandfonline.com/doi/abs/10.1080/01431161.2016.1277042", etc. There are many other well-known methods that can also help to make an argument strong. Good Luck to the authors.
Author Response
Comments of Reviewer#1:
Comments and Suggestions for Authors
Authors have paid much attention to address the comments raised by this reviewer and the editor, thus the revised version has been significantly improved, however, there are some minor comments that can also be incorporated as the authors claims that their method is iterative thus serval active/self/interactive learning frameworks has been proposed in the literature which I believe deserve to be discussed either way in SOA section or results and discussion. These methods may include "https://www.mdpi.com/2072-4292/11/9/1136", "https://ieeexplore.ieee.org/document/8075429", "https://www.mdpi.com/2072-4292/10/7/1070", "https://docs.lib.purdue.edu/dissertations/AAI3507324/", "https://www.tandfonline.com/doi/abs/10.1080/01431161.2016.1277042", etc. There are many other well-known methods that can also help to make an argument strong. Good Luck to the authors.
Response: Thanks for your comments. We added a paragraph at the last of Results and Discussion section, that is: “It should be mentioned that the SS-ELM method performs classification iteratively. The iterative procedure is also adopted by the active learning method [48-51]. Both the active learning and the semi-supervised learning methods use the unlabeled and labeled data to improve their learning ability. The main idea of the active learning is different from that of the semi-supervised learning. The active learning method needs an external entity to annotate the request, whereas the semi-supervised learning method does not need manual intervention. The active learning method shows good performance on hyperspectral image classification based on neural network, graph, spatial prior fuzziness pool or 3d-gabor. The limitation of this method is that it cannot establish a reasonable network, graph, fuzziness pool or effective features for the classification of low-resolution images. The SS-ELM algorithm combines two robust classifiers, i.e. SVM and ELM, and shows excellent stability for the cases with low resolution images. Moreover, the classified results of using the SVM and ELM classifiers are even more reliable than the manually classified results. This advantage is particularly prominent in the classification of the complex agricultural planting structure.”
Sincerely yours
Guanhua Huang
On behalf all authors
Author Response File: Author Response.zip
Reviewer 2 Report
The paper presents a method for crop classification based on AI. Although the work as a whole presents good results, it requires a significant amount of rewriting. This is especially important in the introduction. Liaison-emphasis expressions are abused (However, In adition, In contras, Thus, Generally...). Check the whole text and ask yourself when it needs to be used one by one. Some sentences are too long and confusing. Some examples are:
"it can achieve an effective inversion of surface coverage to some extent" Use pasive form
evapotranspiration: What is it? It is a very complex term that is not explained and is not general culture in the context of work. Also, the relevance of this problem is not explained
"evapotranspiration. The information is significant" Which information? Rewrite
"Recently, the issue of land-use scene classification". "the issue of" can be delated
"To our knowledge, satellite remote sensing". Delete "To our knowledge"
"the classification algorithm is a significant information acquisition technique for hyperspectral imagery, which focuses on distinguishing physical objects and classifying each pixel into a unique label."I am not sure that this definition is correct, the pixel by pixel classification is known as semantic segmentation
"In recent years, to improve the classification efficiency, machine-learning-based methods have been employed for classifying remote sensing images" This was explainde in the previous paragraph
"and cropland. And all these" Delete "And"
Also:
In this type of work, it is better to save words like validation for training, validation and testing sets and replace it with a synonym like evaluation
The number of samples per class has not been counted to see the balancing, as well as a matrix of confusion of some of the results of its algorithm
", it always occurs that some unknown crops cannot be identified from other crops" This sentence needs more explanation or an example
Author Response
Comments of Reviewer#2:
Comments and Suggestions for Authors
The paper presents a method for crop classification based on AI. Although the work as a whole presents good results, it requires a significant amount of rewriting. This is especially important in the introduction. Liaison-emphasis expressions are abused (However, In addition, In contrast, Thus, Generally...). Check the whole text and ask yourself when it needs to be used one by one. Some sentences are too long and confusing. Some examples are:
"it can achieve an effective inversion of surface coverage to some extent" Use pasive form
Response: Thanks for your suggestion. We have changed this sentence to: “an effective inversion of surface coverage can be achieved to some extent”.
evapotranspiration: What is it? It is a very complex term that is not explained and is not general culture in the context of work. Also, the relevance of this problem is not explained
"evapotranspiration. The information is significant" Which information? Rewrite
Response: Thanks for your suggestion. We replaced “and estimating regional evapotranspiration. The information” with “. All these data”
"Recently, the issue of land-use scene classification". "the issue of" can be delated
Response: Thanks for your suggestion. We have delated it.
"To our knowledge, satellite remote sensing". Delete "To our knowledge"
Response: Thanks for your suggestion. We have deleted it.
"the classification algorithm is a significant information acquisition technique for hyperspectral imagery, which focuses on distinguishing physical objects and classifying each pixel into a unique label."I am not sure that this definition is correct, the pixel by pixel classification is known as semantic segmentation
Response: Thanks for your suggestion. We have deleted “ and classifying each pixel into a unique label.”
"In recent years, to improve the classification efficiency, machine-learning-based methods have been employed for classifying remote sensing images" This was explained in the previous paragraph
Response: Thanks for your suggestion. We have deleted this sentence.
"and cropland. And all these" Delete "And"
Response: Thanks for your suggestion. We have deleted it.
Also:
In this type of work, it is better to save words like validation for training, validation and testing sets and replace it with a synonym like evaluation
Response: Thanks for your suggestion. We have replaced “validation” with “evaluation” in the revised MS.
The number of samples per class has not been counted to see the balancing, as well as a matrix of confusion of some of the results of its algorithm
Response: Thanks for your suggestion. This article did not consider weighting the samples, the reason is that the sample imbalance problem is not obvious.
", it always occurs that some unknown crops cannot be identified from other crops" This sentence needs more explanation or an example
Response: Thanks for your suggestion. We have rewritten this sentence as: “it always occurs that crops with small growing area, e.g. vegetables and oilseed crops in HID, cannot be identified from those crops with large growing area, e.g. wheat, maize and sunflowers in HID”.
Sincerely yours
Guanhua Huang
On behalf all authors
Author Response File: 
Author Response.pdf
Round 2
Reviewer 2 Report
Responses by authors were clear and complete. I recommend the acceptance
Author Response
Dear reviewer，
Thank you for your great help. Now the manuscript was edited by a native speaker. Enclosed please find the attached compressed file of the marked version and the clean version.
Best,
Huang
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Dear Authors,
You significantly improved the manuscript and responded to most of the suggestions. However, there are minor issues that need to be clarified and corrected.
Some more specific comments in the following:
- Introduction
In the first ”round” of the review process, it was suggested “… it would be more appropriate that the authors describe the problem clearly and give some existing performance evaluation methods or results.”
Authors added two references and, in lines 75-79, stated “However, it is unlikely for the SS-ELM with the graph Laplacian to construct a reasonable graph due to the low resolution of the remote sensing images and the amount of huge data in large-scale agricultural planting areas. Thus, a new semi-supervised learning algorithm based ELM and co-training will be designed and applied to the classification of agricultural planting structure.” What about other presented research on the application of semi-supervised learning in the field of land cover/land use classification? Application of various semi-supervised learning is widely used and very popular for land cover/land use classification in last years it should also be mentioned and discussed.
- Results and Discussion
In the first ”round” of the review process, it was suggested “… The authors should clearly explain which methodology they exactly used to obtain the classification of agricultural planting structure in HID, and make a connection with the previously presented experiments.” The authors added several sentences to clarify this issue more closely. However, the authors presented 6 experiments, which differ in number, or percentage, of labeled samples. How many labeled samples the authors used to obtain the classification of agricultural planting structure in HID?
Considering that the authors lead the discussion of the results in % (Lines 34, 398-399, …), in Table 5 percent of maize, wheat, and sunflower, should be listed, as well. Maybe in parentheses next to the area value.
A few minor issues:
- Lines 248 and 249: Correct “1.6m x 1.6m” to 1.6 m x 1.6 m.
Once again, I hope the authors find this review helpful, as it is meant to be.
Author Response
We have carefully considered all the comments made by the editors and reviewers and incorporated them into our revised manuscript titled “A Semi-supervised Framework for Classification of the Complex Agricultural Planting Structure”. We sincerely appreciate three anonymous reviewers and the editor for their constructive comments which surely make this manuscript stronger than before. The following are the point-to-point replies to all the comments.
Comments of Reviewer #1:
Comments and Suggestions for Authors
You significantly improved the manuscript and responded to most of the suggestions. However, there are minor issues that need to be clarified and corrected.
Some more specific comments in the following:
- Introduction
In the first ”round” of the review process, it was suggested “… it would be more appropriate that the authors describe the problem clearly and give some existing performance evaluation methods or results.”
Authors added two references and, in lines 75-79, stated “However, it is unlikely for the SS-ELM with the graph Laplacian to construct a reasonable graph due to the low resolution of the remote sensing images and the amount of huge data in large-scale agricultural planting areas. Thus, a new semi-supervised learning algorithm based ELM and co-training will be designed and applied to the classification of agricultural planting structure.” What about other presented research on the application of semi-supervised learning in the field of land cover/land use classification? Application of various semi-supervised learning is widely used and very popular for land cover/land use classification in last years it should also be mentioned and discussed.
Response: Thanks for your suggestion. We added the sentences, see line 91-95: In practical applications, the SSL algorithm has been used in the land cover or land use classification. One of its applications is the self-training scheme which incorporates the SVM classifier and the minimum spanning tree (MST) based graph clustering. Compared with the SVM and MST, the integrated SSL classification has less generalization error [22].
- Results and Discussion
(1) In the first ”round” of the review process, it was suggested “… The authors should clearly explain which methodology they exactly used to obtain the classification of agricultural planting structure in HID, and make a connection with the previously presented experiments.” The authors added several sentences to clarify this issue more closely. However, the authors presented 6 experiments, which differ in number, or percentage, of labeled samples. How many labeled samples the authors used to obtain the classification of agricultural planting structure in HID?
Response: Thanks for your suggestion. We added the description, see line 259-261: In the classification of HID planting structure, the number of the manually labeled samples for each category is 20 for each category, and the number of the unlabeled set is 15,000.
(2) Considering that the authors lead the discussion of the results in % (Lines 34, 398-399, …), in Table 5 percent of maize, wheat, and sunflower, should be listed, as well. Maybe in parentheses next to the area value.
Response: Thanks for your suggestion. We have modified Table 5 as follows：
Table 5. Comparison of the area estimated by remote sensing and the statistical data for maize, wheat and sunflower in different years
|
No. |
Year |
Area estimated by remote sensing (ha) |
|
Statistical area (ha) |
||||
|
Maize |
Wheat |
Sunflower |
|
Maize |
Wheat |
Sunflower |
||
|
1 |
1986 |
36,276 (14.51%) |
151,781 (60.73%) |
61,836 (24.74%) |
|
30,646 |
168,120 |
51,226 |
|
2 |
1990 |
42,272 (11.90%) |
250,158 (70.47%) |
62,539 (17.61%) |
|
40,746 |
228,960 |
69,846 |
|
3 |
1995 |
52,674 (12.02%) |
300,400 (68.46%) |
85,025 (19.40%) |
|
45,553 |
246,206 |
82,806 |
|
4 |
2000 |
53,328 (13.67%) |
209,402 (53.70%) |
127,187 (32.61%) |
|
59,024 |
202,625 |
146,024 |
|
5 |
2005 |
89,593 (22.99%) |
155,997 (40.03%) |
144,051 (36.97%) |
|
77,480 |
173,027 |
143,188 |
|
6 |
2010 |
106,322 (27.87%) |
105,403 (27.63%) |
169,700 (44.49%) |
|
99,754 |
84,384 |
210,620 |
(3) A few minor issues, Lines 248 and 249: Correct “1.6m x 1.6m” to 1.6 m x 1.6 m.
Response: Thanks for your suggestion. We did the change.
Sincerely yours
Guanhua Huang
On behalf all authors
Reviewer 2 Report
Dear Authors, thank you for addressing the comments. However there many things that need to be carefully revised. For instance; The writing of the paper needs to be greatly improved, e.g., line # 15-17, "low computational efficiency" (authors means performance in terms of accuracy?), "Moreover, a large number and good of the manually labeled sample sets are quite needed to guarantee the accuracy of the algorithms". etc.
The title must be revised to improve the readability of your work, your work is doing some qualitative analysis for samples not generating the samples.
What are the major differences in your SS-ELM and previously proposed SS-ELM models for HSI classification?
The abstract and flowchart highly contradict. It seems the authors are trying to propose a co-training model while using two traditional semi-supervised classifiers such as one class SVM and multi-class ELM followed by k-means clustering. There is no strong motivation to use k-means, SVM, and ELM. ELM is known to improve robustness/generalization performance.
Author Response
We have carefully considered all the comments made by the editors and reviewers and incorporated them into our revised manuscript titled “A Semi-supervised Framework for Classification of the Complex Agricultural Planting Structure”. We sincerely appreciate three anonymous reviewers and the editor for their constructive comments which surely make this manuscript stronger than before. The following are the point-to-point replies to all the comments.
Comments of Reviewer #2:
Comments and Suggestions for Authors
(1) Thank you for addressing the comments. However there many things that need to be carefully revised. For instance; The writing of the paper needs to be greatly improved, e.g., line # 15-17, "low computational efficiency" (authors means performance in terms of accuracy?), "Moreover, a large number and good of the manually labeled sample sets are quite needed to guarantee the accuracy of the algorithms". etc.
Response: Thanks for your suggestion. We improved our manuscript.
(2) The title must be revised to improve the readability of your work, your work is doing some qualitative analysis for samples not generating the samples.
Response: Thanks for your suggestion. We changed the title as “A Semi-supervised Framework for Classification of the Complex Agricultural Planting Structure”.
(3) Why did the authors choose a time interval of five years? What are the major differences in your SS-ELM and previously proposed SS-ELM models for HSI classification?
Response: Thanks for your suggestion. For HSI classification, the previously proposed SS-ELM models can construct more reasonable graphs or extract more effective features from the abundant image information. However, due to the low resolution of agricultural planting structure, the previously proposed SS-ELM cannot construct reasonable graphs or extract effective features. Our proposed SS-ELM algorithm is capable of classifying the agricultural planting structure by using two algorithms to jointly label the effective information from low-resolution images.
(4) The abstract and flowchart highly contradict. It seems the authors are trying to propose a co-training model while using two traditional semi-supervised classifiers such as one class SVM and multi-class ELM followed by k-means clustering. There is no strong motivation to use k-means, SVM, and ELM. ELM is known to improve robustness/generalization performance.
Response: Thanks for your suggestion. The corresponding description of the flow chart is in lines 154-159: “The classification method includes three steps: image segmentation with k-means, self-label and planting structure classification (see Fig. 3). First, the non-agricultural regions are segmented using the k-means unsupervised learning algorithm. Then the co-training self-label algorithm based on the SVM and ELM classifiers is used to enlarge the sample set (see Table 2). And finally, the enlarged sample set is used to perform the classification of agricultural planting structure in the large-scale area.” We also improved the abstract, see lines 16-19: Thus, we proposed a new semi-supervised extreme learning machine (SS-ELM) classification framework which consists of the components including image segmentation using k-means clustering algorithm, planting structure classification using the co-training self-label algorithm and accuracy evaluation.
Sincerely yours
Guanhua Huang
On behalf all authors
Reviewer 3 Report
The paper presents an automatic crop classification (mainly) based on SS-ELM. The paper is well structured, the introduction is complete and the results are presented and discussed in detail. I recommend its acceptance prior to some minor issues:
- The introduction of the SS-ELM needs a brief description in one sentence in methodology, to make it clearer how it differs from other traditional methods mentioned in the previous sentences.
- In Figure 5, place "Training" where it does not overlap with a box
- The data shows that the number of samples between classes is not balanced. Have the authors considered weighting the samples? This is not clear in the text.
- Why haven't the authors considered adding the class "others"?
Author Response
Comments of Reviewer #3:
Comments and Suggestions for Authors
The paper presents an automatic crop classification (mainly) based on SS-ELM. The paper is well structured, the introduction is complete and the results are presented and discussed in detail. I recommend its acceptance prior to some minor issues:
(1) The introduction of the SS-ELM needs a brief description in one sentence in methodology, to make it clearer how it differs from other traditional methods mentioned in the previous sentences.
Response: Thanks for your suggestion. We added the sentences in section 3, see lines 159-160 : Compared with the other traditional methods, the SS-ELM algorithm is more suitable to solve the problem of agricultural planting structure classification.
(2) In Figure 5, place "Training" where it does not overlap with a box
Response: Thanks for your suggestion. We have modified Figure 5 as follows：
(3) The data shows that the number of samples between classes is not balanced. Have the authors considered weighting the samples? This is not clear in the text.
Response: Thanks for your suggestion. This article did not consider weighting the samples, the reason is that the sample imbalance problem is not obvious.
(4) Why haven't the authors considered adding the class "others"?
Response: Thanks, we did not consider “others” due to that first after using k-means clustering image segmentation, others category has been pruned. And second there is no unlabeled classification in the final ELM classification.
Sincerely yours
Guanhua Huang
On behalf all authors
Author Response File: 
Author Response.docx
Round 2
Reviewer 2 Report
Thank you, however, the rebuttal is far from convincing, though the writing of the paper and the response letter is still vague. Well, if one of the Machine learning models has been deployed for one application, then we can not claim its novel for other problems as well, since it has already been proposed and deployed by someone else. So I recommend authors to bring novelty in their work rather claiming that they have proposed "A Novel K-Means segmentation followed by SS-ELM". None of these are new, and none of these baselines has been modified! So jointly investigating two of well recognized ML algorithms doesn't mean that it's new!
