Comparison of Different Transfer Learning Methods for Classification of Mangrove Communities Using MCCUNet and UAV Multispectral Images

Round 1

Reviewer 1 Report

please see the attached file

Comments for author File: Comments.pdf

Author Response

Response to Reviewer 1 Comments

 

Point 1: Tittle: The paper title should be changed to “Comparison of different transfer learning methods for classification of mangrove communities using MCCUNet and UAV multispectral images”, because the article has no innovation in the transfer learning method, but only compares three existing transfer learning methods.

 

Response 1: Thanks for your careful comments and suggestions. We have revised the “Tittle” as follow：

Tittle: Comparison of different transfer learning methods for classification of mangrove communities using MCCUNet and UAV multispectral images

 

Point 2: Abstract: The sentence “The performance of different transfer learning methods in classifying mangrove communities has not yet been demonstrated.” is too absolute, there are also some researches on transfer learning methods in mangrove classification.

 

Response 2: Thanks for your comments and suggestions. We have replaced the relevant statement in the " Abstract " as you suggested, as follows:

Mangrove forest classification using deep learning algorithms has attracted increasing attention but remains challenges. Especially, current studies on transfer classification of mangrove com-munities between different regions and different sensors is still unclear.

 

Point 3: Line 222: The table number is wrong, such as table 4 should be table 3.

Response 3: We appreciate your careful comments and suggestions. We have carefully checked each table and ensured that all tables have the correct serial numbers and are properly cited in the results section.

 

Point 4: Figure 11: The meaning of the figure is not clear, such as the land cover types have been given at the bottom of the figure. What do the four types indicated on the left side of the figure mean? And three algorithms were introduced earlier in the article, only two algorithms are used here, it is recommended to add.

Response 4: Thanks for your comments and suggestions. In order to make the meaning of the figure clear, we removed the name of the relevant species from the left side in the revised paper. Please see the attachment for the new figure.

 

Point 5: Figure 13: Why only two feature datasets (OST and OSTV) are used here, OS and OSV?.

Response 5: We appreciate your careful comments. There are 24 classification scenarios in total (three algorithms, four datasets and two regions) in this paper, and if the classification result of each feature dataset is analyzed in detail, there are bound to be many repetitive and cumbersome contents. The aim of this paper is to compare the classification performance of three deep learning algorithm (MCCUNet, DeepLabV3+ and HRNet) under the same dataset. We select the two feature datasets (OST and OSTV) for detailed analysis in the part according to the result of significant differences derived from McNemar's test in Table 7. Table 7 shows that In Regions 1 and 4, there is a significant difference of classifications between MCCUNet and DeepLabV3+ when using the OST feature dataset. Meanwhile, there is also a significant difference classification between MCCUNet and HRNet when using the OSTV feature dataset. So, we used the OST and OSTV feature datasets to verify and compare the classification performance of three deep learning algorithm in Figure 13.

 

Point 6: Figure 20: The meaning of the figure is not clear, the changes in F1-score of each land cover type before and after the implementation of the Ft-TL strategy should be analyzed here.

Response 6: Thanks for your comments and suggestions. We rewrite these relevant contents, and analyze the changes in F1-score of each land cover type before and after the implementation of the Ft-TL strategy. Please read the revised paper. Please see the attachment for the new figure.

 

Point 7: Line 486: For “3.3. Evaluation of the effect of different transfer learning strategies on mapping mangrove communities”, the transfer learning methods F-TL and Ft-TL only analyze region 2 and region 3, and don't consider region 1 and region 4. Figure 15 analyzes five types of land cover, and Figure 16 analyzes six types of land cover, shouldn't the types be the same?

Response 7: We appreciate your careful comments. As we describe in Table 5 of the paper, the source domains of our proposed F-TL and Ft-TL strategies are both Regions 1 and 4, and the target domains are both Regions 2 and 3, which aim to verify the transfer learning ability of the model between different regions. In contrast, the source domains of the SaP-TL strategy are Regions 1 and 4 in January 2021, and the target domain is Regions 1 and 4 in November 2020, which aims to verify the transfer learning ability of the model between different time phases and different sensors. The fundamental difference between the three transfer learning strategies for training the model is the different learning rates. For Regions 1 and 4, the differences in the domains between the image data of different time phases and sensors are relatively large, and the F-TL and Ft-TL strategies may be difficult to fit the model well, so we did not consider the F-TL and Ft-TL strategies in Regions 1 and 4.

The AF (Artifacts) defined in our article is more distinctive and easily identifiable, and its distribution range is mostly located at the periphery of mangrove vegetation, with only a small portion sporadically distributed inside the mangrove. In addition, the focus of our article is on mangrove vegetation and its related natural features, therefore, we do not display the F1-score of AF in the Figure 15.

Author Response File: Author Response.pdf

Reviewer 2 Report

I have carefully read the paper titled “Transfer learning classification of mangrove communities using MCCUNet and UAV multispectral images” which discusses the use of a modified version of DeepLabV3+ for high-resolution UAV-based coastal mangrove wetland classification utilizing three different transfer learning strategies. Overall, the paper is interesting and of high importance. There are several issues that are required to be addressed by the authors as follows:

1. I do understand that the classification of high-resolution wetlands is quite complex, but I am concerned about the distribution of reference data and the high correlation between the training and test data. Utilizing a random data sampling strategy will significantly increase the correlation between train and test data. As such, the evaluations are not reliable, in my opinion.

2. For transfer learning, you have four different regions. Do they have similar spectral characteristics, or do they have any differences? Would it not be better to see how the proposed methods work on more diverse spatial and temporal data? Elaborate in detail on how similar or different those regions of study are.

I really liked the idea of removing highly correlated features; well-done.

This paper is well-written and comprehensive and would be of high interest.

There are two main issues here:

1.      Data sampling strategy. Random data sampling will result in a high level of correlation between training and test results. As such, the results are not reliable. I do understand that in almost 99% of studies, the researchers use random data sampling.

2.      Selection of the region of interest for transfer learning.

With some explanation, I do suggest the publication of this high-quality paper.

Author Response

We thank the editors and reviewers for the valuable comments and suggestions. We have revised the paper substantially; and carefully made corrections according to the reviewers' comments and suggestions. Revised portions of manuscript are highlighted by red color. Our detailed point-by-point responses to the reviewers' comments are given below.

Response to Reviewer 2 Comments

 

Point 1: Data sampling strategy. Random data sampling will result in a high level of correlation between training and test results. As such, the results are not reliable. I do understand that in almost 99% of studies, the researchers use random data sampling.

 

Response 1: We appreciate your careful comments and suggestions. As you mentioned, we used the random cropping method, which is the most common approach in the current researches, to obtain the training data, and it is our oversight to further consider the correlation between training and test results. We had initially considered non-repetitive image subdivision by fixed size, but this would not produce enough number of images to construct training and test datasets. For example, the image size of Region 1 is 7448 × 9981, and if the image subdivision is performed at 256×256 size, we would get roughly 1102 images, and then data enhancement processing can be performed, we would end up with 7714 images, which is much smaller than the 100,000 images used in our experiments. However, the problem of data correlation is indeed not negligible.

In future experiments, we will use a fixed-size window with overlapping areas to crop the images for the binning, in order to try to avoid the uncontrollable correlation of the data obtained with image crop.

 

 

Point 2: Selection of the region of interest for transfer learning.

 

Response 2: Thanks for your comments and suggestions. We have calculated the mean and standard deviations of the different spectral features for each land cover type in the four regions based on your suggestions (Figure 1). The mean spectral reflectance for each land cover type is obtained by summing the reflectance of the pixels of the corresponding category in the spectral feature on a pixel-by-pixel basis based on semantic labels and then averaging.

As can be seen in Figure 1, there are different differences in the mean reflectance values for each land cover type in the six domains for different spectral features. For the visible light features (Blue, Green, and Red), some similarity in the distribution of the mean values of reflectance for each land cover type can be seen in Figure 1 (a), (b) and (c). Among the three mangrove vegetation types, the ranking order of the mean values of reflectance from the largest to the smallest of AM, for example, is as follows: Region 1 (2020.11) > Region 4 (2020.11) > Region 3 (2021.01) > Region 4 (2021.01) > Region 2 (2021.01) > Region 1 (2021.01). Except for the mangrove vegetation, the distribution of the mean values of reflectance in WB, for example, is also consistent in the visible features, with the lowest values in Region 1 (2021.01) and the highest values in Region 4 (2020.11).

For the non-visible features, the trend of the mean values of reflectance did not change significantly for each land cover type, but the difference between the highest and lowest values changed significantly. Among the mangrove vegetation, also in the case of AM, the mean values of reflectance still achieved maximum values at Region 1 (2020.11). Except for the mangrove vegetation, for SA, the ranking order of the mean values of reflectance from the largest to the smallest is also as follows: Region 1 (2020.11) > Region 4 (2020.11) > Region 3 (2021.01) > Region 4 (2021.01) > Region 2 (2021.01) > Region 1 (2021.01).

Please see the attachment for Figure 1.

In addition, we calculated the mean values of reflectance for different spectral features between the domains (without considering different land cover types), as shown in Figure 2. It can be seen that the differences in the mean values of the reflectance of the six domains in the visible feature fraction are small, while in the non-visible, the mean values of the reflectance of Region 1 (2020.11) and Region 4 (2020.11) are significantly higher than those of the other four domains.

Please see the attachment for Figure 2.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper develops the MCCUNet deep neural network to benefit from three transfer learning strategies to tackle the challenges of Mangrove forest classification. Experiments on high-resolution UAV multispectral images demonstrate that the strategies are effective and lead to improved performance compared to the baseline.

  The results of the paper are convincing. However, I think the contribution of this paper is minimal and the results are not unexpected. The paper is only using known methods on a dataset by updating an exiting network architecture minimally. In this sense, I don't find the contributions of this paper sufficient for publication at Remote Sensing.

Author Response

Response to Reviewer 3 Comments

 

Point 1: The paper develops the MCCUNet deep neural network to benefit from three transfer learning strategies to tackle the challenges of Mangrove forest classification. Experiments on high-resolution UAV multispectral images demonstrate that the strategies are effective and lead to improved performance compared to the baseline. The results of the paper are convincing. However, I think the contribution of this paper is minimal and the results are not unexpected. The paper is only using known methods on a dataset by updating an exiting network architecture minimally. In this sense, I don't find the contributions of this paper sufficient for publication at Remote Sensing..

 

Response 1: Thanks for your comments. We response your comments from the two aspects of innovation and workload of our study.

(1) the contributions of our study:

Multi-dimensional datasets can improve the classification performance of the algorithm, but introduction of too many image features also brings the “curse of dimensionality” problem. So, we address to combine the feature recursive elimination (RFE) and principal component analysis (PCA) algorithm to perform dimension reduction on UAV datasets in the study area, and validate the effectiveness of RFE-PCA method for improving the classification performance of mangrove communities.

Current studies have proved that DeepLabV3+ algorithm has good classification performance on terrestrial vegetation, but its simple decoder makes it lose some edge information when performing image segmentation. Therefore, we developed a new deep learning algorithm (MCCUNet) by adding mixed-size depth convolution to the encoder of DeepLabV3+ to improve the perceptual field, and adding additional low-level feature layers to the decoder to improve the segmentation performance. and comparing it with the original DeepLabV3+ and HRNet algorithms. Afterwards, we compare the MCCUNet algorithm with the original DeepLabV3+ and HRNet algorithms to verify the superiority of the MCCUNet algorithm for mangrove vegetation classification.

Mangrove forest classification using deep learning algorithms has attracted increasing attention but remains challenges. Especially, current studies on transfer classification of mangrove communities between different regions and different sensors is still unclear. As a result, we proposed three transfer learning strategies based on existing transfer learning methods, combined with UAV images from different regions and different time phases, and explored the applicability of the three transfer learning strategies.

(2) the workload of our study

We used three deep learning algorithms with four regions as the study area, constructed four image feature datasets, and proposed three migration learning strategies. In conclusion, we have a total of 24 classification scenarios and 12 migration learning scenarios.

In summary, we think the contributions of our study is enough sufficient for publication at Remote Sensing. Please you evaluate our revised paper and our contributions again.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

All my concerned problems have been solved, and i have no other comments

 

Author Response

Thank you for your comments and acceptance.

Reviewer 3 Report

Dear authors,   Thank you for your responses. I still think the contributions of this paper are limited in terms of ML/AI:   1. Using PCA to address curse of dimensionality is a 101 method for this task and there is no novelty in using PCA to reduce data dimension.   2. Adding known layers to a known network architecture has incremental novelty. This contribution can be at the level of an undergraduate course on AI homework.   3. Using deep learning for Mangrove forest classification can be novel but its practical importance is outside my expertise and I don't have an opinion on this matter. I can only say that in terms of AI/ML, this paper has incremental novelty. Perhaps, a reviewer whose expertise Mangrove forest classification can judge whether the practical importance of this work warrants its publication.

Author Response

Thank you for your comments and review.