Classification of River Sediment Fractions in a River Segment including Shallow Water Areas Based on Aerial Images from Unmanned Aerial Vehicles with Convolution Neural Networks

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This study used the CNN method to classify aerial image taken by UAV and further investigate riverbed grain size. This paper is well-written and high quality.

I have two major comments below.

1.      Figure 9 and Figure 10 show distribution map of grain size. What’s the definition of grain size, e.g., d50?

2.      To grain size of riverbed surface, we generally pay attention to the roughness height which is approximately equal to mean axis of gravels or bounders. However, this study seems to classify the geometry of grains in riverbed surface. This issue needs to be discussed.

 

Relevant references:

Pearson et al., 2017. Can high resolution 3D topographic surveys provide reliable grain size estimates in gravel bed rivers? Geomorphology.

Vázquez-Tarrío, D.,Borgniet, L.,Liébault, F., et al. 2018. Using UAS optical imagery and SfM photogrammetry to characterize the surface grain size of gravel bars in a braided river (Vénéon River, French Alps). Geomorphology.

Comments on the Quality of English Language

Need to improve.

Author Response

Dear the reviewer of our manuscript (remotesensing-2704334)

 Thanks for your helpful comments. We will reply to your comments and questions below. 

Regards

 

 1. Figure 9 and Figure 10 show distribution map of grain size. What’s the definition of grain size, e.g., d50?

Strictly speaking, Figures 9 and 10 do not show the grain size, but rather the distribution of the grain size classes shown in Table 1. Continuing from our previous study, samples whose D50 obtained from the sieving falls within the range of each particle size class are trained as belonging to that class, so it can be said that these are samples whose d50 falls within each range.

 

 

2. To grain size of riverbed surface, we generally pay attention to the roughness height which is approximately equal to mean axis of gravels or bounders. However, this study seems to classify the geometry of grains in riverbed surface. This issue needs to be discussed.

 In this paper, image acquisition and training are performed for one specific river. There is a possibility that the classification results will differ if the shape characteristics of the particles differ from the riverbed samples for training. However, as stated in the introduction, the purpose is not to perform a detailed particle size evaluation, but to classify particles into categories that are rougher than the Wolman Pebble Count, which has a relatively wide range of particle size values. Therefore, the influence of differences in particle shape characteristics is relatively small. Also, although I did not try this time, if the training data includes data from all the rivers to be classified, taking into account that the shape characteristics of particles are different, there is a high possibility of correct classification.

In addition, different particle size classifications and names are used when conducting particle size surveys from the perspective of river engineering and from the perspective of the physical environment of living and growing habitats. As a result, even though the same river channel was surveyed, it becomes difficult to compare and integrate the results, which may lead to inefficiencies in the interpretation and analysis of river channel characteristics. Therefore, it is desirable to use the same particle size classification and name as much as possible. From this perspective, in this study, we considered that the particle size classification specified by Japan's Ministry of Land, Infrastructure, Transport and Tourism is a classification that can be commonly applied to all of them, and we set classes accordingly.

​

Reviewer 2 Report

Comments and Suggestions for Authors

This is an interesting paper on the use of UAV imagery to study bed sediments under shallow water. This research addressed the challenges that usually come with the UAV imagery for water resources study. Remote sensing community would be highly interested in reading this paper. However, I think the paper was written poorly and the methods were not designed properly.

One of the major concerns that I found is the combination of the previously conducted similar research with the current research. It is understandable that the major technique was designed in the previous study. But since the current focus is on the shallow water environment, I think it would be a better approach just to use the training and testing samples only acquired in shallow water conditions. Therefore, I suggest revising the methods and the manuscript accordingly.

Some specific comments are as follows:

·         It seems Figures 1 and 3 are almost like what were published earlier (Paper # 55). They need to be modified.

·         Figure 2 is missing (or typo).

·         Figures 5 and 6 need scales.

·         Figure 7 needs a scale/north arrow.

·         Figure 8 - caption needs more explanation.

·         Figures 9 and 10 need to be expanded or split into multiple figures. Difficult to understand.

·         Abstract needs to be revised to better reflect the research outcomes.

 

 

Comments on the Quality of English Language

English language seems OK. 

Author Response

Dear the reviewer of our manuscript (remotesensing-2704334)

 Thanks for your helpful comments. We will reply to your comments and questions below. 

Regards

 

One of the major concerns that I found is the combination of the previously conducted similar research with the current research. It is understandable that the major technique was designed in the previous study. But since the current focus is on the shallow water environment, I think it would be a better approach just to use the training and testing samples only acquired in shallow water conditions. Therefore, I suggest revising the methods and the manuscript accordingly.

The purpose of this method is not just for underwater samples. In the second half of the paper, we focus on the underwater samples, but we are only discussing this as an example to demonstrate the validity of this method by comparison with the topographic gradients. In actual classification, it may be necessary to obtain grain size information for both underwater and terrestrial samples over the entire width of the river channel, such as when evaluating riverbed roughness during high water levels. For such applications, we believe it is more appropriate to perform training and accuracy assessments using both underwater and terrestrial samples.

 

• It seems Figures 1 and 3 are almost like what were published earlier (Paper # 55). They need to be modified.

In Figure 1, the background photo of Site 1 & 2 has been changed to the one taken on the day the particle size survey of the underwater sample was conducted. The approximate locations where particle size analysis of underwater samples was performed are shown. Also, an explanation has been added to Figure 2 (Figure 3 in the old manuscript).

 

• Figure 2 is missing (or typo).

It's a numbering error. I am sorry. Fixed.

• Figures 5 and 6 need scales.

Thanks for your pointing out. I added

 

• Figure 7 needs a scale/north arrow.

Thanks for your pointing out. I added

 

• Figure 8 - caption needs more explanation.

Thanks for your pointing out. I added

 

 

• Figures 9 and 10 need to be expanded or split into multiple figures. Difficult to understand.

I expanded. Thanks for your suggestion

 

• Abstract needs to be revised to better reflect the research outcomes.

I revised some sentences unclear

 

Reviewer 3 Report

Comments and Suggestions for Authors

Please see word file

Comments for author File: Comments.pdf

Comments on the Quality of English Language

English language could be improved. There are also several sentences that are missing parts.

Author Response

Dear the reviewer of our manuscript (remotesensing-2704334)

 Thanks for your helpful comments. We will reply to your comments and questions below. 

Regards

1.    The topic of the paper is appropriate for the Remote Sensing Journal 

Thanks for your reviewing our manuscript. We have made corrections based on the comments we received.

 

2.    Title: I suggest to modify the title. It is unclear to what “v” the title refers to 

“wide range”　was not a clear definition. As an adequate unit for our research, we use “river segment” in the title

 

3.    It is not very clear how the aerial photography was performed: In total 108 images were taken (and only 32 underwater) or were surveys made on 108 locations? 

I revised Line173 in the new manuscript. Line 205-209, we have noticed that the number of our survey was not sufficient but increased the number of available images with the application of BASEGRAIN.

 

4. What is the overlap between images?

The overlap rate was set to 10% to increase the time efficiency of the observation. I add that information Line 399

 

5.  The accuracy of the classification depends heavily on the number of the training data. The  authors failed to present accuracy analysis, location, or the number of accuracy assessment points. Did the authors leave a number of samples unused by the CNN method for accuracy assessment?  

I added the numbers of images used for training and assessment in lines 217-220

 

6. I suggest adding a map with sampling points 

I added the sampling points for underwater samples in Figure 1. The locations of the points for terrestrial sample is considered not so important, just disturb the map. We don’t put the points, as same as our previous study.

 

 

7.    Class 3 - Can a class with particle size less than 2mm be defined in the images where pixel resolution is 2,74 mm? Please for a detailed description read the comment for Line 198 

I revised Table 1.

0.5mm is not the limit in our survey.

 

 

8. What is the number of images and particle size samples per site? 

As shows in Figure 1, all of the underwater samples were taken in the Site1. That is due to the limitation of the survey time. Regarding the terrestrial samples were taken also in the Site 2. In our judge, bed condition and particle shape characteristics of the both sites are generally similar. That imbalance is considered not a matter.

 

9.    There is no Figure 2.  

Sorry for my misnumbering the figures, I revised.

 

10. You don’t mention Figure 3 and Figure 4 in the text.  

Figure 2 (3 in old manuscript ) in Line 211 in the new manuscript, Figure 3 (4 in old manuscript ) in Line 216, I mentioned

 

11. There are two figures named Figure 7 

Sorry for my misnumbering the figures, I revised.

 

 

     Line 15  Please rewrite the sentence (divide in two after “(CNN)” or add “that”?) 

Thanks for your pointing out. I split the sentence.

 

 

     Line 23 Particles most probably did not form a rapid. A rapid was formed in the steep section… I would advise using proper (geo)morphological terms to better describe the form, as well as in Line 480. I suggest Dove et al. (2020) DOI: 10.5281/ZENODO.4075248 or similar

Thanks for your suggestion. “Rapid” is not suitable. I changed it to “riffle”, to express the morphological condition of water surface.

 

 

     Line 47  Please finish the sentence. “a wide range” of what? 

I supposed that it is not necessary to specifically write a wide range “of what” is in this context, which says that it is used for a variety of purposes.

 

 

     Line 49-59  Is this relevant? The mentioned studies do analyze images to perform particle size, but are not made by UAV, and not used underwater, with significantly different resolution. 

 

I omitted the references as you suggested. That’s simple.

 

 

    Line 65 Please remove “his” or rewrite the sentence 

Thanks for your finding. I erased it.

 

 

     Line 72 Part of the sentence is missing 

I revised it in Line 65 of new manuscript.

 

 

     Line 108 Please define abbreviation when using for the first time (SfM) 

I added the full description for the first SfM in Line 101

 

 

     Line 115 Please remove an excessive bracket 

Line 108 in new manuscript, I erased it

 

 

     Line 116-117 I disagree. Please rephrase the statement to make it clear. Laser does penetrate  the water surface and is intensively used to complement other bathymetric underwater methods like multibeam echosounder, as well as in (geo)archaeology. Please consult e.g. DOI:10.1016/j.jas.2012.12.021; DOI  10.2495/CE010261;  MICHAEL DONEUS, NIVES DONEUS, CHRISTIAN BRIESE, GEERT VERHOEVEN AIRBORNE LASER SCANNING AND MEDITERRANEAN 

 ENVIRONMENTS - CROATIAN CASE STUDIES; or similar 

My description was not correct. Green laser can be the solution, but it is still expensive. Line 108-112 express that

 

Line 129 Is it GoogLeNet or GoogleNet or Googlenet (Line 241, 243). Also, please cite the autor(s)

Sorry for my mistyping, I revused and cite as [55] in new manuscript.

 

 

Line 158 River name is written Saigo (as well as in Line 504), but in Figure 1 it is Saigoh 

I revised all to Saigoh

 

Line 154-164 A part of this paragraph is not very relevant to the Article. I suggest making it shorter, only with the accent on sluicing as the reason for a change in sedimentation pattern before or after typhoon  

 

 The change of operation which is unusual event in the field of water resource management made us expect the drastic change of riverbed. I would like to keep it.

 

 

Line 184 Please add a table (Table 1) after it is mentioned in the text, not before 

First “Table1” in the new manuscript appears Line 178, then I placed later than it.

 

Line 186 and Table 1 Please define what classification is used –  Folk, Wentworth… I know that it is stated that JIS standard sieves were used, but this is a national standard, so better clarify. 

We added the information of the classification of Table1

 

 

Line 198-206 Please bring into relation the camera resolution pixel size and Table 1 classes. Explain how it is possible to classify sediment with particle size 0.5 to 2 mm (Class 3) when pixel size is 2.74mm. In my view, it is possible only to classify sediments with particle size less than 2.74mm as a separate class. So not to define a class with a lower boundary (0.5mm), and not to define a class that you cannot detect on the image. I think this is a serious flaw that needs to be addressed in the whole paper and maps. I suggest to redefine Class 3 as >2.74mm (not 2-0.5 mm). 

 

This image analysis method does not distinguish and measure individual particles, but rather performs classification based on the pattern trends of the entire image. The classification criteria for training are based on the results of sieving, and to respect this, we classify them as shown in Table 1. Therefore, I think it is appropriate to set the threshold to 2 mm without considering the resolution. Just I revised the minimum limit for class 3.

 

 

Line 217 Figure 2 – it does not exist as a figure in the text.  I presume Figure 4 is actually Figure 2 

Line 395 Figure 7 is used two times. You use Figure 7 numbering in the previous paragraph as well. This one should be Figure 8 

I revised the numbers of figures. Sorry for my simple error.

 

Line 448-449 It is not necessary to explain water direction two times. Blue arrows on the map are sufficient 

I omitted.

 

Line 491 please rephrase a “little far” to something like “further” or similar 

I revised it as suggested.

 

 

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

 

Thanks for revising the manuscript by addressing my concerns/suggestions. It is in much better shape now. However, I still think there should be some discussions in the discussion section to address my concern about not having the training and testing samples only acquired in shallow water conditions.  

Author Response

Dear the reviewer

Here I reply to your comment. Your suggestion is interesting for the process, but I like to omit that from our manuscript. I just share the Confusion Matrix of underwater samples. 

 Comment: However, I still think there should be some discussions in the discussion section to address my concern about not having the training and testing samples only acquired in shallow water conditions.

Reply: 

As shown in the attached table, we were able to obtain similarly high accuracy even when the learning and accuracy assessment was performed only on the underwater riverbed material. However, this result is not be added to our manuscript as it may mislead readers other than you. Generally, as the number of classification categories increases, the discrimination accuracy decreases, and if the classification method has sufficient accuracy for the final use, Subdividing the classification into a lower number of categories doesn't have a meaning.

I would also like to add Mr. Umino, who conducted this additional analysis, to the list of authors.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Please read a word file attached.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

No further comment

Author Response

Dear the reviewer

Thanks for your helpful comments again. I am replying to your precious comments, as below.

 

Comment: Usually, overlap in surveys and studies is in the range at least 20% to 30 %, recommended  40-50%. 

Thanks for your comment. An overlap rate of about 50% is required when creating orthoimages by overlapping multiple shots, and 80% for SfM. In this shooting, the raw image after shooting was divided into 1m coarse clips, and image classification was performed for each clip. In other words, the classification result is a collection of clips with a coarse resolution of 1m, and we believe that there is no great point in seeking highly accurate position information for each clip. As explained in the paper, the classification results are reprojected on the orthoimage by determining the approximate position from the Yaw angle and coodinates included in the XMPMetadata. The orthoimage of the background was created from the images taken from an altitude of 60 m and an overlap rate of 80%. I added this information in Lines 276-278.

As a method different from the one shown in the paper, it is possible to align all captured images using SfM software, convert them into orthoimages, and then divide them into meshes for image classification. However, there are concerns that the orthorectification process may distort the image pattern information or reduce the resolution, which may affect the classification results. In this study, we focused on maintaining the pattern of the image and chose the reprojection method described above.

Furthermore, if we try to increase the overlap rate, the number of shots will increase and the observation time will be about twice as long, so we will lose some of the advantages of this method, which is that it can shorten work time. Considering the grain size of the river bed material targeted at our study site, a flight altitude of 10 m was appropriate, taking into account the resolution of the photographs. Although photographing at this altitude is a light task compared to the usual volumetric method, it still takes a certain amount of time even with an overlap rate of 10%. We judge that doubling this amount is not realistic in terms of labor.

At sites where the river bed is made up of materials with larger grain sizes, the flight altitude is increased to widen the range of shots taken at a time and increase the overlap rate to create orthoimages, which are then clipped for learning and discrimination. It may be a good idea to try to improve efficiency.

Judging comprehensively from the particle size to be determined at this site and the precision and work time required to identify the position of each clip, we believe that the proposed method is the most appropriate.

 

 

 

 

Comment: I would like to clarify my suggestion added as Comment 5. It is custom, and statistically and scientifically correct, to leave a number of samples out of the training set, to be used later for accuracy assessment. So, to be clear, it is necessary to leave one part of the samples used for grain-size analysis (not images) out of the training set, and that samples should then be used for additional testing called accuracy assessment. These samples are used after the classification is made to assess it’s accuracy. Did you do that? So not with the images but with the sediment samples, because in the end that is what you are mapping, sediment classes. 

 

Thanks for your comment.

In the red frame area in Figures 2 and 3, we obtained the particle size based on the image analysis with BASEGRAIN and the results using the volumetric method, and the consistency of these was shown in our previous study [54]. In addition, grain size analysis was performed using BASEGRAIN in the yellow frame area, and only the images that were determined to be in the same grain size class as the BASEGRAIN result in the red frame were used as test data. The images that were classified as being in a different class were excluded. In other words, the image used in the test is also considered to have the same granularity class as the red frame. Logically, I think this is fine.