Machine Learning and Deep Learning for the Built Heritage Analysis: Laser Scanning and UAV-Based Surveying Applications on a Complex Spatial Grid Structure

Round 1

Reviewer 1 Report

Review Reports

COMMENTS FOR THE AUTHOR:

This research article focuses on the automated segmentation and classification methods of surveying outputs to improve the interpretation and Building Information Modeling from laser scanning and photogrammetric data. There are few observations/suggestions to the authors.

 

Brief Summary

This paper is highlighted on the reconstruction of 3D geometries starting from reality-based data is challenging and time-consuming due to the difficulties of modeling existing structures and the complex nature of the built heritage. The research relies on Artificial Intelligence (Machine Learning and Deep Learning) and focuses on the pilot case study of the grid structure in steel named ‘La Vela’ 21 in Bologna. The work confirms that AI-based techniques may reduce the time and modes of surveying and reconstruction.

 

General concept comments

 

1.       Abstract section should have any numerically results of steps performed in algorithms.

2.       Introduction sections must conclude with the contributions of the research work presented in the article.

3.       Related work must be separated from the Introduction section and research gap must be clearly. mentioned.

4.       How identify specified method to apply reconstruction 3D geometries. Did authors apply any segmentation techniques? Need more clarity on this.

 

5.       The problems of recovery and maintenance of structures from the late 19th - 20th - 21st centuries? Need to highlight in briefly.

 

6.       Authors are suggested to provide the analysis on the evaluation parameters for identifying proper method of reconstruction of 3D geometric shapes.

 

Specific comments 

Abstract

1.       Line no. 17, 18, 19 should be rewrite for more clearification

Introduction

Introduction section written properly so there isfew specific comment for introduction section

Line No. 74 heading should be write according subject of title

Paragraph of lines from 88 to 114 should be rewrite there own language think and rewrite

Paragraph of lines from 190 to 199 should be rewrite accoding their opnion.

Material

                In topic topographic frame work paragraphs of lines from 251 to 258 should be rewrite

                Paragraph of lines from 260 to 268 should be rewrite also.

In topic Range based survey paragraph of lines from 288 to 302 and from lines from 304 to 307 should be write according their own language.

Results

Result and discussion section should be well written according execution steps.

Conclusion

In conclusion also write drawback of automated segmentation and classification methods of surveying outputs to improve the interpretation and Building Information  Modeling from laser scanning and photogrammetric data.

References

Some references are shown extra so plz remove extra references those not related to research paper. All citations should be cross check with references.

Comments for author File: Comments.pdf

Author Response

Reviewer 1

Dear reviewer,

We greatly thank you for the thorough and thoughtful comments provided on our submitted article. It has taken us a rather long time to complete the updated version, that we are submitting again to your attention. We made sure that each one of the reviewer comments has been addressed carefully and that the paper is revised accordingly.

We revised the structure of the paper in order to better clarify the distinction between the proposed methodological approach and the experimental results of its application on the case study, as well as the conclusions (in terms of strengths and weaknesses of the proposed method). We also incorporated more references in order to better illustrate the state-of-the-art section.

Please find attached the detailed responses to your comments. The latter are shown in black and our responses in red.

COMMENTS FOR THE AUTHOR:

This research article focuses on the automated segmentation and classification methods of surveying outputs to improve the interpretation and Building Information Modeling from laser scanning and photogrammetric data. There are few observations/suggestions to the authors.

Brief Summary

This paper is highlighted on the reconstruction of 3D geometries starting from reality-based data is challenging and time-consuming due to the difficulties of modeling existing structures and the complex nature of the built heritage. The research relies on Artificial Intelligence (Machine Learning and Deep Learning) and focuses on the pilot case study of the grid structure in steel named ‘La Vela’ 21 in Bologna. The work confirms that AI-based techniques may reduce the time and modes of surveying and reconstruction.

Thank you for your review and interest in our work.

General concept comments

1. Abstract section should have any numerically results of steps performed in algorithms. The abstract was revised and rewritten accordingly, even better stressing the aims of the paper and numerically indicating the results.
2. Introduction sections must conclude with the contributions of the research work presented in the article. We introduced Paragraph 1.1 on Research Aims and revised the whole Introductory part.

3. Related work must be separated from the Introduction section and research gap must be clearly mentioned. We separated Related work from Section 2. State of the art: supervised ML and CNN-based classification methods. Previous Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century was removed to rather concentrate the Related work section on the State of the art of ML and CNN-based classification methods.

4. How identify specified method to apply reconstruction 3D geometries. Did authors apply any segmentation techniques? Need more clarity on this.

We have added a sentence, in section 4.3 Data fusion and construction of the BIM model, in which we clarify that the reconstruction method by primitive fitting follows the RANSAC algorithm fir shape detection: ‘The Random sample consensus (RANSAC) algorithm is considered for the detection of basic shapes in the point cloud, following reference [47]’

5. The problems of recovery and maintenance of structures from the late 19th - 20th - 21st centuries? Need to highlight in briefly.

We edited the Introduction highlighting the issues of recovery and maintenance of structures from the late 19^th – 20^th – 21^st centuries.

6. Authors are suggested to provide the analysis on the evaluation parameters for identifying proper method of reconstruction of 3D geometric shapes.

Specific comments 

Abstract

1. Line no. 17, 18, 19 should be rewrite for more clarification

The whole abstract was revised and rewritten, to better address the scopes of this work. Lines 17-19 were rewritten accordingly.

Introduction

Introduction section written properly so there isfew specific comment for introduction section

Line No. 74 heading should be write according subject of title

The State-of-the-art section was reduced, so as to concentrate the Related work on the most relevant section of supervised ML and CNN-based classification methods. The heading was edited accordingly.

Paragraph of lines from 88 to 114 should be rewrite there own language think and rewrite

This section was removed.

Paragraph of lines from 190 to 199 should be rewrite accoding their opnion.

This part has been rewritten to improve sentence structure in English (Lines 124-138).

Material

In topic topographic frame work paragraphs of lines from 251 to 258 should be rewrite

This part was rewritten, too (Lines 191-198).

Paragraph of lines from 260 to 268 should be rewrite also.

This paragraph was rewritten, improving the syntax of the English language (Lines 200-208).

In topic Range based survey paragraph of lines from 288 to 302 and from lines from 304 to 307 should be write according their own language.

Paragraph 3.2.4 was corrected (Lines 220-242) and the structure of the text was improved.

Results

Result and discussion section should be well written according execution steps.

We have added an introductory cap to Section 5 – Results and Discussion, in which we specify that, for each of the three phases identified as part of the proposed methodological approach, the results and related discussions are presented in the Section, respectively following: i) supervised ML for the classification of laser scan data; ii) automated masking of UAV images via DL for photogrammetric processing; iii) data fusion and construction of the BIM model (Lines 420-424).

Figures 8, 9, 19, 20, 32, 34 and 37 have been improved.

Conclusion

In conclusion also write drawback of automated segmentation and classification methods of surveying outputs to improve the interpretation and Building Information  Modeling from laser scanning and photogrammetric data.

We added a final sentence for the Concluding section (Lines 819-825):

The proposed workflow, by providing an already classified point cloud data, also speeds up the component inspection and modelling phase in the BIM environment, although the reconstruction of components is currently based on the recognition of simple geometries such as those detected by RANSAC. Extending the methodology to more complex classes of building components may require the implementation of more sophisticated shape reconstruction techniques, such as visual programming language algorithms and automatic modelling.

References

Some references are shown extra so plz remove extra references those not related to research paper. All citations should be cross check with references.

Unnecessary citations were removed from the paper, and the references have been reduced to a number of 47. All citations are cross-checked with references.

Reviewer 2 Report

The manuscript deals with the problematics of post-processing data from photogrammetry and laser scanning methods used to create a 3D model with the help of artificial intelligence. The authors' team is developing a 3D model of a steel structure which (if I understand correctly) they want to preserve as a heritage and make something like a snap image of the actual state of the construction for further deterioration analysis. The authors have done much work, and the output is fascinating. Authors also would like to use already trained neural networks and other algorithms for future classification of different structures, thus saving working time and costs.

Besides the fact that all this work is exciting and the authors have put in a lot of effort, and their own time in the research, this manuscript somehow lacks novelty. If there are nowel parts, they disappear. All the used techniques and outcomes are already known. It is hard to tell what in this manuscript is new and original. The average reader can easily get lost in the ideas because there are not described properly. It seems the authors tried putting together three approaches – scientific current contents article, survey, and case study in one manuscript.

The manuscript needs to be modified, as the actual state can be confusing for the reader. The question for authors is - what is the main output of this research? A 3D model of the construction? The trained neural network? A new set of rules for the identification of structural components? The merge of two methods? This needs to be clarified.

 

Also, the manuscript needs a mathematical backup as it only describes the outputs of each phase of research/survey/study.

 

Issues that need to be addressed:

1. The article is extremely long, and it contains unnecessary passages. There are many descriptions and paragraphs in which one or two sentences can describe the main idea.
2. The second chapter could be abandoned or shortened and merged with the Introduction or the third chapter.
3. Figure 1 is in the middle of the paragraph.
4. AI is sometimes spelt as IA
5. Different terms used in the manuscript (manuscript, paper, article).
6. In Figure 6 I believe you meant Perspective.
7. Figure 9 – DEEPL LEARNING.
8. There should be a dot instead of a comma when representing numbers with decimal parts.
9. The appendix could be included in the manuscript as a substitution for unnecessary parts.

 

Author Response

Dear reviewer,

We greatly thank you for the thorough and thoughtful comments provided on our submitted article. It has taken us a rather long time to complete the updated version, that we are submitting again to your attention. We made sure that each one of the reviewer comments has been addressed carefully and that the paper is revised accordingly.

We revised the structure of the paper in order to better clarify the distinction between the proposed methodological approach and the experimental results of its application on the case study, as well as the conclusions (in terms of strengths and weaknesses of the proposed method). We also incorporated more references in order to better illustrate the state-of-the-art section.

Please find attached the detailed responses to your comments. The latter are shown in black and our responses in red.

The manuscript deals with the problematics of post-processing data from photogrammetry and laser scanning methods used to create a 3D model with the help of artificial intelligence. The authors' team is developing a 3D model of a steel structure which (if I understand correctly) they want to preserve as a heritage and make something like a snap image of the actual state of the construction for further deterioration analysis. The authors have done much work, and the output is fascinating. Thank you for your interest in our research. Authors also would like to use already trained neural networks and other algorithms for future classification of different structures, thus saving working time and costs.

Besides the fact that all this work is exciting and the authors have put in a lot of effort, and their own time in the research, this manuscript somehow lacks novelty. If there are novel parts, they disappear. All the used techniques and outcomes are already known. It is hard to tell what in this manuscript is new and original. The average reader can easily get lost in the ideas because there are not described properly. It seems the authors tried putting together three approaches – scientific current contents article, survey, and case study in one manuscript.

The manuscript needs to be modified, as the actual state can be confusing for the reader. The question for authors is - what is the main output of this research? A 3D model of the construction? The trained neural network? A new set of rules for the identification of structural components? The merge of two methods? This needs to be clarified. Also, the manuscript needs a mathematical backup as it only describes the outputs of each phase of research/survey/study.

Based on your suggestions, as well as on the comments provided by Reviewer 1, we improved the structure of the paper and better stressed the aims and novelty of our work, by the following:

• We revised and rewrote the abstract, to better address the aims of the paper and to numerically indicate the results;
• Paragraph 1.1 on Research Aims was added and the whole Introductory part was revised. As such, the Introduction section concludes with the contributions of the research work presented in the article.
• Related work was separated from Section 2. State of the art: supervised ML and CNN-based classification methods. Previous Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century was removed to concentrate the Related work section on the State of the art of ML and CNN-based classification methods.

Issues that need to be addressed:

The article is extremely long, and it contains unnecessary passages. There are many descriptions and paragraphs in which one or two sentences can describe the main idea.

The second chapter could be abandoned or shortened and merged with the Introduction or the third chapter.

In the second chapter, we removed Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century, to reduce the Related work section, and synthesized the Introduction. Table 4 was removed, as deemed unnecessary, and the Table numbering was edited accordingly.

Figure 1 is in the middle of the paragraph. Corrected

AI is sometimes spelt as IA Corrected in various parts of the paper

Different terms used in the manuscript (manuscript, paper, article). We replaced ‘manuscript’ and ‘article’ with the word ‘paper’ wherever these words appeared.

In Figure 6 I believe you meant Perspective. Yes, we properly corrected.

Figure 9 – DEEPL LEARNING. Corrected. Other typographical errors of this Figure were corrected. Please note that, following the suggestions of other Reviewers, Figures 8, 19, 20, 32, 34 and 37 were also properly edited:

• Figure 8 - We added the measurement unit for the numbers associated to the color scale to the right.
• Figure 19 - We added the measurement unit.
• Figure 20 - We improved the readability of the image by augmenting the points size in the original point cloud.
• Figure 32 - We added the measurement unit.
• Figure 34 - We added the measurement unit.
• Figure 37 - We added the measurement unit.

There should be a dot instead of a comma when representing numbers with decimal parts. Corrected.

The appendix could be included in the manuscript as a substitution for unnecessary parts.

Appendix A (Figures A1 and A2) and B (Figure B1, Table B1 and Table B2) are included in the paper as supplementary documents that further ease the reader's understanding of our research but are not essential to the core argument.

Author Response File: Author Response.pdf

Reviewer 3 Report

Congratulations to the authors for this work!

The paper is very interesting, well written and structured.  It undoubtedly deserves publication.

I have provided a few little suggestions in the attached file. I also suggest using a larger font in the legends of the figures.

Comments for author File: Comments.pdf

Author Response

Dear reviewer,

We greatly thank you for the thorough and thoughtful comments provided on our submitted article. It has taken us a rather long time to complete the updated version, that we are submitting again to your attention. We made sure that each one of the reviewer comments has been addressed carefully and that the paper is revised accordingly.

We revised the structure of the paper in order to better clarify the distinction between the proposed methodological approach and the experimental results of its application on the case study, as well as the conclusions (in terms of strengths and weaknesses of the proposed method). We also incorporated more references in order to better illustrate the state-of-the-art section.

Please find attached the detailed responses to your comments. The latter are shown in black and our responses in red.

Congratulations to the authors for this work!

The paper is very interesting, well written and structured.  It undoubtedly deserves publication. Thank you for your review and interest in our work.

I have provided a few little suggestions in the attached file. I also suggest using a larger font in the legends of the figures.

The figures have been modified and improved accordingly with the suggestions. In the specific:

• Figure 8 - We added the measurement unit for the numbers associated to the color scale to the right.
• Figure 9 - We corrected many typographical errors.
• Figure 19 - We added the measurement unit.
• Figure 20 - We improved the readability of the image by augmenting the points size in the original point cloud.
• Figure 32 - We added the measurement unit.
• Figure 34 - We added the measurement unit.
• Figure 37 - We added the measurement unit.

Please note that, according to the suggestions provided by other reviewers, we improved the structure of the paper by:

• revising and rewriting the abstract, to better address the aims of the paper and numerically indicate the results;
• adding Paragraph 1.1 on Research Aims and revising the whole Introductory part;
• separating Related work from Section 2. State of the art: supervised ML and CNN-based classification methods. Previous Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century was removed to concentrate the Related work section on the State of the art of ML and CNN-based classification methods.

Author Response File: Author Response.pdf

Reviewer 4 Report

The article explores the complex topic of surveying and 3D modeling of complex grid structures. It confirms that AI-based techniques may reduce the time and modes of surveying and reconstruction. I recommend publishing the article after revision. （1）The figures should be improved. （2）Table 4 is not needed.

Author Response

Dear reviewer,

We greatly thank you for the thorough and thoughtful comments provided on our submitted article. It has taken us a rather long time to complete the updated version, that we are submitting again to your attention. We made sure that each one of the reviewer comments has been addressed carefully and that the paper is revised accordingly.

We revised the structure of the paper in order to better clarify the distinction between the proposed methodological approach and the experimental results of its application on the case study, as well as the conclusions (in terms of strengths and weaknesses of the proposed method). We also incorporated more references in order to better illustrate the state-of-the-art section.

Please find attached the detailed responses to your comments. The latter are shown in black and our responses in red.

The article explores the complex topic of surveying and 3D modeling of complex grid structures. It confirms that AI-based techniques may reduce the time and modes of surveying and reconstruction. I recommend publishing the article after revision. Thank you for your review.

（1）The figures should be improved. Figures 8, 9, 19, 20, 32, 34 and 37 have been improved as follows:

• Figure 8 - We added the measurement unit for the numbers associated to the color scale to the right.
• Figure 9 - We corrected many typographical errors.
• Figure 19 - We added the measurement unit.
• Figure 20 - We improved the readability of the image by augmenting the points size in the original point cloud.
• Figure 32 - We added the measurement unit.
• Figure 34 - We added the measurement unit.
• Figure 37 - We added the measurement unit.

（2）Table 4 is not needed. We properly removed Table 4 and edited the Table numbering accordingly

Please note that, according to the suggestions provided by other reviewers, we improved the structure of the paper by:

• revising and rewriting the abstract, to better address the aims of the paper and numerically indicate the results;
• adding Paragraph 1.1 on Research Aims and revising the whole Introductory part;
• separating Related work from Section 2. State of the art: supervised ML and CNN-based classification methods. Previous Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century was removed to concentrate the Related work section on the State of the art of ML and CNN-based classification methods.

Author Response File: Author Response.pdf

Reviewer 5 Report

this research focus on the grid structure construction combined with random forest and deep learning using UAV images and photogrammetric point clouds. details are clear and the results are encouraging. 

some words like 'intradox' and 'estradox' can be improved in figure 9.

Author Response

Dear reviewer,

We greatly thank you for the thorough and thoughtful comments provided on our submitted article. It has taken us a rather long time to complete the updated version, that we are submitting again to your attention. We made sure that each one of the reviewer comments has been addressed carefully and that the paper is revised accordingly.

We revised the structure of the paper in order to better clarify the distinction between the proposed methodological approach and the experimental results of its application on the case study, as well as the conclusions (in terms of strengths and weaknesses of the proposed method). We also incorporated more references in order to better illustrate the state-of-the-art section.

Please find attached the detailed responses to your comments. The latter are shown in black and our responses in red.

This research focus on the grid structure construction combined with random forest and deep learning using UAV images and photogrammetric point clouds. details are clear and the results are encouraging.  Thank you for your review and interest in our work.

some words like 'intradox' and 'estradox' can be improved in figure 9. We have changed the text in Figure 9 to the correct wording.

Please note that, according to the suggestions provided by other reviewers, we improved the structure of the paper by:

• revising and rewriting the abstract, to better address the aims of the paper and numerically indicate the results;
• adding Paragraph 1.1 on Research Aims and revising the whole Introductory part;
• separating Related work from Section 2. State of the art: supervised ML and CNN-based classification methods. Previous Section 2.1 – 3D surveying and modeling of grid structures of the late 19^th – 20^th – 21^st century was removed to concentrate the Related work section on the State of the art of ML and CNN-based classification methods.

Moreover, Figures 8, 9, 19, 20, 32, 34 and 37 have been improved as follows:

• Figure 8 - We added the measurement unit for the numbers associated to the color scale to the right.
• Figure 9 - We corrected many typographical errors.
• Figure 19 - We added the measurement unit.
• Figure 20 - We improved the readability of the image by augmenting the points size in the original point cloud.
• Figure 32 - We added the measurement unit.
• Figure 34 - We added the measurement unit.
• Figure 37 - We added the measurement unit.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Dear author´s team, thank you for your response and cover letter. Most of the issues were addressed, and problematic parts were corrected and updated. I still see this manuscript as too long, but the final decision should be on the academic editor, so I´m leaving this decision up to him. I recommend accepting this manuscript in its present form.