Keyframe Selection for Visual Localization and Mapping Tasks: A Systematic Literature Review

Round 1

Reviewer 1 Report

The reviewed article deals with a systematic review of the scientific literature dealing with the topic of keyframes selection in visual localization and mapping. The topic of visual localization and mapping has been addressed in the scientific literature for more than 12 years, but a systematic review of the literature has not yet been carried out. The evaluated article fills this gap and maps the given area of literature with a scientific approach. I consider the contribution of this article to be mainly the summarization of the basic concepts of visual localization and mapping in chapter 2.1, the statistical processing of topics and the number of scientific articles in chapter 4, and the classification of research approaches to the selection of keyframes and its statistical evaluation in chapters 4.1 to 4.4. Therefore, I consider this article relevant to the topic of visual localization and mapping. I consider the conclusions resulting from the presented research to be correct. The authors state the lacks identified from the literature review and thus open up space for further research in this area. The number of references directly related to the topic of visual localization and mapping is above average. I find the article very useful and I recommend to publish it in the presented form.

Author Response

Dear reviewer,

We sincerely appreciate the valuable feedback you provided on our manuscript, and we are grateful for the time and effort you dedicated to reviewing our work.

Best regards

Reviewer 2 Report

Date: 5/5/2023

Journal: Robotics

Title: keyframe Selection for Visual Localization and Mapping tasks: A Systematic Literature Review Comments:

 

This systematic literature review analyzes keyframe selection methods in visual localization and mapping tasks. Keyframes reduce the amount of data to be processed and eliminate useless information during optimization. The authors identify a lack of research on this topic and propose an updated categorization of the methods while highlighting the dependence on heuristics and the need for a systematic testing and validation process. The review follows a methodology proposed by Kitchenham and Charters and includes study selection criteria and a data extraction form. The paper concludes with a summary of findings and suggestions for future research. However, the authors should provide more detailed information and comprehensive comparisons. In conclusion, a minor revision is suggested before publishing this article.

 

1.     There is a lack of research on the keyframe selection topic within visual localization and mapping tasks.

2.     There is a deficiency in the literature regarding the validation and testing process, as well as a lack of rigorous tests, validation procedures, and public benchmarks, preventing a fair comparison of different approaches.

3.     There may be limitations in the selection process, as only articles that meet the inclusion criteria were considered, potentially leaving out relevant studies.

4.     Clarify the contribution and innovation of the presented robot design in more detail, especially in comparison to similar robots in the literature. What are the specific advantages and limitations of the proposed system, and how does it address the existing challenges and requirements for customer service robots?

5.     Provide more technical details and results about the implementation and evaluation of the autonomous navigation and gesture recognition systems, such as the performance metrics, datasets, algorithms, and computational resources used. How robust and accurate are these systems in different scenarios and conditions?

6.     Discuss the ethical and social implications of deploying humanoid robots in public spaces, particularly in terms of privacy, security, and human-robot interaction. What are the potential risks and benefits of using such robots, and how can they be managed and regulated?

7.     Explain the limitations and future directions of the presented work, such as scalability, adaptability, and usability. What are the potential challenges and opportunities for further research and development of customer service robots, and how can they address the evolving needs and expectations of customers and stakeholders?

 

 

 

The English of the title must be modified.

Author Response

Dear Reviewer,

We sincerely appreciate your valuable feedback on our manuscript and the time and effort you invested in reviewing our work. We have thoroughly considered your comments, and we would like to share our responses to address your concerns.

Point 1: There is a lack of research on the keyframe selection topic within visual localization and mapping tasks.

Response 1: As stated in the manuscript, to the best of our knowledge, this review represents the first comprehensive analysis of this particular topic. The results of the quality assessment clearly indicate that a significant number of studies inadequately report information on keyframe selection methods and their evaluation, or fail to directly address the underlying problem.

Point 2: There is a deficiency in the literature regarding the validation and testing process, as well as a lack of rigorous tests, validation procedures, and public benchmarks, preventing a fair comparison of different approaches.

Response 2: This observation is further reinforced by the study quality assessment, which not only helps weigh the significance of individual studies but also provides valuable guidance for future research recommendations. The majority of the quality assessment questions revolve around the validation and testing process. It is evident from the results that a significant portion of the works either partially addressed or failed to provide a positive response to these specific questions.

Point 3:  There may be limitations in the selection process, as only articles that meet the inclusion criteria were considered, potentially leaving out relevant studies.

We have heeded your suggestion and made a conscientious effort to enhance the manuscript's conciseness. While the Systematic Literature Review maintains scientific rigor by adhering to a well-defined methodology, it is crucial to acknowledge the inherent risk of bias in the process. Consequently, we have included a new section titled "Risk of Bias" that discusses the potential biases that may have influenced our conclusions.

Unfortunately, after careful consideration, we were unable to identify how Points 4, 5, 6, and 7 could be effectively incorporated into the discussion of this manuscript.

Kindly Regards

Reviewer 3 Report

The aim of this work is to identify, analyze and summarize existing methods in the literature for selecting keyframes in the context of localization and visual mapping using the Systematic Literature Review (SLR) as the basis of the method.

The paper is very well structured, very well written, in addition to having applied the SLR in a very rigorous way. Therefore, it presents an excellent contribution to the research area.

During the revision of the text, I did not find points to be improved in the text, precisely because of its quality. It's an excellent job!

Just as a suggestion to enrich the article, I have two suggestions:

- include figures to graphically demonstrate some important points of the keyframe selection process;

- include a taxonomy to compactly demonstrate the important areas and aspects revealed by the work.

Author Response

Dear Reviewer,

Thank you for your valuable feedback on our manuscript. We appreciate the time and effort you dedicated to reviewing our work. We have carefully considered your comments and made the following updates:

• In the introduction, we have added figures to illustrate the applicability of keyframe selection.
• In section 4.1, we have incorporated a figure that visually represents the newly proposed taxonomy in a hierarchical form.

Best regards.

Reviewer 4 Report

This review focuses on summarizing the contributions and limitations of the technical literature for key frame detection in visual localization and mapping tasks. I think this review is relatively comprehensive in its presentation, but there are some areas that need to be revised and added, and the following are my comments.

(1)   I believe that the contribution of the method proposed in the literature should be evaluated taking into account its field of application. The evaluation metrics and performance focus of similar methods in different fields are different, and this manuscript does not take this into account well when evaluating the literature.

(2)   I think that identifying the limitations of the literature should be accompanied by suggested solutions or pointing out the objective bottlenecks.

(3)   The evaluation of the literature in the manuscript seems to be a bit harsh, because different methods have different emphasis on performance, so various methods may be applicable to different applications, and the manuscript is too comprehensive in its requirements for the literature.

(4)   There is some value in the manuscript in summarizing the current state of research by counting the contents of the literature, but I think more theoretical analysis should appear in the manuscript to illustrate the strengths and weaknesses of the various approaches.

(5)   A more detailed description of the shortcomings of the heuristic algorithm for the key frame detection problem is suggested.

Author Response

Dear Reviewer,

We sincerely appreciate your valuable feedback on our manuscript and the time and effort you dedicated to reviewing our work. Your insights have been carefully considered, and as a result, we have included a dedicated section on further research in the manuscript. Furthermore, we would like to address your specific concerns by sharing our comments on them.

Point 1: I believe that the contribution of the method proposed in the literature should be evaluated taking into account its field of application. The evaluation metrics and performance focus of similar methods in different fields are different, and this manuscript does not take this into account well when evaluating the literature.

Response 1: One of the primary objectives of this review was to avoid confining itself to a specific application domain or algorithm, given the limited availability of literature reviews in this area. By adopting this broad scope, we aimed to facilitate a comprehensive understanding of the problem modeling and its unique characteristics within diverse application contexts.

Point 2: I think that identifying the limitations of the literature should be accompanied by suggested solutions or pointing out the objective bottlenecks.

Response 2: As highlighted in the conclusion, we argue that the limitations identified in the literature can be attributed to a notable gap in adequately addressing this problem, despite its acknowledged significance as a fundamental determinant of algorithm performance.

Point 3: The evaluation of the literature in the manuscript seems to be a bit harsh, because different methods have different emphasis on performance, so various methods may be applicable to different applications, and the manuscript is too comprehensive in its requirements for the literature.

Response 3: As addressed in Point 1, our primary focus was to comprehensively summarize the existing knowledge present in the literature, rather than limiting the scope to specific application domains. This approach was motivated by the observation that many visual-based methods for localization and mapping tasks are derived from similar principles, thus encompassing a broader understanding applicable across various domains.

Point 4: There is some value in the manuscript in summarizing the current state of research by counting the contents of the literature, but I think more theoretical analysis should appear in the manuscript to illustrate the strengths and weaknesses of the various approaches.

Response 4: Unfortunately, the absence of standardized validation methodologies poses a challenge in objectively identifying the strengths and weaknesses of different approaches. Conducting a direct comparison between methods is necessary to achieve this, which we intend to undertake in future research.

Point 5: A more detailed description of the shortcomings of the heuristic algorithm for the keyframe detection problem is suggested.

Response 5: As mentioned in the manuscript, the primary issue with heuristic methods lies in their dependency on fixed rules and thresholds, which may not consistently yield optimal or adaptable outcomes across diverse situations. This limitation is further emphasized in the further research section, underlining the need to explore alternative approaches that can overcome this challenge.


Best regards