Supporting Human–Robot Interaction in Manufacturing with Augmented Reality and Effective Human–Computer Interaction: A Review and Framework
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsSee the attached file.
Comments for author File: 
Comments.pdf
Author Response
Comments 1: One point that it is misleading is the fact human-computer interaction (HCI) is somehow “neglected” in the title and abstract. The authors have stated the importance of HCI for HRI, see the “Good Design Principles for AR” component of their proposed framework (Figure 16). They even claim that (page 2): “There is a gap in research where HRI meets HCI”. I believe the title should
explicitly mention HCI. Something like this: “Enhancing Human-Robot Interaction via Human Computer Interaction and Augmented Reality: A Comprehensive Review and Framework”.
Response 1: Thank you for pointing this out and we understand where you are coming from so we have changed the title to better reflect the paper. The new title is: Supporting Human-Robot Interaction in Manufacturing with Augmented Reality and Effective Human Computer Interaction: A Review and Framework
Comment 2: Despite the suggestion above to stress HCI, and even the article states that there is a deficiency in the incorporation of HCI principles into HRI research, as I have just mentioned above, and, moreover, the article highlights the significance of user interface (UI) design, it falls short of
thoroughly examining how these principles can be systematically utilised to improve AR systems in
the context of HRI.
Response 2: Thank you for this comment. We have added a few sentences to further clarify how UI/UX principles are applied in the HCI field to improve AR, and why introducing these principles to the HRI field would be beneficial. These sentences can be found in section 6, paragraph 1, lines 602-608. Additionally, Section 8 provides two case studies that employ the proposed structure and utilize UI/UX practices as examples of how the frameworks can be applied in HRI.
Comment 3: Figures 1 and 2 are problematic. What are the sources (scientific databases: Web of Science, IEEE Xplore, …) for these figures? Moreover, it is strange that, for instance in the year 2023, we see more than 2 articles in Figure 2 and only 2 in Figure 1. This might be a flaw of the process to
collect the relevant literature for this review.
Response 3: We understand your concerns regarding the lack of clarity in the literature collection process. As a result, we have rewritten the Methods section to specify the databases and registers used in the search, which can now be found in Section 2. Figures 1 and 2 previously showed trends for two different groups of papers, which is why the numbers differed for the year 2023. However, we agree that the figures were confusing, so we have removed them and replaced them with a PRISMA diagram to illustrate our literature gathering and screening process. The PRISMA diagram is labeled Figure 1 and can be found in Section 2.
Comment 4: I suggest changing the components names of the framework in the text to match the names in Figure 16. On page 19, then change the text of the following paragraph: “The framework, visualized in Figure 16, is divided into three key components: Collaboration, Implementing HCI Principles for Good AR Design, and Evaluation and Continuous Improvement.” Thus, replace these names in red with the components names in Figure 16 (Key Components of ...).
Response 4: Thank you for pointing this out; this was an oversight from an earlier draft of the paper. We have updated the sentence to align with Figure 16, as you suggested. The revised sentence can be found in Section 8, paragraph 1, lines 827-829.
Comment 5: Figure 19 is not mentioned in the text.
Response 5: Thank you for pointing this out. We have added a sentence to reference Figure 19, which can be found in Section 8, paragraph 9, lines 883-885.
Reviewer 2 Report
Comments and Suggestions for AuthorsThis paper has examined the significant advancements in integrating augmented reality (AR) into human-robot interaction (HRI), identified pivotal challenges, and proposed a comprehensive framework designed to effectively address them. It has positive implications for improving assistance, trust, and safety in AR systems in HRI, but there are some statements in the article that are confusing and require further revision or explanation.
Lines 27-28, Figures 1 and 2 illustrate the annual number of publications cited in this paper. Does this refer to the entirety of the literature analyzed by the four researchers described in the article? The meaning of this is unclear, and an explanation would be greatly appreciated.
Please clarify whether the papers referenced in Figure 2 were selected from Figure 1. If this is indeed the case, please explain why the number of papers in Figure 2 for multiple years (e.g., 2002, 2012) is greater than the number of papers for the same years in Figure 1.
On page 2, lines 45, please briefly explain the differences on the HRI and HCI studies and how it relates to your study, which will help the reader better understand the full text.
On page 3, lines 59-61, the term "more than 115" is insufficiently precise; a more exact figure should be provided. In section 2, Materials and Methods, the screening criteria for the literature are not adequately addressed. Please provide the necessary information to fill this gap in the discussion.
Chapter 4 provides a cursory overview of the challenges in HRI. It would have been beneficial if Chapter 5, which delineates the function of AR in bolstering collaboration, trust, and security in HRI, had been accompanied by detailed citations of the current challenges it faces.
Please explain why Chapter 7.2 addresses the use of collision warnings for virtual elements rather than the second point of the proposed framework of good design principles for AR.
Author Response
Comment 1: Lines 27-28, Figures 1 and 2 illustrate the annual number of publications cited in this paper. Does this refer to the entirety of the literature analyzed by the four researchers described in the article? The meaning of this is unclear, and an explanation would be greatly appreciated
Response 1: Thank you for your comment. We agree that Figures 1 and 2 were confusing, so we have replaced them with a PRISMA diagram to better explain the literature collected for this paper. The PRISMA diagram and its explanation can be found in Section 2.
Comment 2: Please clarify whether the papers referenced in Figure 2 were selected from Figure 1. If this is indeed the case, please explain why the number of papers in Figure 2 for multiple years (e.g., 2002, 2012) is greater than the number of papers for the same years in Figure 1.
Response 2: The papers referenced in Figures 1 and 2 were different, which explains the varying numbers for the same year in both figures. These figures have been removed and replaced with a PRISMA diagram to better clarify the literature collection process for this paper. The PRISMA diagram and its explanation can be found in Section 2.
Comment 3: On page 2, lines 45, please briefly explain the differences on the HRI and HCI studies and how it relates to your study, which will help the reader better understand the full text.
Response 3: Thank you for your comment. Additional information explaining the differences between HRI and HCI has been added to Section 1, paragraph 4, lines 44-52.
Comment 4: On page 3, lines 59-61, the term "more than 115" is insufficiently precise; a more exact figure should be provided. In section 2, Materials and Methods, the screening criteria for the literature are not adequately addressed. Please provide the necessary information to fill this gap in the discussion.
Response 4: We agree that the screening criteria were not accurate enough. Therefore, the methods has been rewritten to include the exact numbers, databases, and registers used to gather the literature. The rewritten methods can be found in Section 2.
Comment 5: Chapter 4 provides a cursory overview of the challenges in HRI. It would have been beneficial if Chapter 5, which delineates the function of AR in bolstering collaboration, trust, and security in HRI, had been accompanied by detailed citations of the current challenges it faces.
Response 4: Thank you for your comment. Additional information on the challenges in HCI has been provided and can be found in Section 4, paragraph 1, lines 342-347.
Comment 5: Please explain why Chapter 7.2 addresses the use of collision warnings for virtual elements rather than the second point of the proposed framework of good design principles for AR.
Response 5: We believe the overall structure of the paper may have been confusing, so we have added subsections to Section 6 to ensure all aspects of the proposed framework are thoroughly explained. Additionally, we have moved the 'Evaluation' and 'Continuous Improvement' sections to Section 7, so that all parts of the proposed framework are covered before Section 8, which discusses the implementation of the framework.
Reviewer 3 Report
Comments and Suggestions for AuthorsPlease read the attachment. Thank you.
Comments for author File: Comments.pdf
Author Response
Comment 1: Keywords: Please provide between 5 and 10 keywords, and do not repeat the one
on the title.
Response 1: Thank you for your comment, new keywords have been provided, they can be found on lines 13 and 14.
Comment 2: Figure 6 is too small. Please increase its size to more significant.
Response 2: The figure's size has been increased from 7 cm in width to 11 cm. It is now labeled Figure 5 and can be found in Section 3, under line 385.
Comment 3: Figure 10: please change the colors of the backgrounds of the figure to make the
figure more straightforward for readers to read.
Response 3: The figure has been updated with more contrasting colors and is now labeled Figure 9. It can be found in Section 5, under line 438.
Comment 4: Figure 18: This figure's word and number size are too small for the reader to read.
Please revise.
Response 4: Two graphs have been removed from Figure 18 to allow for a larger, more readable image. The updated Figure 18 is now in Section 8, below Figure 17.
Comment 5: Figure 24: it is too blurry.
Response 5: Figure 24 has been removed as per another reviewer's request.
Comment 6: Tables 1-2: table titles should be above their table.
Response 6: Thank you for pointing this out. All table titles have been placed above the tables. The tables can be found in Sections 5 and 8.
Comment 7: Please provide detailed case studies or examples demonstrating how the proposed
framework has been applied or could be applied in different HRI scenarios.
Response 7: Section 8 provides two previous works and one ongoing study that employs the proposed framework. It has been rewritten to show evidence of the framework’s effectiveness and to offer insight into the ongoing study.
Comment 8: Please discuss potential limitations of AR in HRI, such as technical constraints
or user acceptance issues.
Response 8: Thank you for your comment. A limitations section has been added to address the limitations of AR in HRI. It can be found in Section 9, lines 950-982.
Comment 9: How can the proposed framework be adapted to accommodate the rapid
technological advancements in AR hardware and software for Human-Robot
Interaction?
Response 9: Thank you for your question. We present the metrics useful for understanding the impact of AR in HRI in section 7. Followed by two case studies that use this framework and show evidence of the usefulness of this framework in enhancing situational awareness and safety. While we understand that our framework has been implemented using current hardware and software, the framework is conceptual in nature. So it should not have any barriers for implementation on future hardware and software. If anything with better hardware and software the fidelity of sensory information and better design of these future devices should enhance the user experience.
Comment 10: What metrics or methodologies can be used to assess the long-term impact of
AR-enhanced HRI systems on trust and safety in high-stakes environments, such
as healthcare or industrial robotics?
Response 10: Thank you for your questions. Information on long-term metrics and limitations has been added to Section 9, paragraph 3, lines 972-982. We have limited the scope of this paper to HRI in Manufacturing. The new title is: Supporting Human-Robot Interaction in Manufacturing with Augmented Reality and Effective Human Computer Interaction: A Review and Framework.
Reviewer 4 Report
Comments and Suggestions for AuthorsThe paragraph describing the graphs on references and publication trends (lines 26 to 28) should be moved to a more appropriate section later in the document, allowing for better flow within the overall analysis.
All databases consulted in the study must be explicitly indicated to ensure transparency in the search and selection process (line 64).
A detailed description of all search strings should be included to allow the reader to replicate or evaluate the bibliographic selection process (line 66).
As a suggestion, it would be beneficial to include a diagram that visually explains the study selection methodology, such as the PRISMA diagram (https://doi.org/10.1186/s13643-021-01626-4), improving clarity and methodological rigor.
Figure 24 (UEQ) does not significantly enhance the analysis, and its removal or a deeper review of its usefulness within the study context is recommended (line 864).
Author Response
Comment 1: The paragraph describing the graphs on references and publication trends (lines 26 to 28) should be moved to a more appropriate section later in the document, allowing for better flow within the overall analysis.
Response 1: The graphs and the accompanying text were removed because they were confusing and elicited similar comments from other reviewers. For overall clarity, they have been replaced with a PRISMA diagram, as you suggested. The PRISMA diagram is labeled Figure 1 and can be found in Section 2.
Comment 2: All databases consulted in the study must be explicitly indicated to ensure transparency in the search and selection process (line 64).
Response 2: Thank you for your comment. All databases and registers have been listed in Section 2, paragraph 1, lines 71-75.
Comment 3: A detailed description of all search strings should be included to allow the reader to replicate or evaluate the bibliographic selection process (line 66).
Response 3: All string searches have been added and listed by researcher for transparency and can be found in Section 2, paragraph 1, lines 76-86.
Comment 4: As a suggestion, it would be beneficial to include a diagram that visually explains the study selection methodology, such as the PRISMA diagram (https://doi.org/10.1186/s13643-021-01626-4), improving clarity and methodological rigor.
Response 4: Thank you for this suggestion! A PRISMA diagram has been created and added to Section 2 to better explain the literature gathering and screening process.
Comment 5: Figure 24 (UEQ) does not significantly enhance the analysis, and its removal or a deeper review of its usefulness within the study context is recommended (line 864).
Response 5: Figure 24 has been removed from the paper as per your suggestion.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsI believe the article is now ready for publication, after the authors have implemented my previous remarks.
