Programming of Industrial Robots Using Virtual Reality and Digital Twins

Round 1

Reviewer 1 Report

Line 45: Assumed the authors meant “representation” instead of picture?

Line 46: Define artificial reality (in the context of manufacturing engineering)

Line 75: the term “low skills personal” would be more appropriate

Line 82: Typo: Pperator should be Operator.

Line 111: Define what “Learning skills” are for a robotic station

Line 119 to 127 (section 2.4): the explanation of AR and its use to achieve robot programming is vague. It is acceptable in its form, since the focus of the paper is not AR. However, the overall quality of the publication would benefit from improving this section.

Line 208-210: What is form in the sentence “virtual cleaning of the form”, “movements of actual form” etc.?

 

Conclusions: 

The overall concept behind this wok is relevant to the industry, as programming or industrial robots remains indeed a domain in which improvements to the engineering process can be made. 

The concept of programming by demonstration using motion tracking is not new (Fereirra 2014, Ge 2013). Using a VR set up is interesting because of its simplicity and affordability. 

However, the use case application and conclusions are superficial at best. The presentation of results stops at the provision of TCP pose information (position and orientation) captured by the VR tracking system (fig 12). No qualitative analysis is provided (i.e. accuracy of the system, repeatability, resolution and frequency of data capture) and no information is given as whether the capture of the apparently complex task was successful. 

Finally, the translation of this information to a usable (i.e. elbow configuration. limit joints value, avoidance of collisions with other cell components, etc) robot program is not described of even mentioned at all, making this work (in its current form) of very little value in the context of practical industrial application.

It is suggested that elements related to the next steps of the research are provided in the conclusion section before publication.

 

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

The work in this paper gives an idea (and its prototype) using digital twins for robotic mould cleaning, the idea seems to be quite good and works the way you intended. English expression in manuscript is good and understandable. However, the data in test-result section is not enough to support author's idea had been fully validated, more evidence or context needs to be given in section 4.

 

Regarding the details over the paper, the reviewer has the following comments:

 

In Figure 1, it is obviously existing great differences in (a) the actual mould and its digital twin: (b) virtual mould. They should be identically same, shouldn’t it ? And why?

 

Regarding digital-twin system, the reliability of the functionalities provided by the virtual robot is highly dependent on the level of physical and dynamic consistency to the real robot (such as deformation cause by various temperature or stress loading). In this work, what is the techniques to capture the real robot specific features for modeling the cyber robot ? Please make more detailed description if possible.

 

As for test result for author's idea, it is lack of data to show the replicability of tool-path between virtual and real robot, the only data provided in manuscript is the position and orientation of the virtual robot. Highly suggest authors to add real robot’s behavior (corresponding to tool-path generated in the virtual twin) on your manuscript for validation.

 

The authors could discuss some other more scenarios where their digital twin solutions could be used, as well as their limitations.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

The article presents a method of programming industrial robots using virtual reality and digital twins. From my point of view, I really don’t see any large scientific contribution of the paper. The authors spent more than 1/3 of the paper on the Introduction and Robot Programming Methods sections. These sections integrate the figures such as Figure 3 and Figure 4 that we can find in any old book about robots. Moreover, Robot Programming Methods are well known to anyone and authors shouldn’t spent so much place in the paper explaining them. In the rest of the paper authors superficially presented the idea of functioning of the virtual reality system. I’m not sure if they developed or investigated anything, since there is no details about developed software architecture or interfaces. Using the Robot Studio with VR is nothing new and is already part of the commercial ABB solution https://youtu.be/JnxHUcvUhU0. To show new application of this solution doesn’t give enough value to be a presented as a journal paper. Also the conclusion is very superficial and without future work.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 4 Report

 

Introduction and literature overview:

There is a nice overview of Digital Twins but when it comes to industrial robotics combination with VR – additional research on literature is required, like where is the combined use of those three technologies (VR, DT, and Industrial Robotics). Such kind of additional description can be beneficial to this paper. There is a number of papers available in the connection of virtual and physical worlds with Industrial robot use cases.

Method and experiment:

Paper describes the interesting idea of guiding robots with the assistance of VR headset and saving achieved robot routine as a sequence for the task. However, idea exploits ABB RobotStudio with embedded Oculus support from ABB. A more comprehensive description of methodology and expanding this method to other software and robot types could be included in this section. Right now, the content of this paper is enough for publishing in some conference proceedings but unfortunately not for journal publication.

Reviewer suggestion is to enlarge the methodology part and add more use cases not limited to ABB, as right now it is not clear scientific research and development contribution.

Moreover, it would be great to see in an evaluation of how much time can be saved by implementing the proposed method and what are the system advantages and limitations.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

As I can see, there are several changes and improvements. However, I still don’t see any significant scientific contributions in the paper.

Authors wrote “Reviewers always pay attention to the appropriate exposure of the issue and provide a background for the presented research.” That is correct, but why to pay attention on something, such as Sektion 2, that is already well known to anyone. With one simple reference on the hundreds of papers that summarize the Robot Programming Methods, you could save the space and dedicate the entire section 2. only to the Programming Using Augmented Reality issues and SOTA. In this way would the section have more sense considering that the focus of your paper is the Augmented Reality anyway.

Authors state in their response that they used the standard ABB software and involved a physical object that according to them makes the implemented task more realistic. Nevertheless, this is still usage of standard software in another application and nothing new.

I appreciate that that authors expanded the conclusions and  gave the objectives of further work.

To sum up, I'd like to say that I still think that this paper should be more improved and show more than just an application to be published as a journal paper.

Author Response

Dear Reviewer,

Thank you for taking the time to read the revised manuscript and submit your comments. Answers to your comments are listed below.

 

Reviewer: Authors wrote “Reviewers always pay attention to the appropriate exposure of the issue and provide a background for the presented research.” That is correct, but why to pay attention on something, such as Sektion 2, that is already well known to anyone. With one simple reference on the hundreds of papers that summarize the Robot Programming Methods, you could save the space and dedicate the entire section 2. only to the Programming Using Augmented Reality issues and SOTA. In this way would the section have more sense considering that the focus of your paper is the Augmented Reality anyway.

 

Answer: Section 2 has been reduced to subsection 1.4 and now has less than half a page.

Reviewer: Authors state in their response that they used the standard ABB software and involved a physical object that according to them makes the implemented task more realistic. Nevertheless, this is still usage of standard software in another application and nothing new.

(...)

To sum up, I'd like to say that I still think that this paper should be more improved and show more than just an application to be published as a journal paper.

Answer: In our Department, we deal with robotization of special processes for the high technology sector, mainly the aviation sector. That is why our works are of application nature. It can be said about most application solutions that they use standard commercial solutions. However, the authors believe that just combining solutions in a new way and showing that they are applicable is valuable. The more, that commercial software solutions enabling such an application have only been available for a year. We try to publish our achievements as soon as possible so that they do not become outdated after two or three years. Therefore, at this stage, we do not have to present other applications that would extend the article, nor do we have a comparison with commercial solutions of other companies, because they do not exist. We work on systems offered by other leading companies such as KUKA (Sim Pro software), Kawasaki (K-Roset software) and FANUC (Roboguide software), but none of them enables the implementation of the presented programming method with VR goggles. KUKA is just working on such a solution.

Corresponding Author: P. Gierlak

 

Reviewer 4 Report

Thank you for improving the paper.

It is answered why it is being used ABB RobotStudio, however, I believe, use-case should be enlarged not only with Industrial Brand manufacturing software. Some simulation software and open-source packages give the ability to control machines from VR already and also evaluate solution from other perspectives. 

The conclusion is now more clear and interesting to read. 

Those are just notes and ideas for future research.

Author Response

Dear Reviewer,

Thank you for taking the time to read the revised manuscript and submit your comments.

The article presents an approach based on technical solutions dedicated to industry. In our department, we deal with the development of special processes for the aviation industry, and our partners are international concerns from the high technology sector. Therefore, we do not deal only with simulation solutions or open-source software that cannot be later used in the industry due to commercial use and no guarantee of proper operation. We cannot afford to offer this solution to an industrial partners. We have several large projects a year and we just can't test solutions that will never be implemented. That is why we do not have such solutions to present in the article. I hope the reviewer shows understanding in this aspect.

The article made one more significant change at the request of a second reviewer. Chapter 2 on robot programming methods was reduced and only one section in the introduction was devoted to it.

Corresponding Author: P. Gierlak