Behavior Analysis for Increasing the Efficiency of Human–Robot Collaboration

Round 1

Reviewer 1 Report

The paper presents a behaviour analysis for improving the efficiency of human-robot collaboration. The paper is interesting and suitable for the journal. However, the following points need to be carefully addressed to improve the quality of the manuscript.

1) The main contributions of the paper should be better highlighted and commented, especially with respect to similar works previously published on similar topics.

2) The methodology lacks rigor. The description of the proposed approach and its implementation of the real experimental setup should be improved.

3) The results should be improved. The comparisons of the methods should be ehnanced by considering not only the total task time, but also additional fluency metrics, such as the number of robot stops, the robot idle time, the human idle time, ecc.

4) No insights into the path and trajectory planning of the robot are reported in the manuscript. It would be suitable to describe how the robot is controlled, how the trajecotories are planned and how the robot is stopped in case of a potential collision between human and robot in motion.

5) The literature review should be improved, by considering additional approaches to improve efficiency and productivity in a human-robot collaborative cell. Some suggested references are the following:

Scalera, L., Giusti, A., Vidoni, R., Di Cosmo, V., Matt, D., & Riedl, M. (2020). Application of dynamically scaled safety zones based on the ISO/TS 15066: 2016 for collaborative robotics. International Journal of Mechanics and Control, 21(1), 41-49.

Lin, C. H., Wang, K. J., Tadesse, A. A., & Woldegiorgis, B. H. (2022). Human-robot collaboration empowered by hidden semi-Markov model for operator behaviour prediction in a smart assembly system. Journal of Manufacturing Systems, 62, 317-333.

Himmelsbach, U. B., Wendt, T. M., Hangst, N., Gawron, P., & Stiglmeier, L. (2021). Human–Machine Differentiation in Speed and Separation Monitoring for Improved Efficiency in Human–Robot Collaboration. Sensors, 21(21), 7144.

Zhang, Z., Peng, G., Wang, W., Chen, Y., Jia, Y., & Liu, S. (2022). Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model. Sensors, 22(11), 4279.

 

 

 

Author Response

Enclosed please find the response.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper proposed a system based on evaluating human behaviors, including their weariness, adaptability, and preferred sequence that can increase the performance of individuals working with robots, background on previous studies about the body posture and movements estimation. With  behavior recognition, the author increase the efficiency of HR collaboration with some experiments. The innovations focus on  the behavior analysis, a human preference sequence assembly using POMDP and human adaptation ability based on LCs  and human fatigue during assembly as an observation factor. Together with LSTM, the experiments show its efficiency and improvement.

 

But, the paper can be refined and made some modifications:

1) the author should add some  cited references about LSTM to testify its  availability and the network structure should be illustrated to make readers understand it extensively;

2）the author can refine the introduction part to stress the main points of this paper, and arrangement should be added to methodize the content; in general, the Fig.1can be moved to other chapter;

3) there appears abbr. POMDP more than 20 times in the paper, what is its full spelling? the author should give us its full spelling when it appears for the first time;

4) there is not any reveal of the control computer, the author can give us the controller this system runs on to let us estimate the practicability of this method proposed;

The paper can be accepted with minor revisions.

Author Response

Enclosed please find the response.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The paper has been slightly improved with respect to the previous version.

I suggest discussing more in detail the advantages and disadvantages of the proposed approach, not only with respect to the previous work of the same authors, but also with respect to other approaches available in the literature.

The description of the experimental setup should be improved by reporting the robot model, the camera model, as well as the software used to control the robot, track the human and perform the experiments. 

No insights into the path and trajectory planning of the robot are still reported in the manuscript. It would be suitable to describe how the robot is controlled, how the trajecotories are planned (in terms of way points, motion law, values of velocities and accelerations, etc.) and how the robot is stopped in case of a potential collision between human and robot in motion (how the robot is stopped in case of a potential collision? It is not clear what scenario of safe collaboration is implemented, speed and separation monitoring, power and force limiting?). It is not clear whether all the risks concerning safety in human-robot collaboration have been considered, as for instace, the risk of gripping a finger of a hand of the operator, or clamping a hand between robot and table.

The overall style of the article should be improved: the quality of the figures is low and not sufficient for a journal publication. Please correctly cite the reference from where the figures have been taken (es. Figure 1). Figures 2 and 13 show a different robot model with respect to the one used in the experiments. Furthermore, the link of the supplementary video should be removed from the abstract.

The literature should be improved. I suggest adding the following references to the state-of-the-art review:

Application of dynamically scaled safety zones based on the ISO/TS 15066: 2016 for collaborative robotics. International Journal of Mechanics and Control, 21(1), 41-49, 2020.

Human-robot collaboration empowered by hidden semi-Markov model for operator behaviour prediction in a smart assembly system. Journal of Manufacturing Systems, 62, 317-333, 2022.

Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model. Sensors, 22(11), 4279, 2022.

 

 

Author Response

Reviewer 1

Concern:

 I suggest discussing more in detail the advantages and disadvantages of the proposed approach, not only with respect to the previous work of the same authors, but also with respect to other approaches available in the literature.

Address:

Thank you for your input. We have already given the mentioned suggestion on Section 3.1 system overview and the Table 2 – Comparison with another model

 

2. The description of the experimental setup should be improved by reporting the robot model, the camera model, as well as the software used to control the robot, track the human and perform the experiments.

Address:

Thank you for your suggestion. For the detailed experiment setup, we have already given the additional information in the revised manuscript in Table 6. For tracking the human we used Mediapipe library to extract the human feature information. We discussed that matter in Section 3.4

 

3. No insights into the path and trajectory planning of the robot are still reported in the manuscript. It would be suitable to describe how the robot is controlled, how the trajecotories are planned (in terms of way points, motion law, values of velocities and accelerations, etc.) and how the robot is stopped in case of a potential collision between human and robot in motion (how the robot is stopped in case of a potential collision?

Address:

• Thank you for your suggestion.
• The path and trajectory planning were described on the revised manuscript section 4.2
• We also already provide explanation on Section 3.2- Robot Action.

The reward system also provides safety measurements during the operation. An uncomfortable situation during collaboration with the robot will lead the robot to work in slow mode during collaboration. Some uncomfortable situations during assembly are:

(1) The human stops working with the robot;

(2) The human does not trust the ability of the robot;

(3) The human does not need the robot's help. 

(4) In case of collision, the robot will return to the idle position and re-assess the condition. The possible reward for this kind of situation is (-50) which leads to less effective HR collaboration.

 

4. It is not clear what scenario of safe collaboration is implemented, speed and separation monitoring, power and force limiting?). It is not clear whether all the risks concerning safety in human-robot collaboration have been considered, as for instace, the risk of gripping a finger of a hand of the operator, or clamping a hand between robot and table..

 

Address:

• Thank you for your concern. In this manuscript we don’t specifically mention the safety mechanism in Human Robot Collaboration. As this topic would sufficient enough to be a new full paper discussion. For this manuscript, we focus on the contribution as mentioned on section 2.4
• Speed and Separation (SSM) and Power Force Limiting (PFL) are two major topics in HR safety research. Each method provides a different approach based on context. Thus, as we mentioned in the previous comment, we don’t necessarily discuss safety context (either SSM or PFL-based) in this manuscript.
• For instance, we provide these videos as an example:

1. https://www.youtube.com/watch?v=9wJbrgnP-EM - A Speed and Separation Monitoring demo with a Universal Robots UR10 and a Microsoft Kinect v2 - https://www.researchgate.net/profile/Nick-Van-Oosterwyck/publication/326717734_Real_Time_Human_Robot_Interactions_and_Speed_Control_of_a_Robotic_Arm_for_Collaborative_Operations/links/5b6054bda6fdccf0b2037ce9/Real-Time-Human-Robot-Interactions-and-Speed-Control-of-a-Robotic-Arm-for-Collaborative-Operations.pdf
2. https://www.youtube.com/watch?v=xspk5b6kQrs - Stop State and Speed & Separation Monitoring
3. https://www.youtube.com/watch?v=-TC4WsdJJAM - Safe physical HRI: Toward a unified treatment of speed and separation monitoring together with power and force limiting, in 'Intelligent Robots and Systems (IROS) - https://arxiv.org/abs/1908.03046 (Paper)

These research are based on safety context, which focuses on how establishing a safety step and algorithm in HR Collaboration.

 

5. Please correctly cite the reference from where the figures have been taken (es. Figure 1). Figures 2 and 13 show a different robot model with respect to the one used in the experiments. Furthermore, the link of the supplementary video should be removed from the abstract.

 

Address:

Thank you for the address.

• We have already revised the figure quality and citation
• We have already removed the supplementary video from the abstract

 

 

 

6. The literature should be improved. I suggest adding the following references to the state-of-the-art review

 

Address:

Thank you for the suggestion. We have already added the mentioned papers on the revised manuscript as well as the comparison of the approach.

 

 

 

 

 

Author Response File: Author Response.docx

Round 3

Reviewer 1 Report

The paper has been improved with respect to the previous version. However, I still suggest checking English grammar again (especially in the latest parts added). Furthermore, the quality of the figures should be improved (Figures at pages 15 and 16 still report a different robot with respect to the one used in the experiments).

Author Response

Reviewer 1

Concern:

 

1. However, I still suggest checking English grammar again (especially in the latest parts added).

Address:

Thank you for your suggestion. We have already checked the latest parts.

 

2. Furthermore, the quality of the figures should be improved (Figures at pages 15 and 16 still report a different robot with respect to the one used in the experiments).

Address:

Thank you for suggestion. We have already improved the figures on pages 15-16. As the figure is shown below, we try to draw a robot similar to the real one because the manufacturer does not provide the CAD model.

Author Response File: Author Response.pdf