Autonomous Critical Help by a Robotic Assistant in the Field of Cultural Heritage: A New Challenge for Evolving Human-Robot Interaction

Round 1

Reviewer 1 Report

- The paper does not consider the long history of museum guide robots since Rhino, Minerva, Cicerobot, to mention a few of them. The authors should better frame their paper in this line of research.

- The authors should clearly address what are the main contributions of the paper and the main outcomes.

- Most important, the paper should address the fact that the study does not consider the user satisfaction of a control group of people not using robots at all, or using audio-guides or tablets.

Author Response

Thank you for the precious comments to our work, they were very helpful in order to make the work much more complete. We hope to answer to all your doubts. We will answer point by point to each issue.

Point 1- The paper does not consider the long history of museum guide robots since Rhino, Minerva, Cicerobot, to mention a few of them. The authors should better frame their paper in this line of research.

Answer 1: We have modified the introduction by introducing also a “Related Work” section in which our work has been framed in the line of the research proposed by you. In any case, our experiment refers to a model of help that is not found in previous experiences and works. The novelty lies in the fact that in our model the system solicits a request from the user but then analyzes it critically, also taking into account the actual collections of the museum and how these can best satisfy the profile that the system has built of the same user.

Point 2- The authors should clearly address what are the main contributions of the paper and the main outcomes.

Answer 2: The main contribution of our work consists in investigating the possibility of offering help that does not necessarily correspond to the explicit and declared request of the user. This kind of collaboration, which tries to offer an answer to protect the interests and goals of the user that the same user is not always able to perceive, represents a novelty in the panorama of human-robot collaboration. The satisfactory results we have witnessed show how this experiment, albeit preliminary, goes in the right direction. We introduced this concept also in the section “Contribution”, already present in the paper.

Point 3- Most important, the paper should address the fact that the study does not consider the user satisfaction of a control group of people not using robots at all, or using audio-guides or tablets:

Answer 3: Since we are facing a preliminary experiment, in fact in our case the so-called "control group" can be referred to cases in which the system decides to make a choice without critical help but, using our technical language, as a "literal" help. The comparison between critical help and literal help in this sense reveals interesting evaluations.

Reviewer 2 Report

the following issues should be handled before final acceptance:

1. there is no related work in this paper. the authors must prepare an overview of similar work from other researchers.

2. the applied refrences are almost old, which shows the topic is not very interesting to the community of research

3. representation of the results is very weak. The authors have more than 4 research questions. however, there is only one digaram to show the results. different aspects of the results should be disscussed more and clear.

4. in such studies, beside the plan for experiments, there is a need of discussion about limitations and methods for preventing bias. the threat to validity of the experiment design should be discussed in a separet subsecion

Author Response

Thank you for the comments to our work, they proved to be very useful in order to improve our work. We hope to answer to all your doubts. Below we are going to answer the questions point by point:

Point 1: there is no related work in this paper. the authors must prepare an overview of similar work from other researchers.
Answer 1: we modified the introduction by providing a section named “Related Work”

Point 2: the applied references are almost old, which shows the topic is not very interesting to the community of research

Answer 2: After introducing the “Related Work” section we inserted much more recent works. Among the “older” works, we have considered only the ones that are much cited, given their relevance.

Point 3: representation of the results is very weak. The authors have more than 4 research questions. however, there is only one diagram to show the results. different aspects of the results should be discussed more and clear.

Answer 3: As we discussed in the paper, we tried to answer to two research questions:
    RQ1: How risky/acceptable is the critical help compared to the literal help? Does the heuristic proposed help to make this help much     more acceptable?
    RQ2: Given the risks that the critical help in any case determines, in what situations and how much critical help can be useful?

At the end of each visit to the museum, the robot offers the user a questionnaire consisting of 5 questions (Q1, Q2, Q3, Q4, Q5). These questions are designed in order to understand what is the impact of the robot’s ability to propose to the user a tour in fact different from what she expected. In this experiment the questions Q1, Q2, Q3 have been asked to contextualize the user and to ensure the user could focus on specific questions, before answering to the questions Q4 and Q5. In this way we try to get the user’s attention to the contents of the tour and therefore focus on these and not only on the quality of the interaction with the robot. In conclusion, the relevant data to analyse are those one emerged by the answers to questions Q4, Q5. These results allow us to answer to the two research questions mentioned above, and they let us to conclude that, despite the literal help, compared to the critical help, results to be more satisfactory for users, in most cases where users have received critical help, they have evaluated positively the museum tour recommended by the robot. This result is particularly relevant by virtue of the fact that users do not know the reasons that led to a choice different from the one they expected. Despite this, even though they were surprised by the tour recommended, they maintain high levels of satisfaction, after the visit of the exhibition. 

Point 4: in such studies, beside the plan for experiments, there is a need of discussion about limitations and methods for preventing bias. the threat to validity of the experiment design should be discussed in a separeted subsection

Answer 4: We integrate the paper with a section named “Experiment Limitations”.

Round 2

Reviewer 1 Report

The authors thoroughly addressed my previous concerns. Now the paper is far better.

Author Response

Dear Reviewer,
Thank you for the efforts spent in helping us to improve the manuscript. We hope that the manuscript is now acceptable for publication. Please do not hesitate to contact me if there are any questions.

Author Response File: Author Response.pdf