Psychophysiological Methods to Evaluate User’s Response in Human Robot Interaction: A Review and Feasibility Study

Abstract

Implementing psychophysiological measures is a worthwhile approach for understanding human reaction to robot presence in terms of individual emotional state. This paper reviews the suitability of using psychophysiological assessment in human-robot interaction (HRI) research. A review of most common psychophysiological parameters used in a controlled laboratory setting is provided and advantages and challenges of their utilization in HRI experiments are described. Exemplar studies focused on the implementation of psychophysiological measures for the evaluation of the emotional responses of the participants to the robots’ presence are described. Based on the reviewed literature, the paper also describes the results of our own research experience to make the most of the emerged recommendations. We planned and performed a study aimed at implementing psychophysiological measurements for assessing the human response of two groups of older adults (Healthy vs. Mild Cognitive Impairment subjects) towards a telepresence robot. Finally, the paper provides a summary of lessons learned across the field in using psychophysiological measures in HRI studies.

1. Introduction

Humans generally interact with robots in the same way they might interact with other people, establishing social relationships and emotional ties with them [1,2,3,4]. One of the main characteristics relative to assistive social robotics is that a robot proactively engages users in a social manner creating an interaction with the person for the purpose of giving assistance and support in certain activities of daily living and care [4,5,6]. Gross et al. [7] report that the application of assistive social robots for domestic use has gone one step further towards the introduction of robotics technologies in real homes.

As assistive social robots are being designed to become a part of our life, an extremely important issue regards the development of methods to assess user’s experience with a robot, while understanding how humans feel during their interaction with it [8]. A robot operating at home, in nursing care or in a rehabilitation unit is required to have determined social capabilities, like natural language communication skills or emotion expression and/or perceptual skills. Such features, in fact, maintain and facilitate the interaction with human users [9,10]. How people respond and react to the robot’s social skills, and in general, to other features such as its functionality, role and appearance, is an important aspect of social interaction, which affects the acceptance and use of the robot. In addition, it is difficult to imagine social interactions occurring without emotional responses and therefore the analysis of the interactions between a human and a robot should be based also on a clear understanding of the human’s reaction to the robot in terms of emotions. An emotional response against a stimulus, such as the daily interaction at home with a robot, implies an involvement of the central nervous system (CNS) through multiple cortical and subcortical regions and an activation (in terms of arousal) of the autonomic nervous system (ANS), which can be registered by physiological measurements [11,12,13,14]. In brief, the emotional states can be categorized along two main dimensions, one that explains the valence of the emotion (in a continuum of pleasantness-unpleasantness) and another that refers to the corresponding level of arousal/physiological activation. According to the “circumplex model of affect” [15], every emotion can be explained as a linear combination of valence (positive or negative) and intensity of activation (arousal). An additional dimension (dominance/control) has been frequently suggested for emotions with the same level of arousal and valence (e.g., fear and anger), but that are distinguishable in everyday life [16].

In this paper, the main focus is on the psychophysiological measurements of human response to a robot. Psychophysiological measures are considered one of the main methods of assessment used for human studies in human robot interaction (HRI) together with self-report, behavioral measures and task performance analysis, as Bethel et al. report in [8]. In this survey, the authors provide a detailed overview of the process of designing and planning user evaluation based on a psychophysiological approach applied to detecting and/or identifying users’ emotions, modifying in real-time robot control and behavior, as well as evaluating the user’s reactions to technology. The aim pursued in this review is to expand the knowledge on the assessment users’ reactions to technology by means of psychophysiological measures as reported in [8]. In the scope, we have taken into account recent researches aimed at determining how humans react in terms of physiological response during an interaction with a robot in order to establish the real effectiveness of human-robot interaction. The comprehension of these aspects will encourage users to collaborate and share the environment with a robot designed, for instance, to provide them with social, health-care and rehabilitative services.

The paper is organized as follows: Section 2 briefly presents a description of psychophysiological terminology along with a survey of applications in the human-computer interaction research. Advantages and disadvantages of using psychophysiological measures are also reported. Section 3 focuses on the most common psychophysiological measures used in human-robot interaction studies in a controlled laboratory setting. Section 4 provides an overview of the most recent studies in the field of HRI that apply psychophysiological methods to evaluate the human’s reaction to a robot in terms of affective-emotional response. Section 5 presents our feasibility study focused on the assessment of the human response to a telepresence robot in a group of healthy older adults and in a group of older patients suffering from mild cognitive impairment (MCI). Section 6 summarizes the results of both the reviewed literature and of our study, leading to the final discussion.

2. Psychophysiology for Human Studies

Citing from [17]: “Psychophysiology concerns the study of cognitive, emotional and behavioral phenomena as related to and revealed through physiological principles and events”. Through the monitoring and interpreting of physiological signals during the interaction between a human user and an assistive robot, it is possible to obtain information about the user’s inner experience in terms of affective and cognitive states [8,18,19].

Methods based on psychophysiological assessment have been previously used in human computer interaction (HCI) to evaluate users’ cognitive workload [20], emotional responses [21], and cognitive-emotional states [22] during interaction with a technology. A significant increase of electromyography activity and hearth rate, and a significant decrease in blood volume pulse were found in research focused on individual’s response to audio and video degradations in videoconferencing software [23]. Scheirer et al. [24] used electromyography activity and blood volume pulse to distinguish subjects’ frustration during the utilization of a pre-programmed mouse delay. Particularly, the psychophysiological game evaluation measures the effects that games have on the players [25], how people respond to game features [26], users’ experience and emotional engagement [27]. Mandryk et al. [28] analyze user experience with entertainment technology. Participants in their experiment report higher electromyography and electrodermal activity when playing against a friend than against a computer. In another study, Mandryk and Atkins [22] evaluate different methods for modeling emotion using physiological data through two fuzzy logic approaches. The first fuzzy logic model transformed electrodermal (GSR), electromyography (smiling and frowning) and cardiovascular activity (HR) into arousal and valence. The second model transformed arousal and valence into five emotions. Their results prove the reliability of physiological metrics to model emotional user’s experience with interactive play technologies.

Psychophysiological measures have been more recently used in HRI human studies [8] in combination with subjective research methods, as behavioral and self-report measurements, in order to achieve a more complete and reliable knowledge of the person’s experience. Generally, when a person communicates his/her own personal opinion about a research issue, as in our case about the interaction with a robot, this may be affected by mood and/or social desirability. In addition, observer bias may have an effect on participants’ responses and consequently reactions or behaviors may not reflect the exact feeling of the persons involved in an evaluation study of HRI [8,29].

The psychophysiological measures offer numerous advantages and disadvantages that are well identified in the literature [8,18,30,31]. One of the main benefits of using these techniques is the objectivity of the assessment since participants generally cannot manipulate their own physiological reactions. Moreover, psychophysiological monitoring is a non-invasive method that is able to detect continually and in real time different cognitive and affective states when they occurs in response to a stimulus. However, the use of psychophysiological techniques poses also different challenges related to the modality of data acquisition and interpretation. Moreover, the quality of the recording may be influenced by both confounding environmental factors, such as noise or lightning, and by individual psychological internal state during the evaluation that might undermine the reliability and the correct interpretation of the gathered data. Physiological monitoring requires the use and management of specialized equipment. The accurate placement of electrodes on the body of the participant is essential for a correct acquisition of the parameters. Generally the monitoring of these measures occurs in a laboratory setting where the naturalness of the task or the interaction is lacking and the participants’ response may be biased.

Wherever designing a study aimed at the evaluation of any human-system interaction by means of psychophysiological measurements, specific guidelines should be taken into account in order to gather useful data for a precise analysis of the user’s experience [32]. Every experimental design should consider a set of phases. First of all, a researcher should select the appropriate psychophysiological parameters to be employed in order to identify definite cognitive and affective user’s states during the interaction with a robot. It is necessary to determine the sample size and characteristics. After determining the experimental setting (laboratory or real-world) and equipment, a pre-testing assessment should be performed with the purpose of verifying potential confounding factors. Participants should be informed about the aim and the procedure of the study through a consent form to be signed. Prior to the experimental evaluation, the researcher should gather some health and psychological information of the participants. Prior and following the assessment, the experimental design should also consider the administering of other tools, such as questionnaires/interviews or video-recording, to acquire additional information that might help the interpretation of the psychophysiological signals and of the users’ experience. The gradual adaptation to the stimulus, or in this case to the interaction with a robot, could modify participants’ answers. Finally, a debriefing phase should be conducted to reflect on the experience and make it meaningful by identifying any issue that may have had an effect on the participant’s response and behavior.

These guidelines are well discussed in Bethel et al. [32], and although the psychophysiological approach can be useful in human-robot interaction analysis in order to assess participants’ reactions and responses in terms of affective and cognitive states, in the experimental design, a researcher should always bear in mind advantages and disadvantages of the use of psychophysiological measures. The evaluation of the participants’ experience should focus on different methods assessment in order to gain a more complete understanding of the aspects that affect the quality of the interaction between a service robot and a human.

The next section illustrates the psychophysiological measures mostly applied in HRI.

3. Common Psychophysiological Measures in HRI

The nervous system can be divided into two parts: the central nervous system and the peripheral nervous system. The peripheral nervous system is subdivided into the sensory-somatic nervous system and the autonomic nervous system. As mentioned in the introduction, the central and autonomic nervous system responses can indicate possible modifications in our emotions, motivation, attention, and preferences. Without going into details, it can be stated that a system of multiple cortical (e.g., frontal, temporal, and parietal) and subcortical (e.g., basal ganglia, thalamus, amygdala, and hippocampus) regions contributes to the experience of emotion. Each cerebral area has a different role and a specific contribution to the elaboration of the stimulus [12]. According to right-hemisphere and valence hypothesis, this cerebral hemisphere is dominant for emotional processing, and in particular, for decoding negative emotions differently from left-hemisphere specialized for positive emotions [33,34]. The ANS is part of the peripheral nervous system (PNS) containing all the nerves that lie outside the central nervous system (CNS). The key role of the PNS is to connect the CNS to the organs, limbs and skin and to adapt organisms’ internal state to the internal and environmental demands maintaining homeostasis. The ANS is further divided into two branches: the sympathetic system regulates the “flight-or-fight” responses, while the parasympathetic system helps to maintain normal body functions and conserves physical resources. An activation of ANS is also recognized as consequence of emotional responses [35]; negative emotions seem to activate a stronger autonomic reaction than the positive ones. The functioning and activation of the ANS can be assessed through psychophysiological parameters. Recently, a psychophysiological approach has been adopted to assess the internal state of subjects interacting with the technology. In this section, we briefly introduce some widely used psychophysiological measures applied in human-robot interaction studies.

3.1. Cardiovascular Activity

The main measures of cardiovascular activity are heart rate (HR), blood pressure (BP), blood volume pulse (BVP) and heart rate variability (HRV). HR is the number of heartbeats per unit of time-typically expressed as beats per min (bpm). In terms of autonomic responding, increases in sympathetic activity are associated with heart rate rise, and relative increases in parasympathetic activity are associated with heart rate reduction [36,37]. The force exerted by blood on the walls of blood vessels is the blood pressure. Systolic blood pressure (SBP) is a measure of blood pressure when the ventricles are contracting and the heart beats. Diastolic blood pressure (DBP) is a measure of blood pressure while the heart is relaxed. Sympathetic stimulation causes an increase in blood pressure, while on the contrary, the parasympathetic system decreases cardiac activity reducing blood pressure [38]. HRV is the sequence of time intervals between heart beats. This interbeat interval time series is used to calculate the variability in the timing of the heart beat in response to changing physiological conditions and factors such as respiration rate or emotional states. Consequently, HRV is a clear effect of regulation by the sympathetic and parasympathetic nervous system. Sympathetic activity is associated with the low frequency components (LF) while parasympathetic activity is associated with the higher frequency components (HF) of modulation frequencies of the HR [39].

Other cardiovascular measures include respiratory sinus arrhythmia (RSA) defined as the natural variation in heart rate that is primarily driven by breathing patterns and its regulating influence of sympathetic activity on the heart. Although the role of RSA in emotional response and regulation is not yet clear, according to the literature, its response measures might be a reliable index of emotional valence in situations demanding emotion regulation [40,41]. Inhaling inhibits the activity of the sympathetic system, increases heart rate, and decreases HRV. Exhaling activates the sympathetic branch, decreases heart rate, and increases HRV.

Several studies have investigated the relationship of cardiovascular indicators with arousal and cognitive load [42,43]. Cardiovascular response is a good marker of emotional states of the subjects [35,44,45,46]. An increase in HR, SBP, DBP and in respiratory activity, but not in HRV, has been reported in anger, anxiety and happiness responses [40,47,48].

3.2. Electrodermal Activity

Electrodermal activity (or galvanic skin response or skin conductance) is based on the variation of the skin electrical resistance in response to various emotional stimuli. This variation is mainly due to the level of sweat of the skin as a result of the action of the eccrine glands [49,50,51]. Sweat glands are primarily driven by sympathetic innervations. There are two main components of electrodermal activity:

(1)

the tonic component (Skin Conductance Level) is the absolute value of the skin conductance at a given moment in absence of a specific stimulus onset. So-called nonspecific skin conductance responses also occur in absence of specific (individual) emotional stimuli, with their frequency proportional to arousal.

(2)

the phasic component (Skin Conductance Response) is the response to an emotional stimulus.

Electrodermal activity is measured placing two electrodes on the skin surface of two adjacent fingers and passing a tiny electric charge. When the subject responds to a stimulus, the skin immediately becomes a slightly better conductor of electricity. External emotional stimuli produce a drop in the skin electrical resistance and in particular at the level of the palm and soles of the feet.

Electrodermal activity is related to the level of arousal elicited by a wide range of psychological and emotional states with either positive or negative valence. Different studies investigating anxiety, anger, fear and also joy experience report increased electrodermal activity [52,53]. Electrodermal activity is also an indicator of the cognitive load, stress and arousal [42,51,54,55].

3.3. Electromyography Activity

Electromyographic signals (EMG) are produced by the activation of the muscle motor unit and provide an index of the muscle activity. A muscle is composed of a multitude of motor units that are activated by electrical signals elicited by the nervous system. When a motor unit is activated, an electrochemical impulse is fired and propagates through the nerve fibers. Along the nerve fibers, the impulse stimulates the skeletal muscle creating muscle contraction, and consequently, the movement of human limbs [56]. Surface EMG is a method of recording muscle activity by electrodes mounted directly on the skin. Surface EMG is evaluated by means of the analysis of the signal amplitude in microvolt. The muscles more frequently monitored are those reflecting the human emotional state and that can serve to assess the emotional regulation (frontalis, corrugator supercilii, zygmaticus major, levator labii superioris, orbicularis oculi) [11,57]. The facial electromyography gives information related to the affective state of subjects. A rise of zygomatic major muscle activity results in a happy facial stimuli, whereas people tend to contract corrugators supercilii in presence of angry facial stimuli [35,58,59]. The zygomatic muscle lifts up the lips to produce a smile, whereas the corrugator muscle knits the eyebrows forming a frown facial expression. Dimberg et al. [60] show that electromyographic responses to positive and negative stimuli can be generated without any conscious awareness of the emotional eliciting stimuli.

3.4. Brain Activity

Psychophysiological reactions based on the central nervous system activity can provide information about the brain responses to emotional stimuli. Electroencephalography (EEG) is a technique that records the brain electrical activity through the placement of electrodes over the surface of the scalp. The electrodes record the rapidly fluctuating of brain wave patterns, which are a sign of continuous electrical changes in the brain. EEGs show a different involvement of the two hemispheres of the brain (cerebral asymmetry) in responding to positive and negative emotional stimuli. A higher activation of the left hemisphere indicates a positive reaction, whereas the right hemisphere is sensitive to negative affect [35,61]. Specifically, prefrontal cortical regions of the brain are involved in emotional responding. Several studies show that frontal EEG asymmetry is considered a mediator of emotional processing [62]. Some of the earliest studies about the frontal EEG asymmetry and emotions were done by Davidson et al. [61], and concerned differences in hemispheric activity during the viewing of emotional films. Results showed that more active right hemisphere activity was related to negative content in films while more left hemisphere activity was related to happy films. Recently, evidence suggests a relationship between brain activity and anxiety [63]. Distinct patterns of brain activity are associated with different types of anxiety. For example, anxious apprehension involves an increased left-hemisphere activity and anxious arousal is related to a greater right-hemisphere activity [64]. Electroencephalography activity can also be used as a measure of mental workload. Chaouachi et al. [65] present a workload index based on features extracted from EEG signals. Wilson [66] classifies operators’ workload level by taking EEG as an input and reports results of 90% accuracy.

Functional magnetic resonance imaging (fMRI) is a further method used to evaluate human response during the interaction with a robot. This technique displays the hemodynamic response (changes in blood flow and oxygenation) related to neural activity in the brain or spinal cord. When nerve cells are active, they consume oxygen carried by hemoglobin. The effect of this consumption is an increase in blood flow in regions where there is a greater brain activity. When measuring changes in blood flow associated with increasing neuronal activity, it is possible to map the activation of brain areas during an activity or in response to a stimulus. Specific brain areas (amygdala, medial prefontral cortex, anterior cingulated cortex, insula) are supposed to be involved for emotional processes [67]. Saygin et al. underline the value of an interdisciplinary collaboration between robotics and cognitive neuroscience to improve the understanding of how robots’ actions, behavior and embodiment are perceived and processed in the human brain [68]. Some relevant research has focused on Theory of Mind (ToM) [69]. When we interact with each other, it is possible to infer the interlocutor’s internal state, observing behavior, ways of speaking and facial expressions. The other’s internal state refers to their intentions, goals and beliefs, on the basis of which, it is possible to interpret the behavior of others. Brain imaging methods, such as fMRI have been widely applied also to identify brain areas for ToM in order to explore how human users respond, perceive and represent robots’ actions, behavior and embodiment during an interaction [70].

4. Psychophysiology Applied in HRI Studies: An Overview

As mentioned in the introduction, Bethel et al. identify three main categories of research studies implementing psychophysiological measurements in HRI [8]. The first is focused on detecting emotions expressed by human users, based on the development of control architecture, behavior system or online emotion detection algorithms able to recognize, classify and interpret participants’ affective state [19,70,71,72,73,74,75,76,77,78,79,80]. The second category is related to the analysis of subjects’ psychophysiological responses for development and implementation of real-time robot control architectures, and adaptation of robot behaviors in order to, for instance, maximize users’ motivation, engagement and performance during rehabilitation therapy [64,81,82,83,84,85,86]. The third category is the objective of this review, and it comprises research studies aimed to evaluate human experience and emotional reaction to the technology. The intent in the next subsection is to review the most relevant and recent works specifically oriented to using psychophysiology measures in understanding humans’ emotional response and reactions towards a robot. A separate subsection describes a set of recent studies implementing functional magnetic resonance imaging (fMRI) as a method to evaluate how the human brain reacts to the robot’s features and behavior.

4.1. Psychophysiological Studies Evaluating Humans’ Response to Robots

In the field of research that implement psychophysiological measures to assess how humans react to a robot, Kulić and Croft evaluated the feasibility of physiological measures as indicators of human subjective response towards motions of a robotic manipulator arm [72]. Electrodermal (GSR), electromyography (corrugators muscle) and cardiovascular activity (HR) of 36 participants were measured during an experiment in which the human response to two different robot motions was assessed. The robot could be attracted to the target location and be rejected avoiding obstacles (potential field planned robot motions) and it could have the capability to avoid obstacles (safe planned robot motions). Two different tasks were performed by the manipulator robotic arm: (1) pick and place (2) reach and retract. Three different speeds for the manipulator arm were provided for each task.

Psychophysiological reactions were analyzed using a fuzzy inference engine as an estimation method for the relationship between physiological responses and emotional categories. Fuzzification is the process of converting the input variables’ values into a set of membership values in the interval {0,1} in the corresponding fuzzy sets [87]. Physiological signals are not measured directly via the fuzzy inference engine but they are preprocessed. The relevant features extracted are used for hypothesis and for establishing a set of rules base to estimate the human emotional state. In this research, the fuzzification was derived using data from psychophysiological research [17].

Psychophysiological data were compared with subjective responses based on a Likert scale developed to assess anxiety, calm and surprise levels of the participants. Results showed a correlation between anxiety and speed, and surprise and speed of the robotic manipulator arm. In addition, subjects tended to show a stronger response in electrodermal, electromyography and cardiovascular activity (in terms of higher levels of anxiety and surprise and lower levels of calm) for potential field planned motions than safe planned motions. An estimate of arousal and valence of human emotional state was obtained from the fuzzy inference engine, but it has not proven a capability to successfully estimate valence.

In a second study, researchers developed a new human affective state estimator based on the hidden markov model (HMM). They performed the same experimental method and procedure in [72] and compared the HMM with the fuzzy inference engine. Their results showed that HMM is a better affective state estimator, able to identify the relationship between physiological signals and valence and arousal [19].

Dehais et al. [88] evaluate human response to different types of robot motions. They compare different ways to move a robotic arm close to the “human space”. Twelve participants were recruited in order to evaluate the effects to three different robot motion types (Motion 1. with planner, with grasp detection, medium velocity; Motion 2. no planner, no grasp detection, high velocity; Motion 3. with planner, no grasp detection, low velocity) on physiological parameters, such as electrodermal activity, deltoid and ocular activities. The task consisted of taking a bottle from the robot’s grip and participants were then asked to report their opinion on a nine-point visual analog scale, reporting the legibility of the different types of robot motions, safety and physical comfort perceived during the experiment. Their results, similar to those of Kulić and Croft in [72], show a higher electrodermal activity for high velocity robot motion compared to the other two motions characterized by medium or low velocity. The participants interpreted the electrodermal activity with a higher level of arousal and anxiety in Motion 2. The lowest deltoid and ocular activity (in terms of fixation time) was found for Motion 1. with planner, grasp detection and medium velocity.

In the results of the above studies, a common aspect lies in the belief that by combining different measurements (physiological parameters and self-report measures), it is possible to obtain information on the nature of the interaction between a human and a robot, and on the specific human reaction. The results show that the robot’s behavior has an important effect on the user’s reactions and it can influence acceptance toward a robot and the user’s perception of interaction safety.

Human physiological responses are also involved in the assessment of specific user’s responses to different tasks performed through a robot. Goljar et al. [89] recruited three different groups of subjects: a subacute stroke (N = 23 subjects), a chronic stroke (N = 10 subjects) and a control group (N = 23). Electrodermal, cardiovascular activity and respiratory rate were monitored during a reaching and grasping exercise task performed through a manipulator robotic arm and a haptic robotic interface. The aim of the study was focused on understanding if stroke patients’ physiological responses were different from the responses of healthy subjects in relation to a physically undemanding task, and which physiological responses were most reliable in evaluating psychological states in motor rehabilitation training. Two different types of each were presented to the subjects (virtual task vs. harder virtual task), having different complexity, hence being different in their mental demand. The virtual task required to catch a ball before it reached the lower end of a virtual table. The manipulator guided the subjects’ arms in order to move left or right and reach the ball. In the harder virtual task, subjects performed the same exercises but the robot had inverted left-right controls. Physiological data was also compared with 9-point arousal and valence scales from the Self-Assessment Manikin, in order to have an index for the participants of their level of emotional valence and arousal during the experiment. Results showed that the electrodermal signal appears to be the most valuable for obtaining useful information about participants’ arousal state. Specifically, electrodermal activity of the control group was higher than the subacute and chronic stroke groups during both types of task. A significant correlation between electrodermal activity and Self-Assessment Manikin was also found in all three groups. Beside some limitations of the study discussed in the paper, results confirmed the utility of selecting appropriate psychophysiological measures to get supplementary information about users’ psychological state and the efficacy of robots in clinical practice.

Also in this study, feedback obtained from the user’s physiological parameters is useful for evaluating the emotional status of a human who performs a rehabilitation task through a robot. Individual reactions in rehabilitation practices may have an effect on the patient’s performance that could be affected by the robot’s behavior. The efficacy of human-robot interaction may depend also on individual motivation and engagement so it is important to ensure that the robot’s behavior is able to adapt to a user’s state, needs and capabilities.

Recently, Swangnetr and Kaber [90] recruited 24 participants at a senior center in order to evaluate the relationship between physiological response and emotional states, and to develop a machine-learning algorithm for their accurate classification. Heart Rate (HR) and electrodermal activity were collected during a simulated medicine delivery experiment in which participants interacted with a PeopleBot robot. The experimental design counted three independent variables: robot facial feature (abstract vs. android); speech capability (synthesized vs. digitized voice; and different modes of user interaction (visual messages vs. physical confirmation of receipt of medication with a touch screen). All participants were exposed to each variable. At the end of each experimental session (14 test trials), subjects were asked to complete the Self-Assessment Manikin in order to indicate their emotional response to the different robot patterns. In general, respect to the variables on the robot feature type, the results showed a significant difference in terms of an increased HR and electrodermal activity to the condition in which the robot required physical confirmation of receipt of medication with a touch screen. Referring to the development of a machine-learning algorithm, for HR and electrodermal activity features extraction, a wavelet analysis was used with the purpose of classifying human emotional state in real time.

Results showed that HR was a stronger index of valence compared to electrodermal activity, while arousal states had a strong relationship with electrodermal activity. The wavelet analysis was confirmed as a valuable method for identifying associations between physiological response and arousal and valence. A correct classification of human emotional state in response to a robot in a healthcare setting could contribute to ensure efficacy and quality of the human-robot interaction.

Similar results in Zhang and colleagues [91] showed that interactivity features of a service robot stimulated higher cardiovascular and electrodermal response in 24 older participants during a set of experimental trials in which a robot executed a medicine delivery task. Participants’ cardiovascular (HR) and electrodermal activity was collected during the experimental trials during the interaction with the robot. A fuzzy logic model was adopted in order to gather physiological data in terms of arousal and valence. The objective of the study was to understand how different service robot interfaces (face, voice messaging and user interactivity) affect elderly people’s perceptions and emotional response. The robot had an abstract or human-like face and a synthesized or digitized human voice. User interactivity consisted in two levels of interaction based on the reading of a visual message and the request to confirm medicine delivery. Participants indicated a more positive emotional experience interacting with a service robot with anthropomorphic features based on interactivity aspects.

Broadbent and colleagues [92] explored if mental schemas about robots’ humanness were associated with their psychophysiological reactions to a robot. They recruited 57 adult participants and asked them to draw their idea of a healthcare robot before seeing a real robot (PeopleBot), explaining how to attach the blood pressure cuff for measuring it. Blood pressure was monitored at baseline and then during the interaction with a robot. Participants completed also the positive and negative affect scale (PANAS) [93]. An increase in blood pressure, as an index of the ANS, was registered in participants who had drawn the robot more human-like compared to those that imagined a more machine-like robot when they interacted with the real robot. In addition, participants who drew a robot more human-like reported also more negative emotion after the interaction with the real robot compared to those who drew a machine-like robot.

A possible interpretation of these results may refer to the high responsiveness to stress and arousal by the ANS. Assessment and interpretation of human physiological responses towards features and behavior of a robot provide valuable elements for understanding the degree of applicability of a robot based on the context of use, on the type of task, on interaction modality and on users’ needs and preferences.

4.2. fMRI Studies Evaluating Humans’ Responses to Robots

A further technique to assess the human reaction during the interaction with a robot is found in the study of brain response. The assessment of brain activity is an interdisciplinary area involving different disciplines, as cognitive neuroscience, psychology, robotics engineering and social science, and gaining a great interest in the field of HRI studies. Saygin et al. [68] recruited 20 participants in a fMRI experiment in which they viewed video clips of three agents carrying out actions, such as drinking water from a cup or waving their hand. The three agents differed in form: human; robot with human form; robot with nonhuman form. The aim of the study was to explore whether the human brain would show a different specialization area for human form and motion. Researchers expected similar brain responses for human and robot with human form, and differentials for robot with nonhuman form, while for motion they supposed similar responses for robot with human form and robot with nonhuman form, and differentials for human. Results of this study demonstrated that robot with human form was processed in a different brain area compared to both human and robot with nonhuman form. Specifically they found an activation in the parietal cortex for the robot with nonhuman form. The researchers supposed a prediction error based on the incongruity between the human-like form and the mechanical movements that the brain is not able to integrate. Chaminade et al. [94] used fMRI as a method to confirm the hypothesis of a preference in autistic patients for a small humanoid robot during a computerized computer game. They recruited two experimental groups of subjects: 18 healthy students and 12 participants with Autism diagnoses. All subjects participated in the rock-paper-scissors game in a computerized version. Participants were told that they would play against three different opponents: the experimenter, an artificial agent and a generator embedded in a computer. The subjects knew that the first two opponents (active opponent) playing strategies used to win. The fMRI data were collected during the three game sessions. Results confirmed that autistic patients considered the artificial agent as a human opponent since the same level of brain activity in the temporal area of autistic patients was registered by means of fMRI when they interacted with the robot and with the human. In another study [95], fMRI was used to test the hypothesis of a reduced brain activity in areas involved in motor and emotional processing during the interaction with a robotic agent compared to human agent. The experimental paradigm consisted of 13 healthy young participants and 16 experimental conditions in which human and robot agents impersonated four facial expressions (anger, joy, disgust and speech). Participants observed the human and robot agents performing the gestures by way of short videos while their brain activity was scanned. An increased activity of occipital and posterior temporal regions was found in response to the gesture of robotic agent compared to the human agent, while a response in insula and orbitofrontal cortex respectively for the perception of disgust and anger emerged only for human gesture.

In order to determine the brain human response to the robot’s embodiment during an interaction, Miura et al. [96] analyzed fMRI data of 39 students during an interaction with bipedal or wheel drive humanoid robots. A human like embodiment of the robot would involve a greater emotional response in terms of brain activity compared to a robot with a more machine-like embodiment. The experimental procedure consisted in six conditions in which a human, a bipedal robot (more human-like) and a wheel drive humanoid robot (more machine-like) performed emotionally positive and neutral daily life actions (e.g., walking happily). The fMRI scanning was along the six conditions. Results revealed that the left orbi-frontal cortex was more active during the interaction with the bipedal humanoid robot performing emotionally positive actions compared to the wheel drive humanoid robot. It is worth noting how the same brain response was found during the interaction with the human.

The studies described above confirm that the psychophysiological approach is well applicable to assess emotional response and quality of experience during human robot interaction. The benefits from the use of physiological parameters in HRI research are well documented by [8].

5. Using Psycho-Physiological Measures to Evaluate Older Adults’ Response to a Telepresence Robot: A Feasibility Study

To ensure an effective human-robot interaction, it is necessary to understand also how end users react to the presence of a robot and to its features (e.g., abilities, appearance, behavior). Given the mentioned benefits of psychophysiological approach and drawing inspiration from the reviewed literature, we designed a study aimed to implement psychophysiological measurements for assessing the human response towards a telepresence robot (results in this section partially appeared in [97]). Telepresence robots have been more and more often proposed to be used to enhance older adults’ social and daily functioning and to supplement human care-giving. Robots, as in Giraff [98], Texai [99], and Kompai [100], have been put forward as a proposed form of assistive and communicative device, able to deliver health and social services. To date, literature has not focused on empirical studies that employ a psychophysiological approach specifically for human-telepresence robot interaction. Our intent was to explore the impact of a telepresence robot in terms of emotional human response in older adults. We have relied on the use of psychophysiological measures presuming also that this approach may offer valuable information also about potential problems in the human-robot interaction, that need to be taken into account in order to make the proposed technology more usable, satisfactory and adaptable to the user’s needs. The objective of the study was to examine the effectiveness of a remotely controlled telepresence robot, called Giraff [101], as a tool for health and social services. Subjective and psychophysiological measures were collected in order to evaluate older users’ responses towards robots’ presence and to obtain initial feedback about the suitability of a telepresence robot to support older adults.

5.1. Apparatus

Giraff is a social robotic telepresence system manufactured by the Giraff Technologies AB (Figure 1). It is a human-height physical robot integrated with a videoconferencing system mounted on a mobile robotic base. A user (called pilot) can remotely connect through an Internet connection to the robot via a computer interface and can move around in the environment where the robot is located and interact with other humans (called local users). Giraff is part of the ExCITE European Project [102] dedicated to study the use of robotic telepresence as a means to promote social interaction and inclusion of older adults [98].

Figure 1. The Giraff robot.

Figure 1. The Giraff robot.

The experimental setting was composed of two rooms. In the first room participants interacted directly with the human experimenter, whereas the second room was used by the experimenter to interact through the telepresence robot. The experimenter maneuvered the robot through the pilot from the second room and met the subject remotely.

5.2. Physiological Parameters

The physiological monitoring during the study was performed through a Holter monitor. Cardiovascular activity in terms of heart rate (HR) and heart rate variability (HRV) was measured. As described in the previous sections, HR and HRV result from the activation of the ANS, which consists of two components: the sympathetic and the parasympathetic systems. Based on this assumption, an increase in sympathetic activity was expected, associated with heart rate rise, as well as relative increases in parasympathetic activity, associated with slower heart rate [4]. HRV is a consequence of regulation by the sympathetic and parasympathetic nervous system. The most frequently used HRV components are spectral parameters such as low-frequency (LF) band and high-frequency (HF) band percentual powers. Respectively, the LF band is considered as markers of the sympathetic nervous system, while the HF band reflects parasympathetic (vagal) activity A same pattern of activation of ANS was expected both in the interaction between human experimenter and participants and in the interaction with the robot. The protocol should have included an analysis of the combination of several physiological parameters, but some issues, related to the timing of the collaboration between the parties involved in the research, have limited the study only to the analysis of HR and HRV. We do not exclude the possibility of expanding the results by analyzing other physiological measures in later stages of the study.

5.3. Subjective Measures

The administered self-report measures were:

(1)

The State-Trait Anxiety Inventory [103] (STAI). It is a self-report questionnaire consisting of two scales (Y1 for state anxiety and Y2 for trait anxiety) containing 20 items each. All items are rated on a 4-point scale (i.e., from 1= almost never, to 4= almost always).

(2)

The Positive and Negative Affect Schedule [93] (PANAS). It consists of two scales containing 10 items each, onemeasuring positive affect and the other measuring negativeaffect. Each item is rated on a 5-point scale (i.e., from 1 =very slightly or not at all, to 5 = extremely).

(3)

A concluding interview on the robot experience wasadministered in order to get information about: social presence (“I had the feeling that the investigator was in the same place where I was”), utility and advantages of the robot (“I think that Giraff could be useful in the home”), engagement (“I felt physically involved in the experience with Giraff”), physical aspect of the robot (“Giraff is pleasing to see”), privacy issues (“It may be annoying to receive virtual visits through the Giraff at home”).

5.4. Participants

Seventeen participants were recruited at the rehabilitation department of the Fondazione “Don Gnocchi” of Rome. Given the intent to validate the use of the telepresence robot as an aid for health and social services, it was decided to focus on a specific type of older users, recruiting a group of elderly with a diagnosis of mild cognitive impairment (MCI) and a group of healthy older adults. The elderly group with a diagnosis of MCI was or had been engaged in cognitive rehabilitation interventions at the Fondazione. The characteristics of the sample are as follows:

(1)

Normal control group (NC), which consisted of 9 participants (age 70 ± 4.3 years, age range 65–75, 6 males, 3 females) with no cognitive impairment. Subjects were tested with the Mini-Mental State Examination (MMSE) reporting a score of 30. Mean score on Trait Anxiety scale (Y2) was 32.3 (with Spielberger’s recommended cut-off of 39/40 [82]).

(2)

MCI group, which consisted of 8 participants (age 73.5 ± 5.2 years, age range 65–79, 5 males, 3 females). MCIs were assessed by a battery of standardized neuropsychological tests and their score on MMSE was between 25 and 30 (mean 27 ± 1.18). Mean score on Trait Anxiety scale (Y2) was 34.5 (with Spielberger’s recommended cut-off of 39/40 [82]).

5.5. Procedure

The study has received the consent by the Ethical Committee. On the basis of agreements with representative health professionals of the Fondazione, experimental procedure took place over the course of four-day laboratory sessions performed at the Fondazione “Don Gnocchi”. Upon arrival to the first-day laboratory session, participants were asked to read the consent form and to sign it. They were also asked to fill the STAI-Y2 and a demographics form. Participants were asked to avoid nicotine, caffeine, alcohol, other substances, and strenuous exercise at least three hours before their scheduled laboratory session. The experimental procedure is show in Figure 2 and it was articulated into four main sessions in which the participants interacted:

Figure 2. Experimental procedure.

Figure 2. Experimental procedure.

(1)

directly with the human experimenter during the first two sessions;

(2)

through the telepresence robot during the subsequent two sessions.

Two sessions for each type of interaction were scheduled in order to allow participants to know the experimenter and the robot (the sequence of the sessions was established according to the agreement with the limitations imposed by the Foundation for the patient’s safeguard). Each interaction session had three main phases. During the Baseline (T0), HR and HRV assessment at rest was performed. Baseline was followed by an Interaction Phase (T1) in which the experimenter administered cognitive stimulation tasks to the participant. Finally a Recovery Phase (T2) was monitored for regaining HR and HRV steady state. Physiological assessment was continuous during each phase of each session. During the laboratory session the participant was seated in front of the experimenter and the Holter electrode was applied to the subject’s chest.

As indicated by the data literature of human studies in HRI measures, a timing between 2 to 5 min is reported to compute cardiovascular parameters [64,84,91]. In line with this indication, and according to the healthcare professionals of the Fondazione, we chose a duration of 3 min for the baseline phase, during which participants were asked to rest. Following the Baseline Phase (T0), participants were engaged in 10 min of Interaction Phase with cognitive stimulation (T1). Following the interaction phase, participants were asked to remain seated for 3 min (recovery phase, T2). STAI-Y1 and PANAS scale were administered at the end of the recovery phase of the first and third session, in order to gain feedback in terms of anxiety and positive/negative affects for both types of interaction (human and robot). At the end of the fourth day laboratory session, participants met the experimenter again for the final interview focused on their experience with the telepresence robot.

Cognitive Stimulation Tasks

Cognitive stimulation tasks were usual activities for the participants of our experiment. We have decided to employ some tasks similar to the cognitive exercises of the cognitive rehabilitation interventions of the Fondazione to which patients had been or were engaged previously. This choice was purposeful also for examining the human/robot interaction in a rehabilitation context.

Particularly, in accordance to the cognitive therapist of the Fondazione, we have chosen to administer the following cognitive exercises:

(1)

Word list memory: a list of 10 words is read to the subject at a constant rate of 1 word every 2 s. The word list is presented 3 times to the subject. At the end of each of the 3 presentations, the subject is asked to recall the list of words.

(2)

Verbal fluency test: subject is asked to name in one minute a list of words within a specific semantic (e.g., vegetables or professions) or phonetic category (words that begin with letter F, A, and S).

(3)

Story recall: the participants are asked to memorize the story once and then to complete an immediate recall test, at the end of the Interaction Phase.

(4)

Digit span Forward / Backward: in the forward digit-span task a series of numbers (e.g., 7–3–9) are presented to the subjects and they have to immediately repeat them back in the given order. If they do this successfully, they are given a longer list. In the backward digit-span task the participant needs to reverse the order of the numbers.

(5)

Numerical and abstract reasoning: in the first case participants are asked to resolve simple operations in addition, subtraction, multiplication or division; for the abstract reasoning subjects have to interpret some popular proverbs.

5.6. Results

For statistical significance, we used a multivariate analysis of variance (MANOVA) with one between subjects factor (Group: NC or MCI) and three within subjects factor (Presence_Type: human or robot; Session: first or second observation; Phase: baseline, interaction and recovery). A comparison of the HR variables in all phases of the experiment showed no significant difference in the baseline values.

5.6.1. Heart Rate

As we supposed before, an increase in sympathetic activity associated with heart rate rise was expected in response to the cognitive stimulation activity. Results showed a main effect for the variable Phase, F(_2.28) = 35.7, p < 0.05, η² = 0.71, indicating that heart rate was significantly higher during the interaction (T1) with respect to the baseline (T0) for both groups. A significant interaction Phase x Group (NC or MCI), F(_2.28) = 5.26, p < 0.05, η² = 0.27, indicated a higher increase in HR at T1 in NCs compared to MCIs (means are reported in Figure 3). A tendency to significant interaction was found for Presence_Type and Group F(_1.15) = 3.79, p < 0.07, η² = 0.20, indicating a higher HR in NC group compared to MCI during participants-experimenter direct interaction. There were no significant Presence_Type and Session effects in HR response during the various phases suggesting that there is a comparable cardiovascular reaction in terms of HR both in the participants-experimenter direct interaction and in the robot mediated interactions. A possible explanation of this result might lead to consider the change in Heart Rate in NCs as an index of a higher arousal in response to the cognitive stimulation activity independently on the type of presence (human or robot).

Figure 3. Mean for HR for phase (T1, T2, T3) for both groups during the interaction with human and telepresence robot.

Figure 3. Mean for HR for phase (T1, T2, T3) for both groups during the interaction with human and telepresence robot.

5.6.2. Heart Rate Variability

The mean of low frequency (LF) and high frequency (HF) were considered for each phase of each experimental session. We expected a predominance of the LF band as a marker of sympathetic nervous system activation and a decrease in HF band as result of parasympathetic in response to cognitive stimulation activity during the interaction both in the participants-experimenter direct interaction and in the robot-mediated interactions. A significant interaction Phase x Presence_Type (human or robot), F(_2.28) = 4,18, p < 0.05, η² = 0.21 was found. This result indicated that both groups showed a higher increase in LF percentual power at T1 during the interaction with the robot. There was a significant interaction Phase × Presence_Type × Group, F(_2.28) = 4,6, p < 0.05, η² = 0.23. Both groups showed a decrease for HF percentage power at T1. A significant effect for Presence_Type was found in both groups but the MCIs showed a higher decrease of HF percentage power at T1 during the robot-mediated interaction with respect to the NC group. A possible explanation might lead to consider these results as an indicator of a higher stress level in MCIs in response to a cognitive stimulation activity during the robot-mediated interaction.

5.6.3. Subjective Measures

State Trait Anxiety Inventory (STAI): no significant differences were observed between groups in terms of state anxiety (p = 0.33). In addition, no major effects were observed with respect to the Presence_Type (human or robot). No significant correlation was found between state anxiety and physiological parameters.

Positive and Negative Affect Scale (PANAS): mean scores were calculated for participants-experimenter direct interaction and robot-mediated interaction. Mean higher scores on Positive Affect subscale were found in both groups for both Presence_Type. NC group’s mean score for participants-experimenter direct interaction = 37.3, while mean score for robot-mediated interaction = 37.5; similarly MCI group’s mean score for participants-experimenter direct interaction = 32, while mean score for robot-mediated interaction = 31.4. Comparison between the Positive and Negative Affect Scales score and physiological parameters collected indicates meaningful correlation for Negative Affect. We found that Negative Affect indicated higher significant correlations with the HR Phase at T1 (r = 0.545, p < 0.05) e T2 (r = 0.503, p < 0.05) during the first session of participants-experimenter direct interaction and T3 (r=.573, p < 0.05) of the first session and T0 (r= .560, p < 0.05) T1 (r = 0.571, p < 0.05) T2 (r = 0.558, p < 0.05) of the second session of robot-mediated interaction. These results indicated an experience of negative emotions, especially for robot-mediated interaction.

Final interview: in general our participants affirmed they felt physically involved in the experience with the robot (%NC = 100 and %MCI = 87). The experience with the robot was judged as rather pleasant (%NC = 89 and %MCI = 75) and involving (%NC = 78 and %MCI = 62) and totally spontaneous. NCs and MCIs subjects have a sufficient perception of the experimenter, as she was in the same place where they were (%NC = 56 and %MCI = 62). In addition, 78% of NC and 62% MCI participants asserted they would feel safer with the robot at home. However, only 50% of MCI group in case of need would be willing to use the robot compared with 79% of NC group. In case of need, a telepresence robot could be useful at home for 89% of NCs compared with only 25% of MCIs. Most of the participants stated that the presence of the robot would not disturb their life (%NC = 78 and %MCI = 75) but 75% of MCI subjects, compared with 44% of the NC group, considered the robot as annoying as a possible visit from a person. Lastly, the robot was judged pleasant to see (%NC = 100 and %MCI = 75) but all the participants of NC group, compared with 25% of MCI, judged the robot too big.

5.6.4. Summary and Discussion of Results of the Study

The feasibility study demonstrated that the robot presence did not cause adverse effects in terms of cardiovascular response and these results support the possibility for using a robotic mediated interaction by a telepresence system in clinical practice and also for a specific target patient as older adults with MCI diagnosis. During the participants-experimenter direct and robot-mediated interactions, HR data showed a pattern similar for the NCs and the MCIs group as a response to cognitive stimulation tasks. HR was higher during the interaction (T1) with respect to the baseline (T0) for both groups. However, the significant main effect of the Group on heart rate response showed higher mean heart rate for NC group than MCI group. The NC group’s heart rate increases from baseline during the interaction period and this could be considered as an indicator of higher arousal. The increase of arousal was slightly greater in the NC group during participants-experimenter direct interaction.

In terms of ANS response, the MCI group showed a higher mean sympathetic activity (predominance of LF band) and a reduced parasympathetic activity (decreasing in HF band) compared to the NC group. The higher sympathetic activity dominance found in the MCI group could be an index of the MCIs higher mental stress during interactions mediated by the telepresence robot. Despite the MCI group exhibited significant more mental stress, they reported subjective anxiety scores similar to the NC group for participants-experimenter direct and robot-mediated interactions. A general positive attitude of the telepresence robot was reported by participants, and this aspect might influence older adults’ perceptions of the usefulness and ease of use of robots [104].

Some limitations of our study are consistent with other studies mentioned in this review [68,88,90,91,94] and are worth being highlighted. First, the small sample size of this study could preclude the generalizability of our results. The recruitment of MCI patients has encountered several difficulties. For instance, patients were asked to come to the Fondazione, and this request represented more effort for participants’ relatives. Second, a weak association between physiological responses and self-reported measures in the case of the MCI participants would require further investigation. Although there is a clear difference between the groups in HR response during the interaction phase, we may affirm that the presence of the robot has not caused anxiety in the participants. However, based on the correlation between the Negative Affect scale of PANAS, the robot’s presence may have had a slight negative connotation in terms of affects. Third, the MCI’s medications might have altered some physiological responses (especially for HR) despite the two experimental groups showing the same cardiovascular clinical pattern. Fourth, the indications for experimental procedure suggested by the representative healthcare staff of the Fondazione did not allow us to control possible habituation effects. Finally, the laboratory setting/experiment may have increased the task engagement in the NC group because they could be more motivated to achieve a satisfactory performance.

Notwithstanding the limitations of the study, our results confirm the efficacy of the psychophysiological approach in studying human robot interaction as a valuable method also for comparing the reactions of different user groups. Telepresence robots, such as Giraff, have the potential to guarantee continuity of care, allowing users to be video-monitored by caregivers or healthcare providers remotely. Before the implementation of a robot in a user’s life environment, it is recommended to evaluate users’ characteristics and their emotional reaction to the robot. A positive reaction will affect the quality of experience and the effectiveness of the interaction. Psychophysiological assessment is a viable method to understand how a patient with specific diseases can react to the robot’s presence and to its features, offering valuable feedback to understand the feasibility of an assistance program based on a robotic device.

6. Some Lessons Learned from the Literature

Starting with the assumption that assistive social robotics has been increasingly proposed as a tool for health care and social services, especially for older adults and their caregivers [105], a very important issue must be to expand our knowledge about how humans may react and interpret the robot’s behavior, robot-performed human-like gestures, the robot’s ability to perform specific tasks, and how the robot’s physical embodiment may affect an interaction. The psychophysiological approach is confirmed as a valuable method to investigate the human’s response towards an interacting robot. This section is intended to provide a summary of the guidelines underscored above, re-examined in the light of the reviewed literature and of our study on the older adults’ responses to a telepresence robot.

Psychophysiological measures and their detection: Cardiovascular, electrodermal, electromyography and brain activity are widely employed in the studies of this review and their results are in line with previous studies on physiology of emotion [11,35], confirming that electrodermal (GSR) and cardiovascular (HR) measures are the most commonly assessed indices of ANS activation and of stimulus valence and arousal activation [14,90,106,107]. Consequently, it is recommended to refer mainly to cardiovascular and electrodermal variables in order to obtain an autonomic assessment. Psychophysiological parameters should be suitable for the experimental environment’s features (laboratory, virtual or field setting), for the participant’s profile (children-young and old), and as already mentioned, for the specific emotional reaction to measures. Similarly, the selection of the most appropriate equipment for detecting variables of research interest have to be considered one of the strategies for the overall control of the research plan. In a psychophysiological assessment, it is important that the experimenter learn to manage the equipment for the detection of the physiological parameters. In addition, it is suggested that the experimenter be supported by neuropsychopathology staff for the collection and interpretation of data. Physiological equipment or devices are not always easy to wear, and some of them may be invasive, affecting the participant’s behavior and reaction. It is recommended, whenever it is possible, to choose non-invasive devices that do not exceedingly obstruct the person’s physical body movements. Regarding this aspect, a phase of equipment functioning pre-assessment, events timing, and acquisition of the data to be recorded, should be taken into account.

Robots and participants as part of the experiment: The participants involved in the studies of this review interacted and reacted to different robot’s features regarding the behavior, the level of anthropomorphism and the type of task performed. These variables can affect the robot’s safety and effectiveness of the interaction perceived in relation to the user needs and context of use. In [22,68,94] participants reported positive emotions in response to human-like robots, confirming the results of Minato et al. [108] based on the Uncanny Valley hypothesis [109]. Human-like robot’s features in terms of gestures and movements generate a positive response among the participants and consequently affect their attitude and acceptance towards the robot. The most used robots are generally robotic manipulator arms [19,72,89] social assistive robots, such as PeopleBot [91,92], mechanical humanoid robots [68,94,96], and robotic haptic interfaces [89], with which participants do not have sufficient familiarity. As reported in the literature, user’s previous experience and familiarity contribute to modulate acceptance, especially in older adults [110,104]. It is likely that a lack of familiarity may have an adverse effect on the user’s reaction. In the recruitment of the sample it is useful to consider all the external variables (such as technology experience) as well as the users’ characteristics that may generally affect their attitude and performance [104]. Among users’ characteristics, healthy status and psychological information should be evaluated at the beginning of the experiment. Clearly, the cardiovascular pattern will affect the participant’s reaction and it will be taken into account for possible confounds. One point is worth being underscored: the subjects recruited for sampling in many of the HRI studies described in this review are university students or young adults (e.g., [84,89,111]). Furthermore, stroke victim subjects tend to be younger adults with an average age of 50 years. The average age of an elderly subject enrolled in several studies was maximum 69 years. Just two of the mentioned studies refer to a sample composed exclusively of older adults [90,91] with an average of 80 years. The sample size varies from a minimum of 12 [88] to a maximum of 57 participants [92]. In the research of the category of study considered in this review, the issue of sample size identified by Bethel persists [8]. However, in [8] Kulic et al. [72], they had the largest sample (with 36 individuals), and in this review Broadbent et al. research had the largest sample recruited (with 57 participants). Determining the sample size before collecting data is important in order to achieve sufficient significance from the results [32]. This is not always easy, especially when the sample, such as ours, includes people with a specific disease whose recruitment and participation in the experiment can be demanding. As suggested by Bethel [32], a power analysis would always be advisable to determine the sample size required to detect an effect of a given size.

Combining objective and subjective measures: The experimental method of the studies is based on the main recommendations in HRI research when using a psychophysiological approach [32,112,113]: (1) combining multiple psychophysiology measures in order to find correlations in the results; (2) employing psychophysiology parameters as a valuable complement of self-report measures. Among the most commonly used self-report measures administered, the Self-Assessment Manikin [16] is a very common assessment technique to measure the pleasure, arousal, and dominance associated with participants’ emotional reaction to robot stimuli as reported in [89,90]. Other research includes self-report assessment developed ad hoc [19,72,88] or a scale for evaluating affective response as positive and negative affect schedule (PANAS) [93]. A multimodal approach can be convenient to facilitate the understanding of the human response based on the evaluation of the correlation between behavioral feedback/physiological indicators in response to the stimulus (i.e., the robot) and what the participant reports about the experience with the robot itself. In addition, objectivity of psychophysiological methods is a resource to avoid social desirability effects because we know that self-report assessment is subject to sources of error and bias; the participant usually wishes to provide a “socially acceptable” answer rather than to report his/her true feelings [114]. A study protocol should also consider the habituation effect that is very common in psychophysiological studies. Habituation has several implications for the reliability and stability assessment of results produced by studies based on the psychophysiological approach. Administering self-report measures or asking participants to evaluate their experience in the presentation of each task or stimulus of the experiment [8] is a consolidation strategy to avoid or reduce the negative effects of habituation, as reported in [72,88,89,90,91].

Briefing and Debriefing: In the planning phase of a study designed to evaluate the effectiveness of human-robot interaction through emotional response, it is advisable to define a written study protocol including, in addition to the different phases of the research, a phase of initial briefing to give all the necessary instructions and to obtain consent forms from the participants, as well as a final debriefing phase. A debriefing phase is considered, for example, in [90] by means of a final interview performed after the experiment. Debriefing is a worthwhile means to gather information about more significant aspects of the user’s experience and to ensure that the experience did not have a negative impact on the participants. In this way it is possible to reduce the effect of possible confounds and to get more information for a correct understanding of the physiological data collected during the experiment.

Experimental setting: The laboratory setting remains the primary experimental environment, because the methods for physiological parameters assessment require a controlled setting where it is easier to use technical equipment and to prevent unexpected events that could affect participants’ reaction. In addition, the laboratory setting allows experimenters to operate faster on any malfunction of the robot and on the equipment used for the experiment. One of the main disadvantages of a laboratory setting consists in the minor ecological validity of the results. Field experiments take place in the participant’s natural environment and consequently their results can be more easily generalized. Therefore, whenever it is possible, the experimental setups should be as close as possible to the participants’ real life environment or even so familiar, using known contexts that are representative of real life situations (as a clinic for cognitive rehabilitation) in which the participant is already used to environmental and organizational features.

The lessons learned from this review confirm the value of developing guidelines for both planning and conducting the experiments that implement psychophysiological assessment in HRI. We believe that pulling out guidelines from different experiences in this research field is the best strategy for defining standardized methods to maximize reliability and generalizability of the results.

7. Conclusions

This paper presented a literature review of studies conducted in the HRI field research focused on the psychophysiological assessment as an evaluation method for measuring human emotional reactions during the interaction with a robot. The interest lies in expanding our knowledge of how humans react and what their affective and emotional responses are when they interact and collaborate with a robot. The skills and features of assistive social robots represent an important aspect of human robot interaction which, involving human emotional response, may contribute to affect both the quality and effectiveness of the interaction itself. From the psychophysiological feedback, it is possible to assess both the valence and levels of arousal associated to the interaction experience between a human and a social assistive robot. According to a “circumplex model of affect”, emotional responses occur from two fundamental neurophysiological systems, one related to valence (a pleasure-displeasure continuum) and the other to arousal [15,115]. An emotional response, such as stress, is modeled as high arousal and low valence, while joy would be modeled as high arousal and high valence. This kind of information may be helpful to clarify if the user interprets the interaction with a robot as a negative or a positive experience. Positive experience will facilitate human-robot interaction and encourage collaboration between robot and user. In a context of real utilization, the assessment of individual emotional responses, as well as the analysis of user needs, is essential to the implementation of a robotic device.

We reported significant studies in this research field, and we also included studies implementing fMRI technique to assess the feedback from brain activity relative to how human users respond, perceive and represent robots’ actions, behavior and embodiment during an interaction. From the analysis of this literature we derived the elements useful to contribute to the reinforcement and development of guidelines to standardized methods based on the psychophysiological approach.

Overall, HRI studies based on the psychophysiological approach can be demanding, but following a careful planning of the research and taking into account some useful recommendations, it is possible to obtain encouraging results. The value of the psychophysiological feedback provides a key factor in increasing the knowledge about how the human being reacts during interactions with robots. Using these type of measures, it is possible to explore issues like understanding: (i) how to facilitate human-robot interaction; (ii) which users are more suitable to interact with a particular type of social assistive robot; (iii) what is the adequate level of anthropomorphism; and (iv) what is the best behavior that the robot should adopt relative to its features, the user’s characteristics and the task to be performed for the human. Based on this information, we believe that it will be possible to render the human-robot interaction increasingly effective. A successful interaction will have a positive impact on individual motivation, attitude and performance, and will encourage the user to collaborate with the robot. We believe that the emotional response assessment based on psychophysiological measures could be a helpful approach to match individuals with the most appropriate robotic device depending on their needs and use preferences.