Next Article in Journal
Knowing Knowledge: Epistemological Study of Knowledge in Transformers
Next Article in Special Issue
Improving Usability in Mobile Apps for Residential Energy Management: A Hybrid Approach Using Fuzzy Logic
Previous Article in Journal
Long-Term Clinical and Radiographic Analysis of Platform Matching and Platform Switching Implants in the Esthetic Zone: A Retrospective Cohort Study
Previous Article in Special Issue
Research on Equipment and Algorithm of a Multimodal Perception Gameplay Virtual and Real Fusion Intelligent Experiment
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 13
Issue 2
10.3390/app13020676
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction

by
Huashuo Liu
1,
Fei Wang
1,2 and
Dan Zhang
1,2,*
1
Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China
2
Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(2), 676; https://doi.org/10.3390/app13020676
Submission received: 19 November 2022 / Revised: 17 December 2022 / Accepted: 3 January 2023 / Published: 4 January 2023
(This article belongs to the Special Issue Progress in Human Computer Interaction)
Browse Figures
Review Reports Versions Notes

Abstract

:
The increasingly central role of robotic agents in daily life requires effective human–robot interaction (HRI). For roboticists to optimize interaction design, it is crucial to understand the potential effects of robotic agents on human performance. Yet a systematic specification of contributing factors is lacking, and objective measures of HRI performance are still limited. In these regards, the findings of research on human–human interaction can provide valuable insights. In this review, we break down the complex effects of robotic agents on interacting humans into some basic building blocks based on human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation in HRI. For each effect, we advise on future directions regarding its implication. Furthermore, we propose that the neural correlates of these effects could support real-time evaluation and optimization of HRI with electroencephalograph (EEG)-based brain–computer interface (BCI).

1. Introduction

As we embrace an era of human–robot interaction (HRI), robotic agents are becoming ubiquitous in our daily life, such as in education [1], rehabilitation [2], and elderly care [3]. To achieve effective HRI, not only do we need to improve robot performance, but it is also crucial to understand the potential effects of robotic agents on human performance [4,5]. Efforts have been made to explore how humans interact with robots in recent HRI research [6]. Two of the primary issues in this field, however, are the lack of a systematic specification of critical factors contributing to effective interaction [7], as well as the limited objective measures of HRI performance [7,8]. To address these issues, it might be helpful to draw on the established theories and findings from human–human interaction research, which has been intensively investigated for hundreds of years, as a considerable portion of them have been proved to dominate aspects of HRI similarly [9]. In this review, we summarize the psychological effects that a robotic agent interacting with a human might induce and, more importantly, introduce the approach of integrating electroencephalography (EEG)-based brain–computer interface (BCI) in HRI application, which can be promising for real-time measurement and optimization of these effects. Due to the limited practices carried out in this area so far, we do not intend to conduct a systematic review on existing applications along this line. Rather, we aim to introduce the related findings from human–human studies, discuss their future implication in HRI, and illustrate how to implement EEG-based BCI in different fields of HRI. More specifically, we first break down the complex psychological influence of interacting agents into some of their basic building blocks based on classical human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we briefly introduce the classical findings in human–human studies (for comprehensive information we refer the readers to related systematic reviews) and give advice on future directions concerning its implication for HRI. Furthermore, we believe these findings can not only guide HRI design but also be integrated into application systems to improve task performance more directly. Therefore, we propose that the neural correlates of these effects hold the potential to support real-time evaluation and optimization of HRI based on the BCI technique (see Figure 1). We then provide a general guideline for developing these BCIs, which is further illustrated with examples from different fields of HRI.

2. The Effect of Physical Presence in Social Interaction

2.1. Social Facilitation

The effect of another individual’s physical presence on the performer in human–human interaction, which is “the fundamental form of interindividual influence” ([10], p. 269), is mainly discussed in the frame of the social facilitation effect. The investigation of social facilitation can be traced back to the early days of modern psychology, and the initial findings are somewhat contradictory: some studies [11,12,13] reveal improved performance, while others [14,15,16] indicate an impaired one when working in the presence of others. Zajonc’s influential paper [10] provides an explanation of the phenomenon based on the drive theory, which states that the presence of an audience might increase one’s arousal level (drive), eliciting the dominant response, which is often correct in simple tasks but incorrect in complex tasks. Although various explanations (e.g., [17,18]; see [19] for a review) have been proposed to account for social facilitation, most theories relate its mechanism to the individuals’ arousal level [20].
Subsequent studies further reveal that the social facilitation effect is modulated by multiple factors, such as the type of task, as well as the characteristic of the audience. For example, a familiar or practiced task showed improvement in the audience condition, whereas learners of a task showed no improvement [21]. When the presence of another individual is felt to be reassuring, the level of arousal (and thus the social facilitation effect) might decrease [10]. Moreover, expert observers would elicit larger facilitation than peer observers [22]. With the numerous empirical studies on social facilitation in various domains, human performance could be improved with the intentional control and design of potential social facilitation based on the specific task.

2.2. The Implication of Social Facilitation for HRI

As one of the classic and well-established findings in social psychology, the social facilitation effect reported in human–human interaction has important consequences in many real-world social scenarios. Meanwhile, some HRI studies have examined the effect of robotic presence on human performance. While Schermerhorn et al. [23] and Riether et al. [24] successfully replicated the social facilitation effect with robotic presence, Irfan et al. [25] reported a number of failed attempts, and therefore call for caution when applying results from interpersonal psychology to HRI. As some of the investigations on social facilitation were conducted in the early years, their research methodologies might not fully meet the higher standards required by the scientific communities today. What’s more, unique issues exist regarding the effect of robotic presence on humans in HRI, which cannot be addressed by simply transferring previous human–human interaction findings, such as the anthropomorphic appearance of the robot and the enhanced sensing and memory abilities permitted by electronic sensors and storage, etc. Therefore, it is urgent to conduct state-of-the-art research on social facilitation by robotic presence tailored to the specific factors of concern in HRI. Still, the established paradigms for investigating social facilitation and the lessons on controlling confounding effects from social psychology could be an important reference. The potential effect of robotic presence, whether similar to findings from human–human interaction, should be taken into consideration when designing HRI, as improper robotic presence might be unnecessary or even counterproductive for task performance.

3. The Effect of Motor Actions in Social Interaction

3.1. Motor Priming

While the presence of another individual has an impact on the observee, humans also actively encode the actions of others as observers. The most significant discovery regarding the effect of another agent’s motor actions on humans is the so-called motor resonance, i.e., the phenomenon that the perceiver’s corresponding motor system is activated automatically during action perception [26,27,28]. Interestingly, action observation would influence action execution per se. Due to the motor resonance mechanism which activates the neural substrates of the observed action automatically, the execution of a similar action with the observed one is facilitated or ‘primed,’ which is called motor priming [29]. This priming effect has been found to increase with the similarity between observed and executed actions [30]. However, if the required action to perform is incompatible with the observed one, the prepotent response to perform the primed action (i.e., the observed one) needs to be inhibited, resulting in the interference of action execution. Since the motor priming effect can be investigated empirically, it provides an objective measure of the magnitude of motor resonance.
With its potential to alter motor behavior, the motor priming effect has been increasingly utilized in a number of real-world application scenarios. For example, some motor priming paradigms have been proved to successfully improve functional recovery after injury [31].

3.2. The Implication of Motor Priming for HRI

Given the increasing presence of embodied robots in our lives, the study of motor priming in HRI is particularly important in a variety of contexts, as priming can be advantageous if used properly, but can also be detrimental if not well-controlled. However, mixed findings have been reported regarding whether motor priming exists when actions are performed by robotic agents. While some studies did not find motor priming for robotic agents [32,33,34], more recent studies suggest that human actions can also be primed by robotic actions [35,36,37,38,39]. It is necessary to clarify these divergencies for a unifying theory on motor priming in HRI.
We therefore suggest a more in-depth exploration of motor priming in HRI with strict control for related factors. Some possible accounts for the mixed findings regarding motor priming in HRI have been proposed. Chaminade and Cheng [40] hypothesize that motor resonance is primarily a perceptual (automatic) process in human–human interaction, but may be more susceptible to the attention of the observer in HRI. Therefore, in order to activate motor resonance in HRI, a clear instruction of paying attention to the relationship between the agents and the objects [41] is needed so that the robot’s movement is viewed as transitive goal-directed action. Langer and Levy-Tzedek [42] propose that the seemingly contradictory findings might be explained by different stimulus presentations, i.e., the embodiment of the robot, as well as how its actions are observed (e.g., through static or dynamic images). They assume that generally, actions performed by an embodied and physically present robot would more probably elicit motor resonance and priming in human subjects. A direct test of these hypotheses with well-controlled experiment conditions regarding (but not limited to) these factors is needed in future HRI research.

4. The Effect of Task Co-Representation in Social Interaction

4.1. Joint Action

The effect of task co-representation, which occurs when a corresponding task representation is activated by the awareness of another’s task [43], is primarily investigated in the context of joint action where multiple individuals coordinate to achieve a shared goal. These studies modified classical interference tasks such as the Simon task [44] or the Eriksen flanker task [45] to joint versions, in which the two individuals are responsible for one of the two responses each. A joint spatial compatibility effect similar to that in the original tasks has been observed [46,47,48,49]. This joint compatibility effect implies that individuals in joint action incorporate another person’s task into their own, thus representing their own and others’ actions in a functionally equivalent way.
Task co-representation leads an actor to prepare for and predict the actions of the other actor, which starts even before the initiation of actual movements (i.e., in the action planning phase). With electrophysiological evidence for predictive shared motor planning, Kourtis et al. [50] show that people in joint action represent each other’s action in advance in order to facilitate coordination. They further report similar motor activation during joint action planning with unimanual movements required for each actor and during individual action planning with bimanual movements [51], implying that participants in joint action engage in motor predictions concerning both their own and the co-actor’s contributions.
Joint actions are pervasive in everyday life. Extending from the effect of physical presence or separate motor actions of another agent, findings of task co-representation in joint actions offer valuable insights into the rich space for the cognitive processes that facilitate humans to collaborate with each other.

4.2. The Implication of Joint Action for HRI

Equipped with multiple cognitive mechanisms to form task co-representation, humans are born experts in joint actions. Curioni, Knoblich and Sebanz [43] propose that joint action in humans could be a model for HRI to inspire the design of robotic agents with necessary interactive tools for collaboration with humans. Empirical HRI studies considering task co-representation in joint actions, however, are still limited. Huang et al. [52] identified the coordination methods in a human–human handover task and implemented these strategies in a human–robot condition. They found that if the robot could monitor their human partner and adapt their movement accordingly, the user experience and task performance could be improved. Curioni, Knoblich and Sebanz [43] commented that this result demonstrated the benefits of incorporating the awareness of the partner’s task into the artificial agent’s planning in HRI, which not only improves efficiency but also fosters desirability of the interaction. On the one hand, we suggest that future HRI studies pay special attention to factors influencing the formation of task co-representation for human actors in human–robot joint actions. On the other hand, roboticists should consider combining specialized mechanisms to include their human partner’s actions in robots’ own action planning.

5. Real-Time Evaluation of HRI with BCI

As mentioned above, the optimal level of these psychological effects in interaction often varies depending on specific HRI scenarios. Take the motor priming effect as an example. Motor priming can be used to facilitate the execution of desired movements for human actors. As a preliminary application of robotic motor priming, in a clinical trial [53], adding a robotic device to prime the required movements has been shown to facilitate rehabilitation compared to a task-oriented approach alone. However, sometime this priming effect can also be detrimental. For example, for the robotic “nurses” designed to hand surgical tools to a surgeon, if the velocity of their movement is too fast, the surgeon might be primed by this action, hence endangering the safety of the patient [42]. Therefore, it is of vital importance to measure these critical psychological effects in HRI in a real-time and quantitative way and find their optimal level.
A promising technique that bears such potential is the brain–computer interface (BCI; [54]). Compared to traditional HRI evaluation methods such as questionnaires applied during or after the interaction, BCI can provide a non-disruptive and real-time evaluation of implicit cognitive processes throughout the interaction [55]. Specifically, due to its non-invasive and portable characteristics, electroencephalography (EEG)-based BCI has become the most popular form among other neural recording modalities [56]. Endeavors have been made using EEG-based BCI to evaluate interactive systems (e.g., [57]). In these applications, the measurement of human mental states such as workload, attention, and emotions has been shown to benefit a lot from the EEG technique [55]. We propose that the psychological effect of robotic agents on humans in HRI could also be quantified with BCI.

5.1. EEG Signatures of the Three Psychological Effects in HRI

To achieve this goal, a critical step is to identify the EEG signatures of these effects, which social neuroscientific studies have shed light on. Starting with the neuroscientific findings related to social facilitation, on the one hand, as arousal is a key concept in most social facilitation theories, i.e., the presence of others could increase individuals’ arousal level thus facilitating their dominant response [20], an EEG approach which can provide direct information about arousal level [58,59] bears the potential to measure the extent of the social facilitation effect [60]. Indeed, EEG indicators of arousal such as alpha power, which is negatively linked to arousal level [61,62,63,64], and beta power, which is positively correlated with arousal [65,66], have already been used in studies on social facilitation [20]. On the other hand, neural correlates of some cognitive processes that may contribute to social facilitation, such as performance monitoring, have been intensively investigated with EEG. Specifically, EEG allows us to identify the event-related potentials (ERPs) instantly following specific events, making it ideal for exploring the fast-changing internal processes involved in social interaction. EEG studies on performance monitoring have paid particular attention to the error-related negativity (ERN or Ne; [67,68,69]), which is induced by an erroneous response. Generally, the ERN is believed to reflect how much one cares about the consequence of his action [70]. Enhanced ERNs are reported for errors committed in the presence of peers [71] or under social evaluation [72,73], proposed to arise from the greater threatening value of errors induced by concerns about being judged by others [74].
The neural substrates of motor resonance recognized by brain imaging studies [75,76,77] are called the mirror system. Multiple brain regions involved in motor resonance, such as the ventral premotor cortex and the pars opercularis of the posterior inferior frontal gyrus (Brodmann area 44), as well as the rostral inferior parietal lobule, are recognized by brain imaging studies [75,76,77]. In EEG research, the event-related synchronization and desynchronization of the mu (8–12 Hz) rhythm have been linked to the activity of the mirror system [78,79]. The power of the mu rhythm decreases when subjects perform or observe an action, known as mu suppression (see [80] for a review). In the context of motor priming, an EEG study [81] revealed a more significant mu suppression when the relevance of observed movement to the required action to perform is higher.
The neural processes involved in joint action (mostly with joint interference paradigms) have also been investigated with EEG. The amplitude of the P3 component is larger for their own No-Go trials in a joint action setting than in an individual condition, suggesting an increased need for response inhibition [82,83]. Another ERP component related to the joint action task is the stimulus-locked N2 reflecting response conflict. Ruissen and Bruijn [84] reported that after nasal oxytocin administration, which has been reported to facilitate social behavior by enhancing self–other integration, participants exhibit larger N2 for Go trials in the joint Simon task compared to the individual setting. Peterburs et al. [85] investigated differences in the processing of social (i.e., indicating that a co-actor should respond) and nonsocial No-Go (i.e., signaling that neither actor nor co-actor should respond) cues in a joint flanker task. The conflict-sensitive N2 showed a compatibility effect only for social No-Go cues, and the P3b component related to response discrimination and evaluation was larger for social relative to non-social No-Go.
We summarize the neural correlates related to the potential psychological effects in Table 1. Note that, however, one should be careful to generalize these neural signatures to tasks different from the ones reported in original studies, as many of them depend on task types, as well as the cognitive processes involved.

5.2. Developing BCI Systems for HRI Evaluation and Optimization

In order to evaluate these psychological effects based on human neural activity in HRI, a passive BCI (as opposed to an active BCI designed for controlling external devices with the user’s brain signal) [86] could be utilized. To implement it in a specific HRI scenario, one should first specify the effect of interest in the given task. Although multiple psychological effects might play a role in the same task, in most cases, the target effect would be more focused. For example, researchers might be most interested in measuring the extent of the motor priming effect in a task where a robot and a human are both involved in intricate physical movements. Therefore, the specification of the effect of interest depends on the purpose and type of different HRI tasks and scenarios.
Once the goal of the BCI system is determined, researchers could identify potential neural signatures associated with the target effect by means of a literature search on related neuroscience studies. Special attention should be paid to the paradigm reported in original studies, as the neural correlates of the desired effect could vary across different paradigms. Pilot studies with a similar setting to the real HRI application, if possible, are recommended to confirm the validity of the chosen neural signatures.
A typical BCI system consists of multiple component stages such as data acquisition, preprocessing, dimensionality reduction, feature extraction, classification, and application [87]. To construct a BCI based on the selected features, we recommend classical handbooks on BCI application (e.g., [88,89]) for technical details. The performance of the BCI prototype should then be evaluated in simulated HRI scenarios by comparing the BCI output to other validated measures of the target effect, as well as by evaluating the usability of the system [90]. If the system performance could not meet certain criterions, one might search for other neural signatures or adjust the algorithm of the system modules. An iterative process is often needed in the development of BCI systems.
The implementation of the BCI in real HRI scenarios enables a real-time evaluation of the effect of interest. The robotic agents could be further designed to adapt to the BCI output in a timely manner to optimize task performance and user experience. The general steps for developing BCIs for HRI evaluation and optimization are schematized in Figure 2.

5.3. Application of BCI in Different Fields of HRI

To illustrate the application of BCI combined with the potential psychological effects in HRI, we next describe a few typical scenarios from different HRI fields for exemplary purposes.
The increasing use of robotic assistance in medicine calls attention to their critical influences on humans, which might even endanger the life of patients under extreme circumstances. The extent of the motor priming effect of a robotic nurse’s movement on a human surgeon could be tracked with a BCI approach. A high degree of motor priming effect indicates that the surgeon might be seriously affected by the robotic movement [42], therefore the velocity of the robot’s movement could be set to an appropriate range, matching the habit of the human surgeon in this situation, rather than as fast as possible, in order to avoid detrimental consequences.
Another field with extensive application of robotics is the manufacturing industry. Indeed, joint action for humans and industrial robots has not only been investigated in studies [91,92], but also become a reality in factories [93]. When an industrial robot and a human worker assembly capital goods together, it is important for both of them to form a unified representation of their shared task, which could be monitored be means of a BCI. If the BCI outcome indicates insufficient task co-representation for the human worker (e.g., weak motor planning EEG activities before the movement of the robot), the robot should be adjusted to display more predictable behaviors in line with the human partner’s expectation, which has been proved to facilitate joint action performance in HRI [43].
Meanwhile, the social facilitation effect induced by robotic presence is ubiquitous in most fields of HRI, including the aforementioned scenarios. With a BCI to evaluate the magnitude of such effect in real time, the electronic monitoring function of the robot could be emphasized when beneficial to human performance and weakened when detrimental, resulting in improved productivity.

6. Conclusions

In this article, we review human–human interaction studies investigating three primary psychological effects which may also play an essential role in HRI, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we discuss its implication for HRI and provide suggestions for future studies. Importantly, we propose that the neural signatures related to these effects could support real-time evaluation and optimization of HRI with EEG-based BCI. A general procedure for developing such BCIs is introduced, the implementation of which is further illustrated with examples from different fields of HRI. Therefore, we believe that psychological findings can not only serve as guidelines for HRI design, but also be integrated into application systems for a direct enhancement of HRI performance. Theoretical and technical challenges remain to be solved in future studies, such as the development of more portable EEG acquisition devices for better BCI user experience and the limited examination of the ecological validity of the neural signatures of the psychological effects in different HRI tasks.

Author Contributions

Conceptualization, H.L., F.W. and D.Z.; investigation and visualization, H.L.; writing—original draft preparation, H.L.; writing—review and editing, F.W. and D.Z.; supervision and funding acquisition, F.W and D.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by the National Natural Science Foundation of China, grant number 61977041, and the Tsinghua University Spring Breeze Fund, grant number 2021Z99CFY037.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Toh, L.P.E.; Causo, A.; Tzuo, P.-W.; Chen, I.M.; Yeo, S.H. A review on the use of robots in education and young children. J. Educ. Technol. Soc. 2016, 19, 148–163. [Google Scholar]
  2. Li, B.; Li, G.; Sun, Y.; Jiang, G.; Kong, J.; Jiang, D. A review of rehabilitation robot. In Proceedings of the 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China, 19–21 May 2017; pp. 907–911. [Google Scholar]
  3. Abdi, J.; Al-Hindawi, A.; Ng, T.; Vizcaychipi, M.P. Scoping review on the use of socially assistive robot technology in elderly care. BMJ Open 2018, 8, e018815. [Google Scholar] [CrossRef] [Green Version]
  4. Licklider, J.C.R. Man-Computer Symbiosis. Ire. T. Hum. Fact. Electf. 1960, 1, 4–11. [Google Scholar] [CrossRef] [Green Version]
  5. Henschel, A.; Hortensius, R.; Cross, E.S. Social Cognition in the Age of Human–Robot Interaction. Trends Neurosci. 2020, 43, 373–384. [Google Scholar] [CrossRef]
  6. Bethel, C.L.; Murphy, R.R. Review of Human Studies Methods in HRI and Recommendations. Int. J. Soc. Robot. 2010, 2, 347–359. [Google Scholar] [CrossRef]
  7. Wiese, E.; Metta, G.; Wykowska, A. Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social. Front. Psychol. 2017, 8, 1663. [Google Scholar] [CrossRef] [Green Version]
  8. Kompatsiari, K.; Pérez-Osorio, J.; Tommaso, D.D.; Metta, G.; Wykowska, A. Neuroscientifically-Grounded Research for Improved Human-Robot Interaction. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3403–3408. [Google Scholar]
  9. Krämer, N.C.; von der Pütten, A.; Eimler, S. Human-Agent and Human-Robot Interaction. In Human-Computer Interaction: The Agency Perspective; Zacarias, M., de Oliveira, J.V., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 215–240. [Google Scholar]
  10. Zajonc, R.B. Social Facilitation. Science 1965, 149, 269–274. [Google Scholar] [CrossRef]
  11. Triplett, N. The Dynamogenic Factors in Pacemaking and Competition. Am. J. Psychol. 1898, 9, 507–533. [Google Scholar] [CrossRef] [Green Version]
  12. Meumann, E. Haus-und Schularbeit Experimente an Kindern der Volksschule; Kessinger Publishing: Whitefish, MT, USA, 1904. [Google Scholar]
  13. Dashiell, J.F. An experimental analysis of some group effects. J. Abnorm. Soc. Psychol. 1930, 25, 190–199. [Google Scholar] [CrossRef]
  14. Moore, H.T. Laboratory Tests of Anger, Fear and Sex Interest. Am. J. Psychol. 1917, 28, 390–395. [Google Scholar] [CrossRef]
  15. Ekdahl, A.G. Effects of attitude on free word association time. Genet. Psychol. Monogr. 1929, 5, 253–338. [Google Scholar]
  16. Burri, C. The influence of an audience upon recall. J. Educ. Psychol. 1931, 22, 683–690. [Google Scholar] [CrossRef]
  17. Bond, C.F. Social facilitation: A self-presentational view. J. Personal. Soc. Psychol. 1982, 42, 1042–1050. [Google Scholar] [CrossRef]
  18. Baron, R.S. Distraction-Conflict Theory: Progress and Problems. Adv. Exp. Soc. Psychol. 1986, 19, 1–40. [Google Scholar] [CrossRef]
  19. Strauss, B. Social facilitation in motor tasks: A review of research and theory. Psychol. Sp. Exerc. 2002, 3, 237–256. [Google Scholar] [CrossRef]
  20. Liu, N.; Yu, R.; Yang, L.; Lin, X. Gender composition mediates social facilitation effect in co-action condition. Sci. Rep. 2017, 7, 15073. [Google Scholar] [CrossRef] [Green Version]
  21. Dube, S.K.; Tatz, S.J. Audience effects in tennis performance. Percept. Mot. Ski. 1991, 73, 844–846. [Google Scholar] [CrossRef]
  22. Oviatt, D.P. Social Facilitation and Motor Performance: A Meta-Analysis. Master’s Thesis, University of Maryland, College Park, MD, USA, 2005. [Google Scholar]
  23. Schermerhorn, P.; Scheutz, M.; Crowell, C.R. Robot social presence and gender: Do females view robots differently than males? In Proceedings of the Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, Amsterdam, The Netherlands, 12–15 March 2008; pp. 263–270. [Google Scholar]
  24. Riether, N.; Hegel, F.; Wrede, B.; Horstmann, G. Social facilitation with social robots? In Proceedings of the 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boston, MA, USA, 5–8 March 2012; pp. 41–47. [Google Scholar]
  25. Irfan, B.; Kennedy, J.; Lemaignan, S.; Papadopoulos, F.; Senft, E.; Belpaeme, T. Social Psychology and Human-Robot Interaction: An Uneasy Marriage. In Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5–8 March 2018; pp. 13–20. [Google Scholar]
  26. Buccino, G.; Binkofski, F.; Fink, G.R.; Fadiga, L.; Fogassi, L.; Gallese, V.; Seitz, R.J.; Zilles, K.; Rizzolatti, G.; Freund, H.J. Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study: Cortical activation during action observation. Eur. J. Neurosci. 2001, 13, 400–404. [Google Scholar] [CrossRef]
  27. Buccino, G.; Binkofski, F.; Riggio, L. The mirror neuron system and action recognition. Brain Lang 2004, 89, 370–376. [Google Scholar] [CrossRef]
  28. Fadiga, L.; Craighero, L.; Olivier, E. Human motor cortex excitability during the perception of others’ action. Curr. Opin. Neurobiol. 2005, 15, 213–218. [Google Scholar] [CrossRef]
  29. Wearden, J.H.; Edwards, H.; Fakhri, M.; Percival, A. Why “sounds are judged longer than lights”: Application of a model of the internal clock in humans. Q. J. Exp. Psychol. Sect. B 1998, 51, 97–120. [Google Scholar]
  30. Brass, M.; Bekkering, H.; Wohlschläger, A.; Prinz, W. Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and Imitative Cues. Brain Cogn. 2000, 44, 124–143. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Madhavan, S.; Stoykov, M.E. Editorial: Motor Priming for Motor Recovery: Neural Mechanisms and Clinical Perspectives. Front. Neurol. 2017, 8, 448. [Google Scholar] [CrossRef] [PubMed]
  32. Castiello, U.; Lusher, D.; Mari, M.; Edwards, M.; Humphreys, G.W. Observing a human and a robotic hand grasping an object: Differential motor priming effects. In Common Mechanisms in Perception and Action, Attention and Performance XIX; Prinz, W., Hommel, B., Eds.; Oxford University Press: New York, NY, USA, 2002; pp. 315–333. [Google Scholar]
  33. Tai, Y.F.; Scherfler, C.; Brooks, D.J.; Sawamoto, N.; Castiello, U. The Human Premotor Cortex Is Mirror Only for Biological Actions. Curr. Biol. 2004, 14, 117–120. [Google Scholar] [CrossRef] [PubMed]
  34. Kilner, J.M.; Paulignan, Y.; Blakemore, S.J. An Interference Effect of Observed Biological Movement on Action. Curr. Biol. 2003, 13, 522–525. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Pierno, A.C.; Mari, M.; Lusher, D.; Castiello, U. Robotic movement elicits visuomotor priming in children with autism. Neuropsychologia 2008, 46, 448–454. [Google Scholar] [CrossRef]
  36. Eizicovits, D.; Edan, Y.; Tabak, I.; Levy-Tzedek, S. Robotic gaming prototype for upper limb exercise: Effects of age and embodiment on user preferences and movement. Restor. Neurol. Neurosci. 2018, 36, 261–274. [Google Scholar] [CrossRef] [Green Version]
  37. Kashi, S.; Levy-Tzedek, S. Smooth leader or sharp follower? Playing the mirror game with a robot. Restor. Neurol. Neurosci. 2018, 36, 147–159. [Google Scholar] [CrossRef] [Green Version]
  38. Vannucci, F.; Sciutti, A.; Lehman, H.; Sandini, G.; Nagai, Y.; Rea, F. Cultural differences in speed adaptation in human-robot interaction tasks. Paladyn J. Behav. Robot. 2019, 10, 256–266. [Google Scholar] [CrossRef]
  39. Press, C.; Bird, G.; Flach, R.; Heyes, C. Robotic movement elicits automatic imitation. Cogn. Brain Res. 2005, 25, 632–640. [Google Scholar] [CrossRef]
  40. Chaminade, T.; Cheng, G. Social cognitive neuroscience and humanoid robotics. J. Physiol. 2009, 103, 286–295. [Google Scholar] [CrossRef] [PubMed]
  41. Gazzola, V.; Rizzolatti, G.; Wicker, B.; Keysers, C. The anthropomorphic brain: The mirror neuron system responds to human and robotic actions. Neuroimage 2007, 35, 1674–1684. [Google Scholar] [CrossRef]
  42. Langer, A.; Levy-Tzedek, S. Priming and Timing in Human-Robot Interactions. In Modelling Human Motion: From Human Perception to Robot Design; Noceti, N., Sciutti, A., Rea, F., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 335–350. [Google Scholar]
  43. Curioni, A.; Knoblich, G.; Sebanz, N. Joint Action in Humans: A Model for Human-Robot Interactions. In Humanoid Robotics: A Reference; Goswami, A., Vadakkepat, P., Eds.; Springer: Dordrecht, The Netherlands, 2016; pp. 1–19. [Google Scholar]
  44. Simon, J.R.; Small, A.M. Processing auditory information: Interference from an irrelevant cue. J. Appl. Psychol. 1969, 53, 433–435. [Google Scholar] [CrossRef]
  45. Eriksen, B.A.; Eriksen, C.W. Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 1974, 16, 143–149. [Google Scholar] [CrossRef] [Green Version]
  46. Sebanz, N.; Knoblich, G.; Prinz, W. Representing others’ actions: Just like one’s own? Cognition 2003, 88, B11–B21. [Google Scholar] [CrossRef] [PubMed]
  47. Atmaca, S.; Sebanz, N.; Knoblich, G. The joint flanker effect: Sharing tasks with real and imagined co-actors. Exp. Brain Res. 2011, 211, 371–385. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Ciardo, F.; Lugli, L.; Nicoletti, R.; Rubichi, S.; Iani, C. Action-space coding in social contexts. Sci. Rep. 2016, 6, 22673. [Google Scholar] [CrossRef] [Green Version]
  49. Ciardo, F.; Wykowska, A. Response Coordination Emerges in Cooperative but Not Competitive Joint Task. Front. Psychol. 2018, 9, 1919. [Google Scholar] [CrossRef] [Green Version]
  50. Kourtis, D.; Sebanz, N.; Knoblich, G. Predictive representation of other people’s actions in joint action planning: An EEG study. Soc. Neurosci. 2013, 8, 31–42. [Google Scholar] [CrossRef]
  51. Kourtis, D.; Knoblich, G.; Woźniak, M.; Sebanz, N. Attention allocation and task representation during joint action planning. J. Cogn. Neurosci. 2014, 26, 2275–2286. [Google Scholar] [CrossRef] [Green Version]
  52. Huang, C.-M.; Cakmak, M.; Mutlu, B. Adaptive Coordination Strategies for Human-Robot Handovers. Robot. Sci. Syst. 2015, 11, 1–11. [Google Scholar]
  53. Hsieh, Y.W.; Wu, C.Y.; Wang, W.E.; Lin, K.C.; Chang, K.C.; Chen, C.C.; Liu, C.T. Bilateral robotic priming before task-oriented approach in subacute stroke rehabilitation: A pilot randomized controlled trial. Clin. Rehabil. 2017, 31, 225–233. [Google Scholar] [CrossRef] [PubMed]
  54. Gao, X.; Wang, Y.; Chen, X.; Gao, S. Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends Cogn. Sci. 2021, 25, 671–684. [Google Scholar] [CrossRef] [PubMed]
  55. Frey, J.; Mühl, C.; Lotte, F.; Hachet, M. Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction. In Proceedings of the PhyCS-International Conference on Physiological Computing Systems, Lisbonne, Portugal, 7–9 January 2014; pp. 214–223. [Google Scholar]
  56. Rashid, M.; Sulaiman, N.; PP Abdul Majeed, A.; Musa, R.M.; Ab Nasir, A.F.; Bari, B.S.; Khatun, S. Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review. Front. Neurorobotics 2020, 14, 25. [Google Scholar] [CrossRef] [PubMed]
  57. Cano, S.; Soto, J.; Acosta, L.; Peñeñory, V.M.; Moreira, F. Using Brain-Computer Interface to evaluate the User eXperience in interactive systems. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2022, 1–9. [Google Scholar] [CrossRef]
  58. Klimesch, W. Alpha-band oscillations, attention, and controlled access to stored information. Trends Cogn. Sci. 2012, 16, 606–617. [Google Scholar] [CrossRef] [Green Version]
  59. Orekhova, E.V.; Stroganova, T.A.; Posikera, I.N. Alpha activity as an index of cortical inhibition during sustained internally controlled attention in infants. Clin. Neurophysiol. 2001, 112, 740–749. [Google Scholar] [CrossRef]
  60. Liu, N.; Yu, R. Influence of social presence on eye movements in visual search tasks. Ergonomics 2017, 60, 1667–1681. [Google Scholar] [CrossRef]
  61. Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 1999, 29, 169–195. [Google Scholar] [CrossRef]
  62. Baek, H.; Min, B.-K. Blue light aids in coping with the post-lunch dip: An EEG study. Ergonomics 2015, 58, 803–810. [Google Scholar] [CrossRef]
  63. Barry, R.J.; Clarke, A.R.; McCarthy, R.; Selikowitz, M.; Rushby, J.A.; Ploskova, E. EEG differences in children as a function of resting-state arousal level. Clin. Neurophysiol. 2004, 115, 402–408. [Google Scholar] [CrossRef] [PubMed]
  64. Tran, Y.; Craig, A.; McIsaac, P. Extraversion–introversion and 8–13 Hz waves in frontal cortical regions. Pers. Indiv. Differ. 2001, 30, 205–215. [Google Scholar] [CrossRef]
  65. Lazzaro, I.; Gordon, E.; Li, W.; Lim, C.L.; Plahn, M.; Whitmont, S.; Clarke, S.; Barry, R.J.; Dosen, A.; Meares, R. Simultaneous EEG and EDA measures in adolescent attention deficit hyperactivity disorder. Int. J. Psychophysiol. 1999, 34, 123–134. [Google Scholar] [CrossRef] [PubMed]
  66. Mann, C.A.; Lubar, J.F.; Zimmerman, A.W.; Miller, C.A.; Muenchen, R.A. Quantitative analysis of EEG in boys with attention-deficit-hyperactivity disorder: Controlled study with clinical implications. Pediatr. Neurol. 1992, 8, 30–36. [Google Scholar] [CrossRef] [PubMed]
  67. Ullsperger, M.; Danielmeier, C.; Jocham, G. Neurophysiology of performance monitoring and adaptive behavior. Physiol. Rev. 2014, 94, 35–79. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Weinberg, A.; Dieterich, R.; Riesel, A. Error-related brain activity in the age of RDoC: A review of the literature. Int. J. Psychophysiol. 2015, 98, 276–299. [Google Scholar] [CrossRef] [PubMed]
  69. Gehring, W.J.; Goss, B.; Coles, M.G.H.; Meyer, D.E.; Donchin, E. The Error-Related Negativity. Perspect. Psychol. Sci. 2018, 13, 200–204. [Google Scholar] [CrossRef]
  70. Alanis, J.C.G.; Baker, T.E.; Peper, M.; Chavanon, M.-L. Social context effects on error-related brain activity are dependent on interpersonal and achievement-related traits. Sci. Rep. 2019, 9, 1728. [Google Scholar] [CrossRef] [Green Version]
  71. Kim, E.Y.; Iwaki, N.; Uno, H.; Fujita, T. Error-Related Negativity in Children: Effect of an Observer. Dev. Neuropsychol. 2005, 28, 871–883. [Google Scholar] [CrossRef]
  72. Hajcak, G.; Moser, J.S.; Yeung, N.; Simons, R.F. On the ERN and the significance of errors. Psychophysiology 2005, 42, 151–160. [Google Scholar] [CrossRef]
  73. Riesel, A.; Weinberg, A.; Endrass, T.; Kathmann, N.; Hajcak, G. Punishment has a lasting impact on error-related brain activity. Psychophysiology 2012, 49, 239–247. [Google Scholar] [CrossRef] [PubMed]
  74. Van Meel, C.S.; Van Heijningen, C.A. The effect of interpersonal competition on monitoring internal and external error feedback. Psychophysiology 2010, 47, 213–222. [Google Scholar] [CrossRef] [PubMed]
  75. Decety, J.; Grèzes, J.; Costes, N.; Perani, D.; Jeannerod, M.; Procyk, E.; Grassi, F.; Fazio, F. Brain activity during observation of actions. Influence of action content and subject’s strategy. Brain 1997, 120, 1763–1777. [Google Scholar] [CrossRef] [PubMed]
  76. Iacoboni, M.; Woods, R.P.; Brass, M.; Bekkering, H.; Mazziotta, J.C.; Rizzolatti, G. Cortical mechanisms of human imitation. Science 1999, 286, 2526–2528. [Google Scholar] [CrossRef] [PubMed]
  77. Rizzolatti, G.; Fadiga, L.; Matelli, M.; Bettinardi, V.; Paulesu, E.; Perani, D.; Fazio, F. Localization of grasp representations in humans by PET: 1. Observation versus execution. Exp. Brain Res. 1996, 111, 246–252. [Google Scholar] [CrossRef]
  78. Muthukumaraswamy, S.D.; Johnson, B.W.; McNair, N.A. Mu rhythm modulation during observation of an object-directed grasp. Cogn. Brain Res. 2004, 19, 195–201. [Google Scholar] [CrossRef]
  79. Oberman, L.M.; Hubbard, E.M.; McCleery, J.P.; Altschuler, E.L.; Ramachandran, V.S.; Pineda, J.A. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Cogn. Brain Res. 2005, 24, 190–198. [Google Scholar] [CrossRef]
  80. Pineda, J.A. The functional significance of mu rhythms: Translating seeing and hearing into doing. Brain Res. Rev. 2005, 50, 57–68. [Google Scholar] [CrossRef]
  81. Silas, J.; Levy, J.P.; Holmes, A. Sensitivity of mu rhythm modulation to the relevance of an observed movement but not to goal congruency. Int. J. Psychophysiol. 2012, 85, 168–173. [Google Scholar] [CrossRef]
  82. Sebanz, N.; Bekkering, H.; Knoblich, G. Joint action: Bodies and minds moving together. Trends Cogn. Sci. 2006, 10, 70–76. [Google Scholar] [CrossRef]
  83. Sebanz, N.; Knoblich, G.; Prinz, W.; Wascher, E. Twin Peaks: An ERP Study of Action Planning and Control in Coacting Individuals. J. Cogn. Neurosci. 2006, 18, 859–870. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  84. Ruissen, M.I.; Bruijn, E.R.A.d. Is it me or is it you? Behavioral and electrophysiological effects of oxytocin administration on self-other integration during joint task performance. Cortex 2015, 70, 146–154. [Google Scholar] [CrossRef] [PubMed]
  85. Peterburs, J.; Liepelt, R.; Voegler, R.; Ocklenburg, S.; Straube, T. It’s not me, it’s you-Differential neural processing of social and non-social nogo cues in joint action. Soc. Neurosci. 2017, 14, 114–124. [Google Scholar] [CrossRef]
  86. Han, Y.; Ziebell, P.; Riccio, A.; Halder, S. Two sides of the same coin: Adaptation of BCIs to internal states with user-centered design and electrophysiological features. Brain Comput. Interfaces 2022, 9, 102–114. [Google Scholar] [CrossRef]
  87. Singh, H.; Daly, I. Translational Algorithms: The Heart of a Brain Computer Interface. In Brain-Computer Interfaces: Current Trends and Applications; Hassanien, A.E., Azar, A.T., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 97–121. [Google Scholar]
  88. Nam, C.S.; Nijholt, A.; Lotte, F. Brain–Computer Interfaces Handbook; CRC Press: Boca Raton, FL, USA, 2018. [Google Scholar]
  89. Clerc, M.; Bougrain, L.; Lotte, F. Brain-Computer Interfaces 1: Methods and Perspectives; John and Wiley and Sons: Hoboken, NJ, USA, 2016. [Google Scholar]
  90. Kübler, A.; Holz, E.M.; Riccio, A.; Zickler, C.; Kaufmann, T.; Kleih, S.C.; Staiger-Sälzer, P.; Desideri, L.; Hoogerwerf, E.-J.; Mattia, D. The User-Centered Design as Novel Perspective for Evaluating the Usability of BCI-Controlled Applications. PLoS ONE 2014, 9, e112392. [Google Scholar] [CrossRef] [Green Version]
  91. Lenz, C.; Nair, S.; Rickert, M.; Knoll, A.; Rosel, W.; Gast, J.; Bannat, A.; Wallhoff, F. Joint-action for humans and industrial robots for assembly tasks. In Proceedings of the RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication, Munich, Germany, 1–3 August 2008; pp. 130–135. [Google Scholar]
  92. Nikolaidis, S.; Lasota, P.; Rossano, G.; Martinez, C.; Fuhlbrigge, T.; Shah, J. Human-robot collaboration in manufacturing: Quantitative evaluation of predictable, convergent joint action. In Proceedings of the IEEE ISR, Seoul, Republic of Korea, 24–26 October 2013; pp. 1–6. [Google Scholar]
  93. Hentout, A.; Aouache, M.; Maoudj, A.; Akli, I. Human–robot interaction in industrial collaborative robotics: A literature review of the decade 2008–2017. Adv. Robot. 2019, 33, 764–799. [Google Scholar] [CrossRef]
Applsci 13 00676 g001 550
Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in human-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and optimization.
Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in human-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and optimization.
Applsci 13 00676 g001
Applsci 13 00676 g002 550
Figure 2. General steps for developing BCIs for HRI evaluation and optimization.
Figure 2. General steps for developing BCIs for HRI evaluation and optimization.
Applsci 13 00676 g002
Table
Table 1. Psychological effects in social interaction and their potential EEG signatures.
Table 1. Psychological effects in social interaction and their potential EEG signatures.
Psychological EffectsRelated FactorCognitive Processes Involved
(And Their Potential EEG Signatures *)		References
Social facilitationPhysical presenceArousal level
(alpha and beta power)		[61,62,63,64,65,66]
Performance monitoring
(error-related negativity, ERN)		[71,72,73]
Motor primingMotor actionSensorimotor mirroring
(event-related mu suppression)		[80,81]
Joint actionTask
co-representation		Response conflict
(N2 component)		[84,85]
Stimulus classification, response discrimination
(P3 component)		[50,82,83,85]
Action planning
(contingent negative variation, CNV; movement-related potential, MRP)		[50,51]
* Note that the EEG signatures of these psychological effects are contingent on specific tasks and cognitive processes involved in HRI.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, H.; Wang, F.; Zhang, D. Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction. Appl. Sci. 2023, 13, 676. https://doi.org/10.3390/app13020676

AMA Style

Liu H, Wang F, Zhang D. Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction. Applied Sciences. 2023; 13(2):676. https://doi.org/10.3390/app13020676

Chicago/Turabian Style

Liu, Huashuo, Fei Wang, and Dan Zhang. 2023. "Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction" Applied Sciences 13, no. 2: 676. https://doi.org/10.3390/app13020676

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop