Robot-Assisted Therapy for Learning and Social Interaction of Children with Autism Spectrum Disorder

Abstract

This paper puts forward the potential for designing a parrot-inspired robot and an indirect teaching technique, the adapted model-rival method (AMRM), to help improve learning and social interaction abilities of children with autism spectrum disorder. The AMRM was formulated by adapting two popular conventional approaches, namely, model-rival method and label-training procedure. In our validation trials, we used a semi-autonomous parrot-inspired robot, called KiliRo, to simulate a set of autonomous behaviors. A proposed robot-assisted therapy using AMRM was pilot tested with nine children with autism spectrum disorder for five consecutive days in a clinical setting. We analyzed the facial expressions of children when they interacted with KiliRo using an automated emotion recognition and classification system, Oxford emotion API (Application Programming Interface). Results provided some indication that the children with autism spectrum disorder appeared attracted and happy to interact with the parrot-inspired robot. Short qualitative interviews with the children’s parents, the pediatrician, and the child psychologist who participated in this pilot study, also acknowledged that the proposed parrot-inspired robot and the AMRM may have some merit in aiding in improving learning and social interaction abilities of children with autism spectrum disorder.

1. Introduction

The increasing deployment of robots in recent decades has inspired new boundaries for human-robot interactions from manufacturing to health industries [1,2,3,4,5]. Particularly, assistive robotics has found new directions in recent years and is being used in numerous applications, including, elderly care and autism therapy [6,7,8,9]. Furthermore, bio-inspired robotics has received extensive attention in therapeutic settings by providing physiological and psychological benefits [10,11,12]. Animal-like robots have received especially notable acceptance in therapeutic settings, as they could provide benefits of animal-assisted therapy (AAT), while overcoming the shortfalls such as biting, allergies, and diseases. For instance, Wada and Shibata [13] used a seal-like pet robot, PARO, to investigate the psychological and social effects in the elderly. The study reported that social interaction of participants increased after interacting with the robot. Additionally, measures of 17-ketosteroid sulfate and 17-hydroxycorticosteroids levels before and after the intervention were taken using urinary samples. The results indicated that the reactions of participants’ vital organs to stress improved after interaction with the robot. Another study used AIBO, a dog-like robot, to describe the human interactions with a pet-like robot [14]. Jaichandar et al. investigated human psychological changes using infrared thermography, galvanic skin resistance, and heart rate during interaction with a dog-like pet robot [15]. Numerous studies have reported evidence for therapeutic benefits when using assistive robots with children with autism spectrum disorder [16,17]. As an illustration, the human-like robot Nao has been used to improve social engagement among children with autism spectrum disorder through imitation tasks [16]. Robota, a child-like robot, has been used to investigate the benefits of using a robot imitator to assess imitation ability of children with autism spectrum disorder [17]. Several pet-inspired companion robots have been researched within therapeutic settings where the robots mimic the form, behavior and multimodal nature of human-pet interactions. Previous work covers studies involving several pets ranging from seal, dog, cat, and bear. These animals have been studied; robotic platforms inspired by them have been developed and evaluated in the context of therapeutic scenarios. Most studies presented in the literature points to some improvement in physiological, psychological and social aspects within therapeutic settings using robots.

Among several animals adopted for robot-assisted therapy, one very relevant species that has received little attention is parrot. Parrots are naturally able to emulate human speech and engage in two-way communications, unlike many of the other animals that provided inspiration to build robotic platforms [18]. This unique feature has the potential to be very useful in teaching children with autism spectrum disorder and to improve their social interaction. Parrots have also been used in AAT settings such as post-traumatic stress disorder, bipolar disorder, and psychiatric tendencies. Nevertheless, parrot-assisted therapy carries the same challenges as other AATs. Particularly, having large and sharp beaks, parrot biting could be very harmful to children. Hence, we developed a parrot-inspired robot, called KiliRo, with the aim to produce the same benefits in improving learning and social interaction abilities of children with autism spectrum disorder without risks of being harmed by real parrots.

Model-rival method and label-training procedures have been used for training parrots in previous studies [19,20]. Dietmar Todt developed the model-rival method for teaching parrots to reproduce human speech [19]. In general, the model-rival method is a technique used in training animals to learn the differentiating features of a target object, such as name and color by observing the interaction between the trainer and potential competitor involved in the conversation about these features. The model-rival method uses two people to play the part of parrot peers. For instance, the parrot observes the individual handling a targeted item. One person trains the second (the model/rival) by showing and asking questions about the item (e.g., "What color?”). The person acting as a trainer rewards the model/rival for each correct response. The second person is a model for the parrot’s responses and its rival for the trainer’s attention, and demonstrates lack of reward as consequences of wrong answers. The label-training procedure is a variant of the model-rival method developed by Irene Pepperberg, an animal psychologist to teach Alex, the African grey parrot [21]. In this study, Pepperberg used two researchers to converse in the presence of Alex. The technique resulted in Alex learning to talk and improve cognitive abilities. Alex has defined a new milestone in animal study by exhibiting its ability to engage in two-way communication with humans. As reported by Pepperberg, Alex could identify 50 different objects, and recognize quantities up to six. It was also able to differentiate five shapes and seven colors, and had a vocabulary of around 100 words.

We adapted these models to develop a novel method called Adapted Model-Rival Method (AMRM) for teaching children with autism spectrum disorder and to improve their social interaction abilities. AMRM is based on the theory that the abilities of KiliRo including speaking, mimicking, and learning will instigate children to learn new things. For example, when the researcher teaches KiliRo in the presence of participants, the learning ability of KiliRo would attract participants to get involved and create a new path for teaching children with autism spectrum disorder. Our interest in this pilot study is to investigate the applicability of this method specifically for children with autism spectrum disorder. Our focus in this experiment is on identifying the factors in KiliRo to which the participants might be attracted to and under which circumstances they might be interested to learn what KiliRo is shown to be learning. We also examined the likeability, acceptance and interaction interests of participants with the robot. Psychological reactions were evaluated using facial images captured during the study. For the experiments presented in this paper, we used a semi-autonomous robot and simulated a set of autonomous behaviors in KiliRo. Using simulated experiments, researchers can test the ability of a system before it is completely developed. The simulated experiment helps reducing the research cost, manpower and time spent on developing the system before it is built. Previous studies have reported usefulness of simulated experiments in research settings [22,23,24]. Hence, in this study, we propose to evaluate the effects on improvements in learning and social interaction abilities of children with autism spectrum disorder through AMRM and parrot-inspired robot using simulated experiments and use the results for robot’s further development. We set two hypotheses in this pilot study: (1) Children with autism spectrum disorder would find the parrot-inspired robot likeable. (2) AMRM would be beneficial to improve those children’s learning and social interaction abilities.

2. Materials and Methods

2.1. KiliRo–System Architecture

A parrot-like robot consisting of one head, two eyes, two legs, two wings and a tail was used in this study. Figure 1 shows the semi-autonomous robot used in this study. KiliRo has two degrees of freedom in each leg, and the head can move left, right, up and down. The tail has navigation ability for moving left and right. Two wireless cameras with microphone were used for real-time video and audio streaming. One speaker was attached on the robot for enabling two-way communication. The robot was fabricated to simulate a parrot-like morphology.

In this study, a human operator remotely controlled the locomotion of KiliRo in such a way to appear to the participants as being autonomous and able to make decisions by itself. The synthesized behaviors included learning alphabets, numbers and recognizing humans. A wireless camera module was used to achieve vision capability in the robot. Similarly, a microphone and a speaker were used to enable KiliRo to mimic hearing and speaking. Infrared remote control was used to control the robot’s head, legs, and tail locomotion. The operator remotely activated the behaviors in KiliRo, based on a live video feed of the situation in which the participants interacted with the robot. For instance, the remote robot operator identified the person standing in front of the robot through a wireless camera and used a microphone to initiate and maintain communication between the robot and the person.

2.2. Procedure

We used ‘The Childhood Autism spectrum disorder Spectrum Test’ (CAST) questionnaire to evaluate the severity of autism spectrum disorder symptoms in the participants. CAST, formerly the ‘Childhood Asperger’s Syndrome Test’ consists of 39 yes or no questions for children’s parents [25]. CAST was developed at the University of Cambridge at the Autism spectrum disorder Research Centre (ARC). We approached local schools for children with autism spectrum disorder using convenience sampling. A formal open invitation was sent to 19 potential participants’ parents, who were given one week to respond to the invitation. The contact details of potential participants were obtained from three special schools in Chennai, India. Information containing the details of the study was sent along with the invitation. Parents, who expressed willingness to involve their children in the study, were requested to complete the CAST questionnaire to evaluate the autism spectrum disorder behavior of their children. Out of 12 respondents, 3 participants were excluded from the study due to CAST scores below 15. The study was performed with nine participants aged between 6 and 16 years. The mean age of participating children was 9.33 with a standard deviation of 3.39.

The study involved both qualitative and quantitative methods. For qualitative analyses, we used informal and semi-formal interviews with nine parents of participating children, a pediatrician and a child psychologist. An open informal interview was conducted with no pre-defined questions by the lead researcher who ran the sessions. The semi-formal interview was conducted as one-to-one session where the researcher had a few pre-defined questions to the participants. These questions include asking about useful teaching methods for children with autism spectrum disorder, their observation capability, indirect teaching methods and their usefulness, children with autism spectrum disorder and their bond with animals, and respondents’ views on using a robot with parrot-like morphology for teaching children with autism spectrum disorder. Interviews with parents were conducted on the third day of the study, and interviews with the pediatrician and the child psychologist were conducted on the fourth and fifth day of the study respectively. Results were analyzed using a descriptive content analysis [26]. This method is particularly useful to gather straight descriptive information about phenomena.

For quantitative analyses, we used a closed format-questionnaire with the same respondents as in qualitative analysis. The questionnaire used in this study has 14 questions and used a 5-level ‘Likert scale’ format ranging from strongly disagree to strongly agree for all questions except for two questions that followed a multiple-choice format. The questions were prepared to evaluate the acceptance of the robot and AMRM and to understand the expected features of KiliRo. The questions used in the questionnaire are illustrated below:

• AMRM could help improve learning abilities of children with autism.
• AMRM could help improve social interaction abilities of children with autism.
• Parrot-like morphology might attract children with autism.
• I recommend the following features in KiliRo (options: talking, walking, flying, wing flapping, respond to sound, respond to touch, recognize face, and recognize voice).
• KiliRo could be a companion for children with autism.
• Parrot-like morphology is suitable teaching children with autism.
• My child tried to communicate with the robot during the study.
• My child observed the learning abilities of KiliRo during the study.
• My child accepted KiliRo as his/her companion.
• I recommend KiliRo for teaching children with autism.
• My child was happy to interact with KiliRo.
• AMRM might help improve cognitive abilities of children with autism.
• I recommend the following morphology for robot-assisted therapy (options: parrot, dog, cat, bear, seal, and human).
• I would use KiliRo for teaching my child.

We have involved parents of participating children, the pediatrician, and the child psychologist to evaluate the study on parental, medical and psychological point of views. The pediatrician and the child psychologist were selected based on their expertise in autism spectrum disorder, willingness and availability to participate in the study. In addition to collecting quantitative data using a questionnaire, an automated emotion recognition technique, Oxford emotion API [27], was used to classify facial images of participating children into one of eight different types of emotions while they interacted with KiliRo. Figure 2 shows an example of emotion values identified using the Oxford emotion API system. In this figure, the emotion values of the primary researcher are illustrated while the other faces in the picture is captured.

The study was divided into 15 sessions, each 15 min in duration, conducted over a period of 5 consecutive days and with 3 sessions each day. The sessions were conducted between 10 am and 12 pm with 30 min break between the sessions. All children participated in all sessions performed. The study was conducted at the same venue with the same participants. In the study, there were four tasks for the robot to perform as described below.

In the first task, a member of the research team remotely operated the robot to walk forward and backward, move its tail up and down, and move its head up and down. In addition to that, KiliRo spoke a few words and communicated with a member of the research team leading the session. This task also involved the participating children where the behaviors of the robot were synthesized by the remote operator in response to the observed situations. This task was conducted to validate the acceptance, likeability, and interaction interest of children towards the KiliRo robot and its parrot-inspired morphology.

In the second task, the researcher taught letters of the English alphabet to KiliRo. After a few minutes of training, KiliRo’s learning ability was checked by being asked to identify a letter presented to it. A member of the research team then answered the question to simulate this proposed behavior of KiliRo.

For the third task, a member of the research team taught the number systems to KiliRo. After a few minutes of training, KiliRo was checked for its ability to learn numbers. This was done by being asked to identify a printed number put in front of it.

The fourth task involved identifying individuals. In this session, a member of the research team showed a real person to KiliRo and taught relationships such as mother or father of a child. For instance, a parent of the participating child was shown to KiliRo and taught that he/she is the child’s father/mother. After few minutes of training, KiliRo was checked for its ability on how it has learnt by showing the same person and asking who he/she was.

2.3. Adapted-Model Rival Method (AMRM)

In this study, we put forward and validate our Adapted Model-Rival Method (AMRM) using a parrot-inspired robot, KiliRo, to improve learning and social interaction among children with autism spectrum disorder. In this method, the researcher teaches KiliRo robot in the presence of children, who are expected to be curious by the ability of KiliRo and start to compete with the robot to identify the objects or persons. The study is designed in a playful environment where the robot and the participating children compete to identify the objects shown by the researcher.

3. Results

During the beginning of the study, the participants were not attracted to the robot and continued doing their own work. When the robot started moving, four out of nine participants immediately responded by looking at the robot. The interest of children increased when the robot started to communicate with the researcher. Notably, Participant 6 grabbed the robot and started talking to it. The fifth day of the study was much more interactive compared to the first day. Two parents stated that they came early to the center as their children were very happy during the interaction with the robot. This indicates the positive attraction towards the parrot-inspired robot, KiliRo.

3.1. Qualitative Analysis

During the interview sessions with parents, most of the parents indicated that they were trying all possible therapies for their children irrespective of any proven results. It was understood that parents were willing to accept new therapies for their children and were happy even if it can provide little benefit to the child’s learning and social interaction. One parent mentioned that they were not attending any social functions due to her child being identified as having autism spectrum disorder. It is clearly noted that the parents were exploring new methods for treating and teaching children with autism spectrum disorder. Most of the parents accepted the AMRM approach and believed that KiliRo can help improve learning and social interaction of their children. During the interview sessions, a few parents insisted not to deploy the flying ability in KiliRo. They argued that children would then be too focused on the robot’s locomotion rather than its learning ability. They also insisted that KiliRo’s speaking ability will attract children to interact and play with it, which is very important for improving social interaction abilities of participating children.

During the 15-min interview with the pediatrician, the scope of the research was explained in detail. He expressed his agreement towards improvement in learning through AMRM using the parrot-inspired robot. His views were very positive towards using parrot-like robots for improving learning and social interaction abilities of children with autism spectrum disorder. Nevertheless, he was concerned with the safety of participating children while interacting with the robot.

The child psychologist raised a few concerns in ensuring the safety and psychological attachment of children to the robot. She insisted that every child in autism spectrum disorder is different. Children with severe autism spectrum disorder behavior may throw the robot at others or use it to self-harm. The child psychologist argued that a robot with less weight would be useful, as it would enable children to carry it around and pet it.

3.2. Quantitative Analysis

3.2.1. Emotion Recognition

For emotion recognition, 580 pictures were taken throughout the study, and all of them were included in our analysis. From these 580 pictures, 2360 individual facial emotion values were obtained for evaluating anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. Figure 3 illustrates the emotional state values percentage of children from day 1 through day 5 of the study. Corresponding values with a maximum of 1 were obtained through Microsoft emotion recognition software. Overall, neutral, happiness, sadness, and surprise are the four emotions exhibited highly by the participants. Anger, contempt, disgust, and fear are found in less than 5% of the cases. Neutral emotion is found to be common in participating children during the study with the highest value on day 2 (83%) and the lowest on day 4 (59.5%). Happiness is found to rank next to neutral emotion. Except for day 1, happiness seems to be increasing day by day in children. On the other hand, sadness is found with minimum of 4 percent and maximum of 10 percent. The participants exhibited a gradual increase in surprise emotion. Through this bar chart illustration, it is seen that the happiness of children increased gradually throughout the study encouraging the development of parrot-inspired robot KiliRo.

3.2.2. Closed-Format Questionnaire

Table 1. Parents’ responses to closed-format questionnaire.

Participant	Q1	Q2	Q3	Q5	Q6	Q7	Q8	Q9	Q10	Q11	Q12	Q14
Parent 1	4	4	3	3	4	2	3	3	4	2	4	4
Parent 2	4	4	4	3	5				4			4
Parent 3	4	5	4	4	5	4	4	3	4	5	3	3
Parent 4	5	4	5	4	4	4	5	5	5	5	5	5
Parent 5	4	4	4	4	3	4	4	4	4	4	4	5
Parent 6	4	4	4	5	5	5	4	5	5	5	5	5
Parent 7	4	3	4	3	4	3	4		3	3	3	3
Parent 8	3	3	3	4
Parent 9	4	4	4	4	5	4			4
Mean	4.00	3.89	3.89	3.78	4.38	3.71	4.00	4.00	4.13	4.00	4.00	4.14
n	9	9	9	9	8	7	6	5	8	6	6	7

illustrates the responses from nine parents for the 12 questions raised on a 5-point scale. Questions 1 to 14 except 4 and 13 are marked between 1 through 5 (1 is strongly disagree, 2 is disagree, 3 is neither disagree or agree, 4 is agree and 5 is strongly agree). Cells with no values indicate that the respondents did not provide an answer. For the questions 4 and 13, respondents mark multiple answers. The majority of respondents indicated that the proposed AMRM would help improve learning and social interaction abilities of children with autism spectrum disorder. The results also point to a high degree of acceptance for the parrot-like morphology. Most of the respondents indicated that they would recommend KiliRo and the AMRM for teaching children with autism spectrum disorder.

Figure 4 indicates the expected features on KiliRo from the parents’ point of view. Seven parents insisted that talking ability is recommended for the KiliRo robot and most of the parents did not recommend the flying feature in the robot. It is also noted that most of the parents recommended wing flapping.

4. Conclusion

This study proposed and evaluated the design and development of a parrot-inspired robot, KiliRo, together with a novel teaching method, the Adapted Model-Rival Method (AMRM), to improve learning and social interaction abilities of children with autism spectrum disorder. Qualitative and quantitative analysis presented in this study points to potential benefits of developing a parrot-inspired robot for teaching children with autism spectrum disorder. Informal and semi-formal interviews with parents of participating children, the pediatrician and the child psychologist indicated the acceptance of parrot-like morphology and the Adapted Model-Rival Method (AMRM). These findings have been analyzed quantitatively using closed-format questionnaire and emotion recognition system. Through facial emotion recognition, the study indicated quantitatively that the children with autism spectrum disorder did not appear afraid of parrot-like morphology and seemed happy to interact with the robot. This study aligns with the previous literatures [28,29,30] that show strong evidence towards children with autism spectrum disorder and their robot acceptance. Another quantitative analysis, closed-format questionnaire also provided positive results for AMRM and the parrot-like morphology of the robot.

Through this pilot study, we conclude that the development of parrot-inspired robot and the novel teaching method, AMRM could create new avenue for teaching children with autism spectrum disorder and improve their learning and social interaction abilities. KiliRo, currently under further development, will incorporate the findings of this study in its morphology, features, and materials to be used. Nevertheless, there are a few limitations identified in the study. Firstly, the informal interview with parents was conducted as a group interview, and there are possibilities that the opinion of one respondent could have influenced the other. Secondly, the closed-format questionnaire was designed specifically for this study. Hence, reliability of the questionnaire has limitations. Thirdly, the age of the parents, who were interviewed and responded to the questionnaire were not recorded in this pilot study. We consider this as an important limitation of our work as the opinion could widely differ among various age groups. Finally, the study was conducted with a small sample size and for a short duration only. The emotion recognition data were unambiguously linked to any specific participants so that comparisons across days were only possible with mean values. While efforts were made to ensure that data were collected from all participants, the possibility remains that some of the differences observed were brought about by disproportionate changes for some of the participants only. Follow-up studies using a larger sample size are necessary to confirm the results of this pilot study. Future work includes, developing the fully autonomous robot, KiliRo, with the expected features described here and evaluating the robot’s performance through short-term and long-term user studies. Another possibility for future work includes using the AMRM and KiliRo for normally-developing children for teaching and to compare with traditional teaching methods. The third possibility for future work includes using the AMRM and KiliRo for elderly care and physical rehabilitation.