1. Introduction
The future of older people in developed countries such as Portugal is connected with technological development, since there are many who now have and use smart mobile devices (smartphones). This trend will grow exponentially. As such, the solutions to propose for the welfare of older adults will also have to go through this path.
Robots in the homes of humans are already a reality—for example, the robot vacuum cleaner. The robots of assistance and supervision of older people at home and host institutions, residences for older adults, are also having a growing amount of development.
In this follow-up, the capabilities of these devices should be explored, not only to monitor the activity and safety of older adults, but also to help improve the cognitive performance of the older people. Leisure activities such as games aim to improve socialization among the older people—for example, in intra and/or inter-institutional games using robots.
Within the scope of the EU project EuroAGE, specialists in humanities and social sciences identified several needs that must be fulfilled to promote Active Aging in the EUROACE Region, within the border of Portugal and Spain. Those are: involvement of older adults in their activities, their capacity for initiative and to establish social interactions, indicators of well-being, and a proactive attitude. Such items are tackled, in this paper, by cognitive and emotional stimulation using robotic games. The games development was done in close contact with older adults of a nursing home, where pre-tests were performed to allow constant feedback from the end-user during the development stage.
The paper presents and discusses a framework to promote older adults’ cognitive and emotional stimulation via Robotic Cognitive Games. Recent works proposed a methodology to evaluate the interactions between older adults and the robot
Pereira et al. (
2018) in terms of initiative and involvement of older adults with the proposed games and the comparison with the use of the robot, to verify if the introduction of the robot enhances social interactions,
Joshi and Šabanović (
2017). However, no systematic evaluation was performed in both the cognitive and emotional fields.
The first contribution in this paper are the robotic cognitive games, based on objects and robots in real environments. With this solution, the robot will cease to be a mere element of monitoring the status of the older people, to perform vacuum cleaning, etc, and will interact with older adults in leisure activities, which stimulate the cognitive process. As such, the proposed framework is suited to evaluate the tools proposed in
Pereira et al. (
2018).
Three cognitive robotic games were developed to stimulate the cognitive behaviors of older adults and to provide active aging in the cognitive component. The interaction between older adults and the robot is performed by a user-friendly interface that allows the control and manipulation of the robot by older adults
Honig and Oron-Gilad (
2018).
From the technological viewpoint, the three games are based on object shapes and colors that the older adult must identify and place correctly in a (3 × 3) squared arena, via the tele-operation of a mobile robot manipulator. In the first game, a given layout of three objects from different colors (red, green, yellow) is shown to the older adult on a screen that she/he must replicate on a (3 × 3) squared arena. The second game relates to different object shapes (circle, square, triangle). The third game refers to a combination of different shapes and colors of the three objects that are shown to the older adult that he/she must replicate in the arena.
The second contribution was to perform a systematic assessment of the cognitive and emotional performance of older adults, using state-of-the-art methods, while playing with the three games using robotic technology (e.g., time spent and scores obtained).
The final goal of the research work was to analyze the gains obtained with the participation in the robotic cognitive games research work, regarding the cognitive functions and emotional state. The experiments with older adults were conducted at “removed for blind review”. The contact with older adults was in May and June of 2019.
The paper presents in
Section 2 the Methods for Cognitive and Emotional older adult Evaluation. In
Section 3, the Robotic Cognitive Games are presented.
Section 4 presents the older Adults behavior characterization methodology and the robotic cognitive games. The paper ends with the presentation and discussion of the results in
Section 5, along with the conclusions and future work directions in
Section 6.
2. Methods for Cognitive and Emotional Older Adult Evaluation
In this section, the methods for cognitive and emotional evaluation are presented. Several methods were analyzed regarding the applicability for the current research work. This subsection is divided into cognitive and emotional fields. Since the methods’ applicability also relies on the target population, an important factor to take into account is previous studies application to the Portuguese geriatric population, and the existence of norms and/or best practices.
2.1. Candidate Methods
2.1.1. Cognitive Evaluation
The Mini-mental state (MMSE)
Folstein et al. (
1975) is a classic method, perfectly suited for paper/pencil tests, easy and quick to apply (5 to 10 min). It is one of the most commonly used cognitive screening tests to assess the existence, or the non existence, of cognitive impairment
Dong et al. (
2013). The test allows a maximum score of 30 points, with the highest scores indicating better performances. It is organized into six cognitive domains: temporal and spatial orientation; retention; attention and calculation; evocation; and language and constructive capacity. According to the normative educational data for the Portuguese population, we should consider the following cutoffs: 15 points for Illiterates, 22 points for 1st Year to 11th Year; and 27 points for more than the 11th year).
The Montreal Cognitive Assessment (MoCA)
Nasreddine et al. (
2005) and its Portuguese application version
Freitas et al. (
2011) is a brief cognitive assessment tool, where higher scores show a better cognitive performance. Developed to detect slight cognitive deficits, it evaluates six cognitive domains: executive function, visual-spatial ability, short-term memory, attention, concentration and working memory, language, and temporal and spatial orientation. This instrument of evaluation was not used because it is more complex than the MMSE
Folstein et al. (
1975) since it penalizes the illiterate subjects in the tasks of executive functions, attention and language, and because there is no normative data for the Portuguese population specific for illiterate people
Freitas et al. (
2011).
The Cognitive Examination of Addenbrooke—Revised
Mioshi et al. (
2006) along with its Portuguese application version
Firmino et al. (
2008) is a brief cognitive screening tool that evaluates five neurocognitive domains: Attention and Orientation, Memory, Fluency, Language and Visual-Spatial. The maximum score is 100 points, where the highest scores indicate better cognitive functioning. In addition to the overall result, the instrument makes it possible to calculate the result obtained in each of the five domains evaluated, as well as the Mini Mental State Examination (MMSE) score. This evaluation tool was not used as well because it took a larger time to apply to older adults, and, like the previous one, it penalizes the illiterate in some tasks.
From the three candidate methods presented in this sub-section, the MMSE was the chosen one. The MMSE is the quickest, for the same evaluation features, presented in the Results section: temporal orientation, spatial orientation, retention, calculus and attention, evocation, language, and constructive capacity. Other methods like MOCA or Addenbrooke’s also provide the same results, although the application time is larger. This suits the purpose of our research work perfectly because we avoid submitting older adults to complex and long tests, summing to the playing time of the robotic cognitive games. As such, older adults are not demotivated after the application of large and complex tests. Another fact, and the most important one that hampers the application of MOCA or Addenbrooke’s methods, is that reference values for the non-illiterate older adults can only be found at the present time in the Portuguese literature
Freitas et al. (
2011). At “Santa Casa da Misericórdia de Vila Velha de Ródão”, do exist potential candidates to the study group that are illiterate. The non-illiterate older adults would be penalized in activities that the knowledge of the alphabet and numbers is mandatory. Examples are: evoke words beginning with P, subtraction from 7 to 7 beginning at 100, or others that need to sort letters and numbers, write a sentence, spell a word, read, and so on.
2.1.2. Emotional Evaluation
For the emotional evaluation, the set of candidates is reduced to the Geriatric Depression Scale, with 30 questions, GDS-30
Yesavage et al. (
1993). In fact, the Portuguese application version
Simões et al. (
2010) showed very good results along with the clinical practice of the authors of this work, namely in the ongoing project (EuroAGE) (
www.euroage.eu) for the cognitive and social enhancement of the resident population of Vila Velha de Ródão municipality.
The GDS-30
Yesavage et al. (
1993) and its Portuguese application version
Simões et al. (
2010) is a scale of thirty questions about affective and behavioral symptoms of depression, manifested in the last week. Specifically developed for the geriatric population, it is possible to distinguish between the absence of depressive symptomatology (0 to 10 points), the presence of mild (11 to 20 points), and severe (21 to 30 points) depressive symptomatology.
2.2. The Evaluation Methodology
Prior to playing the robotic cognitive games, older adults of ‘Santa Casa da Misericórdia de Vila Velha de Ródão’, were present in several sessions where the authors demonstrated the robotic games. From this step, a group of almost thirty older adults showed interest in participating in the study and were submitted to cognitive and emotional evaluations. From this general group, a sub-set of five older adults was randomly chosen to participate in the first study group.
The methods chosen to evaluate the cognitive and emotional status were, as depicted in the previous sub-section, the MMSE, for the cognitive test, and GDS-30, for the emotional test. The tests were performed one following the other, for the same person, at the institution facilities by one of the authors.
After the first tests, several sessions took place where older adults played the robotic cognitive games. These sessions were conducted with the first study group, for evaluation purposes of their cognitive and social progression. The evaluation protocol is depicted in the following
Section 4.2.
3. The Robotic Cognitive Games
In this section, the three robotic games developed are presented, based on classical games, e.g., previously played without a robot intervention. In the first sub-section, the games are presented. It follows the presentation of the developed robot, specific for the game scenario. In the third sub-section, the developed capabilities for the robot that are used to play each game are presented. Moreover, in this section, the perception capabilities of the system that are used to verify older adults’ performance are presented. It follows the presentation of a Smartphone App that the older adult uses to control the robot.
3.1. The Robotic Games
Three games were developed to stimulate the cognitive and emotional behaviors of older adults and to enhance active ageing in both components. All the games are based on classical games, already played in some institutions of Portugal, not the case of “Santa Casa da Misericórdia de Vila Velha de Ródão”. The classical games use objects of several shapes and colors that the older adult must identify and place correctly in a 3 × 3 squared arena. In the first game, a given layout of a given object (circle) from different colors (red, green, yellow) is shown to the older adult that she/he must replicate on a (3 × 3) squared arena. The second game relates to different object shapes (circle, square, triangle) that are color agnostic. The third game refers to a combination of different shapes and colors of the three objects that are shown to the older adult that he/she must replicate in the arena.
From these games, the development of the technology was done in two steps. The first one was the automation of the game with the use of an application. The application shows to older adults the initial objects positions in the arena, and at the end computes the final score of the older adult performance. It also shows in the computer monitor the object positions in the arena, side by side with the proposed initial positions. This first version of the game was tested in previous works from the authors at “Santa Casa da Misericórdia de Vila Velha de Ródão”. The second step was to introduce the robot in the game to stimulate older adults’ visual motor coordination, interested in doing a new activity, with a new tool that only grandchildren normally use.
In the first game (Colors game), the system chooses randomly three positions in the (3 × 3) squared arena. The shape of the object is a circle. The older adult is invited to tele-operate the robot to place the objects in the correct position of the (3 × 3) white arena that is on the floor, after a start button is pressed. When the task is done, a stop button is pressed, and the system evaluates the older adult performance. When all three of the objects are correctly performed, the older adult gets 100 points. For each error, the score is diminished by 20 points. If three errors occur, the older adult gets 40 points. When the older adult takes more than 10 min to finish the game, five points are diminished in the score, and, after more ten minutes, another five points are diminished, and so on.
The remaining two games are similar to the three colors game, in the flow of the play and also in the scoring. The main difference is the type of objects at play.
In game two, the three shapes game, the older adult is invited to tele-operate the robot to place the three different object shapes (circle, square, triangle), shown in the Graphic User Interface (GUI), in the (3 × 3) floor white arena. This game works other aspects of the cognitive processes of older adults.
In game 3, the colors and shapes game, the older adult is invited to tele-operate the robot to place the three objects (chosen randomly from the nine variations of the three previously defined colors and shapes), shown in the GUI, in the 3 × 3 floor white arena. This game is more complex and challenging to older adults.
The physical setup for the robotic games is depicted in
Figure 1. In this figure, an RDG-D camera is present (Kinect (Microsoft, Redmond, WA, U.S.A.) in the middle of the image) to obtain a total view of the white arena in the floor. The colored objects are also depicted, with different shapes (above the arena and close to the robot, in the bottom of the figure). On the top (white projection in the wall), the screen of the game is depicted, where the pre-defined positions of the objects are shown to the older adult for him to place them in the arena (left part). At the end of the game, the older adult can verify his score on the screen, on the right part of the screen. This verification procedure is performed automatically by the perception part of the system, from the image captured by the Kinect camera, using image processing techniques to segment the colors and shape of each object in the (3 × 3) arena. In the figure, game two is presented, where the colors of the object are not important, only its shape.
3.2. The Developed Robot
The robotic cognitive games are based on a tele-operated robot, depicted in
Figure 2. The robot, to be tele-operated by older adults, is an upgrade of a previous version presented in previous works (
Gonçalves et al. 2019). The upgrade was done to improve the robot motion commands, and, more importantly, to have a very nice look, as depicted in
Figure 2.
The control command of the robot can be sent using Wi-Fi or Bluetooth, which allows communication even when there is no Wi-Fi in the nursing home (
Gonçalves et al. 2014). The robot can move to the positions in the 3 × 3 arena. The robot must also be able to grasp the game objects.
As depicted in
Figure 3, the robot has an arm, i.e., a one degree of freedom link, to move the magnetic gripper, to the lower and upper positions. The magnetic gripper task is to grasp the objects. Those are made of cork and have a colored metallic plate to allow the magnetic grasp.
3.3. The Robotic System Capabilities
Each robot deployed in real world applications must have a proper set of capabilities to be able to perform the tasks to which he is assigned. In this research work, the main task is to play the robotic cognitive game with older adults. The robotic system presented in the paper has the following capabilities:
tele-operation: is implemented in the robot using a Raspberry PI3 board, as depicted in
Figure 4. This capability ensures the management of Bluetooth or Wi-Fi to command the robot, and is the server of the the clients that can command the robot via Wi-Fi, i.e., a tablet or a remote device in the cloud.
motion: the robot can move in front, backwards, rotate left and right, along with the arm motion up, down, and grasp. These commands are implemented in the ‘low level control’ in an Arduino board on the robot, as depicted in
Figure 4.
arena perception: the arena perception is implemented in an Ubuntu PC that also runs the game engine, to obtain the object’s position after the game ends.
game engine: this capability is described in the following sub-section, and runs alongside the arena perception.
grasp: the robot can grasp using the magnetic gripper described in last sub-section.
The Game Engine
The game engine was implemented under the
ROS (
2019) environment using the state machine framework
SMACH (
2019). Although this framework is a task-level architecture for rapidly creating complex robot behaviors, it was used in this work because of the authors’ experience, and, more importantly, because the state-machine developed can evolve to a more complex one, when using a robot to play the games autonomously with the older adults.
Figure 5 presents the state machine of the game, where the six states of the machine and all the transitions are depicted. The states are: Home (start); Positions (to define the starting positions of the objects for older adults to play); Play (the state where older adults are playing); Image-Processing (where the image of the arena is captured and the current positions of the objects in the arena are calculated); Score (where the score of each play is calculated); Reset (to reset all the variables and prepare to start a new game).
The states and transitions depicted allow a proper flow of information, from the start of the game, where the pre-set positions of the three objects are randomly defined. When the older adult is ready, a start button is pressed to begin the timing until the older adult, while commanding the robot, finishes the task and the stop button is pressed. After this process is done, the image of the arena is captured and the positions of each placed object are obtained and matched to the pre-set positions. Using this information, the score of older adults play is calculated and shown to all the players. More players can have their turn next, and, at the end of all plays, the system will show the winner of that batch of players. The winner will be the player with more accuracy. The time each player takes to complete the move is used to break the players with equal accuracy.
Another view of the system is presented by showing the Robot Operating System (ROS) nodes and topics that are running and sending information to each other, as depicted in
Figure 6. It can be clearly seen the source of information from the Kinect camera sensor (
node) and object detection (
node), the state machine (
node), and the generated data to be shown in the graphical user interface (
node). The data messages that are sent within the nodes are depicted by the rectangles, as depicted with the directional arrows depicted in.
This GUI, as depicted in
Figure 7, was developed under ROS, by using the QT
RQT (
2019) based framework for GUI development for ROS. On the left side, the desired positions of the objects to be placed in the 3 × 3 arena are depicted. On the right side, the positions of the objects are depicted, obtained from the captured image when the older adult finishes his play. In the middle are the start and stop buttons, the scoring information, along with a numerical help feature for older adults who get confused during the play (this shows, in the GUI, a number at each of the squares, which matches the number in the floor arena). In the bottom part of the GUI is the calibration of the system, possible choices for the configuration of the system (position of the camera relative to the arena), and the choice of the type of game. This GUI is used for the three games, since the main difference is the type of objects that are used in each game.
3.4. The Developed App to Control the Robot
To control the robot, an Android based application was developed after several iterations from older adults’ institutions with which the authors collaborated. From the several options that could be implemented, an Android based with commands on the touch screen was chosen, to the detriment of a joystick based command, as depicted in
Figure 8. The tablet based with the move commands (front, backwards, rotate left, rotate right) on the leftmost and rightmost sides of the screen was chosen after older adults’ and nursing homes staff recommendations, mainly to keep both hands occupied, as depicted in
Figure 9. In the middle of the screen, the buttons to command the one degree of freedom arm is depicted, and the magnetic gripper to grab the objects. The four middle buttons are to command the arm to the lower and upper positions, right side, and to grasp (
’AGARRAR’ button) or leave (
’LARGAR’ button) the object.
Controlling the robot with the tablet is a complex challenge for older adults because the robot command is based on eight buttons on a tablet screen. For that, they must train the fine motor skills in the fingers. Moreover, the hand–eye coordination must also be trained along with the depth perception. Although difficult, older adults loved to play with the robot and have performed quite well with the tablet.
A last note that needs to be mentioned is that the authors dropped written commands, and used mainly figures that better describe the action commands, due to older adults’ illiteracy.
4. Older Adults Behavior Characterization Methodology and the Robotic Cognitive Games
In this section, the methodology used to characterize older adults’ behavior prior to, during the games, and after several sessions of game playing is described.
Within the scope of the research work, the participants selected for the experimental group were subjected to a brief psychological evaluation, performed prior to the games session.
Subsequently, four sessions, with the use of robotic cognitive games, were performed. To evaluate the gains from participation in these sessions, the participants were psychologically re-evaluated, with the same previous methods.
4.1. Participant Inclusion Criteria
For selecting participants for the research work and the experiments performed in this paper, the following criteria were applied:
People living in the municipality of “Vila Velha de Ródão”,
People who attend one of the services of “Santa Casa da Misericórdia de Vila Velha de Ródão”: ‘residential structure’ or ‘day center’,
People interested in participating in the research work,
Persons aged 65 or over,
People without cognitive decline,
People with or without depressive symptomatology,
Illiterate or schooled people.
Using this criteria, a small group was chosen for a first evaluation of the robotic cognitive games setup, presented in this paper. Five older adults were chosen that filled all the criteria, and the most ranked in the criteria ‘People interested in participating in the research work’.
4.2. Evaluation Protocol
The evaluation protocol, was done in three time periods, i.e., prior to, during and after the robotic cognitive game play.
The prior psychological assessment protocol included the:
After the initial psychological assessment, several sessions where older adults played the robotic cognitive games took place within a period of eleven interpolated days. The sessions were spaced apart by at least two days. Moreover, older adults could not play the same game in the same day. These last two conditions were added to ensure that older adults were not faced with the same situation, the same robotic cognitive game, in short periods of time. As such, the game procedure was the following: each older adult played each game four times. Since there are three games, each older adult played twelve games in the two month trials (at most two sessions per week, where all the older persons were in the same room playing). In total, sixty games were played by the older people.
The final psychological assessment protocol included the:
Informed consent is one of the means for protecting a patient’s right to autonomy
Fouka and Mantzorou (
2011). In the group of older adults in this work, all of them do not have diminished autonomy, which can hamper the informed consent knowledge. Instituto Politécnico de Castelo Branco do have an ethics committee that suits the needs of the current research work. All the experiments performed in the nursing homes were approved by the ethics committee. As such, an informed consent document was prepared by the research team, read to older adults, and signed by them. In conclusion, all older adults that are currently in the research program gave his/her consent to the work developed with the three presented games.
After the tests previously described, conducted in the facilities of “Santa Casa da Misericórdia de Vila Velha de Ródão”, the statistical analysis evaluation performed is depicted in the next section, along with the results discussion.
5. Presentation and Discussion of Results
5.1. Group Characterization
The experimental group consisted of five older adults with social response from “Santa Casa da Misericórdia de Vila Velha de Ródão”, presented in
Table 1. The social response can be in two ways, the first one when the older adult is fully institutionalized, i.e., sleeps in the nursing home, and the second one is when the older adult is institutionalized for daily activities, in the ‘day center’. For the group, in this research work, two are institutionalized and three are attending the social ‘day center’ response. They are mostly female and have the first cycle of schooling, basic studies in the Portuguese official curricula of the Ministry of Education. The participants are aged between 70 and 95 (mean age = 82.20 years).
5.2. Cognitive and Emotional Results
In this sub-section, the results of the MMSE and GDS-30 methods are presented, respectively, for cognitive and emotional tests, before and after the robotic games’ sessions.
Table 2 presents the results of the MMSE method for all older adults separately. The total results and the results in each sub-item are presented, to better assess the features that have changed between the clinical evaluations. In general, it is depicted that minor changes took place in the majority of the items and sub-items evaluated for each older adult.
Table 3 presents the emotional results, before and after the robotic games sessions, for all older adults. It is clear that, for the large majority of older adults, the results were quite impressive, showing that the depressive symptomatology diminished.
5.3. Assessment of the Older Adults Progression
In general, and considering only the total results obtained in the MMSE and the GDS-30, of the five people who participated in the first study group of the research work, four of them had an increase in cognitive functioning and a decrease in depressive symptomatology, whereas, in one person, the level of cognitive functioning and the number of sensory depressive symptoms remained stable between the two moments of assessment. This fact is presented in the tables presented in this section and discussion within this sub-section.
In the present work, the statistical treatment of the data was carried out through the Statistical Package for the Social Sciences (SPSS—version 20) (IBM, Armonk, NY, U.S.A.) The characterization of the sample was performed through descriptive statistics (frequencies, averages, standard deviations, minimum, maximum). To test for the existence of a statistically significant difference between the mean results of the same group of subjects on two different occasions, we used the Wilcoxon nonparametric test. Values of less than 0.05 were considered as indicators of statistical significance.
Table 4 presents the statistical results for the cognitive decline and depressive symptomatology. For all older adults, the MMSE test showed an average,
and standard deviation of
, for the pre-test. For the 2nd evaluation, pos-test,
,
, with
z-score = −1.841 and the
p-value = 0.066, suggesting a statistically non-significant increase in the MMSE total values, between the two assessment moments.
In the GDS-30 test, there were also no statistically significant differences between the first evaluation,
,
and the second evaluation
,
, with
z-score = −1.841 and the
p-value = 0.066. However, we verified that the subjects presented lower results in the second evaluation, and that none of the participants presented severe depressive symptomatology. These are indeed strongly encouraging results of the proposed methodology that uses robotic cognitive games to enhance the cognitive and emotional status of institutionalized older adults. This fact is clearly presented in
Table 5.
6. Conclusions and Future Work
The paper proposed a system that implements cognitive robotic games. The robotic system was designed to interact with older adults within nursing homes. The robotic system was developed in the framework of the EU project EuroAGE, aiming to provide robotic technologies to older adults. The system was developed using the ROS framework, for the robot, the tele-operation app, and game engine, to enable the interaction between the game, the robot, and a vision system, to obtain scores for each play.
Proper methods for cognitive and emotional evaluation were used, regarding the applicability to the current research work and aligned to the Portuguese population. Its results obtained from the observation of older adults playing the robots, i.e., in-loco qualitative and quantitative observations. These were discussed in
Section 5, and are indeed encouraging results. From the increase, although statistically non-significant, of the MMSE and GDS-30 total values, respectively cognitive and emotional, between the two assessment moments, we can conclude that the proposed methodology that uses robotic cognitive games is proper to enhance the cognitive and emotional status of institutionalized older adults. Moreover, none of the participants presented severe depressive symptomatology after the robotic game sessions’ evaluations. The older adults that were engaged in the research showed an increase of involvement in their activities, the capacity for initiative, and ability to establish social interactions.
Future works will be further developed by increasing the degree of autonomy of the robot, i.e., in order to the robot to play the robot by itself. Further evaluations will be performed, to increase the sample size of participants, chiefly to other older persons within the same institution, and other institutions. With such increase of sample data, a new analysis will be performed to check the effect of user diversity on the experiment findings, using, for example, a decision tree or similar approach.