A Virtual Reality-Based Cognitive Telerehabilitation System for Use in the COVID-19 Pandemic

Abstract

The COVID-19 pandemic has changed people’s lives and the way in which certain services are provided. Such changes are not uncommon in healthcare services and they will have to adapt to the new situation by increasing the number of services remotely offered. Limited mobility has resulted in interruption of treatments that traditionally have been administered through face-to-face modalities, especially those related to cognitive impairments. In this telerehabilitation approach, both the patient and the specialist physician enter a virtual reality (VR) environment where they can interact in real time through avatars. A spaced retrieval (SR) task is implemented in the system to analyze cognitive performance. An experimental group (n = 20) performed the SR task in telerehabilitation mode, whereas a control group (n = 20) performed the SR task through a traditional face-to-face mode. The obtained results showed that it is possible to carry out cognitive rehabilitation processes through a telerehabilitation modality in conjunction with VR. The cost-effectiveness of the system will also contribute to making healthcare systems more efficient, overcoming both geographical and temporal limitations.

1. Introduction

Comprehensive healthcare involves using the means and resources necessary to maintain people’s health [1]. Technological progress has made it possible to improve these services, with examples including robotic surgery, liquid biopsy, portable sensors and telemedicine [2,3,4]. According to the World Health Organization (WHO), telemedicine is the provision of health services through the use of information and communication technologies (ICT) for the exchange of information between the health professional and the patient. The data obtained by using these interactive systems allow the prevention, diagnosis and treatment of diseases [5]. Technological development applied to medicine is linked to the widespread use of the Internet in public spaces, and most households admit that physical interaction is no longer mandatory [6]. The use of telemedicine should not replace the provision of conventional health services, but rather seeks to integrate them, making them more efficient and accessible to the user. All this should be done without neglecting the principles of confidentiality and professional ethics that should accompany any health practice [7,8]. However, as mentioned by Lawson et al. [9], innovations in the health sector must follow a rigorous testing process to validate their constant use in human beings. Several authors have made a comparison of the methodologies applied in telemedicine in contrast with the results obtained in conventional therapies, achieving good performance in the former [10,11].

Telemedicine offers remote monitoring, storage and delivery of clinical data and real-time interaction between physicians and patients [12]. This interaction is clearly seen in telerehabilitation, which provides rehabilitation services remotely in the patient’s home or other chosen location [13]. Various types of technologies such as videos, computer programs or even live sessions can be used to guide the patient through the rehabilitation process [14]. In the beginning, the focus was on physical rehabilitation, achieving good results, which awakened interest in testing the effectiveness of rehabilitation on cognitive issues [15]. Namely, delirium (affecting consciousness, concentration and attention), amnesia (preceded by attention problems, disorientation and confusion) and dementia (where Alzheimer’s is the most common disorder) [16].

The treatment of cognitive impairment varies according to the type of disease, the degree of severity and other factors specific to the person [17]. Whenever there is a progressive loss of memories, procedural memory rehabilitation symbolizes the basis for a successful therapy. This type of long-term memory will deteriorate the least in cases of dementia and other similar diseases [18]. A widely used neuropsychological rehabilitation technique in dementia is the spaced retrieval (SR) method [19]. This is a technique that consists of learning new information, which is retrieved continuously by increasing the time intervals between the evocations of such information.

This simple technique contributes to the rehabilitation processes of patients who present Parkinson’s disease, Alzheimer’s disease or brain injuries, promoting independence and close social interaction [20]. In fact, this technique has shown good results in Alzheimer’s patients in different areas of memory, for example promoting retention of simple associations [21,22] or improving prospective memory [23,24]. It can be applied by any person (who does not have cognitive impairment) and it is based on the achievement of objectives, such as remembering family names, schedules and places. This is done with the premise that remembering information that is familiar to the patient involves less effort than when using new knowledge [25,26]. It should also be emphasized that in advanced stages of cognitive impairment, the technique is less effective. This rehabilitation technique and others require assessment and follow-up by the medical specialist, who must follow a plan, either in person or remotely, by using technological means or not. This article is organized as follows. Section 2 is devoted to the related work in the fields of telerehabilitation and VR application within the scope of the COVID-19 pandemic. Section 3 deals with the materials and methods used throughout the study, while Section 4 presents the results after the intervention, involving control and experimental groups. Finally, Section 5 presents the discussion and main conclusions of the study.

2. Related Work

As time goes by, patients’ needs have become greater and more diverse. This is why telemedicine has been implemented in several health fields. The studies by Evans et al. [27] and Makhni et al. [28] introduced the use of robotics in the area of surgery (telerobotics), together with the identification of the advances and limitations. Another field of paramount importance deals with the training of health staff, of which the training of various skills and the practice of surgeries remotely are the most relevant aspects [29,30,31]. Telerehabilitation is also a topic of interest when dealing with children [32,33], young adults [34,35] and the elderly [36]. Especially the elderly, whose use is very limited because of physical discomfort and limitations in technology, should be highlighted. Telerehabilitation includes robotic assistance therapy [37], portable devices [38,39] and immersive systems [40], among others. The different techniques and devices used in the interventions have a high degree of acceptance and satisfaction among users, as shown in the review presented in [41].

The use of virtual reality (VR) in rehabilitation processes has been widely applied and the presence of this technology generates an added value in these processes [42]. VR has been used by physicians and researchers as a technological alternative that allows health procedures to be performed with new approaches and results in a safe environment. It also reduces the consumption of resources, avoiding unnecessary trips as an eco-friendly measure. In addition, it offers the monitoring and delivery of immediate results, which allow evaluation of the evolution of the disease from quantitative and qualitative points of view through the application of tests [43]. All of these features (and more) have encouraged the use of VR in the treatment of different cognitive disorders [44]. In fact, the review carried out by Moreno et al. [45], which compiled 22 studies, showed that cognitive rehabilitation with VR generates improvements in memory and visual attention, as well as lowering levels of stress and anxiety. It is used in disorders such as neurocognitive disorders, patients with mild cognitive impairment or Alzheimer’s disease. On the other hand, Park et al.’s work [46] presented 36 investigations that analyzed the use of VR in people with psychiatric disorders. It was identified that VR sessions are effective in treating anxiety, phobias and post-traumatic stress disorder (PTSD). Regarding the treatment of bipolar disorder, schizophrenia, depression, autism, and dementia, despite having positive results, further studies are required according to the authors.

VR has already been used for telerehabilitation of several diseases in recent years. Tieri et al. presented a review that highlighted the positive factors that an immersive environment provides to the patient during treatment [47]. On the other hand, Lindner et al. discussed the good results offered by VR in therapies to mitigate anxiety in public speaking as a support for health staff [48]. Likewise, the study by Isernia et al. [49] proposed telerehabilitation interventions for patients who had developed Alzheimer’s disease to strengthen memory. Other post-stroke VR-based interventions have proven their efficiency compared to conventional therapies [15,50].

The COVID-19 pandemic has resulted in the social isolation of individuals, and thus the development of mental disorders [6]. In the study by Imperatori et al. [51], a brief review highlighted how VR applications have enabled specialists to provide therapy to patients. Mantovani et al. [6] provided an overview of how VR has contributed to cognitive telerehabilitation processes and the intensification triggered by this global health crisis. On the other hand, in [52], a multi-user virtual reality (MUVR) system was developed, in which the therapist interacts on a remote basis with patients with eating disorders. In the study, it is possible to identify how the user’s dialogue with the psychotherapist’s avatar (virtual environment) can reduce tension and stress during activities, instead of using a camera (videoconference). Although the participants did not know the therapists in person before, the real-time communication and virtual representation allowed confidence to develop throughout the sessions.

Internet access is becoming increasingly widespread and this fact has led to innovations in healthcare, offering greater benefits to the patient. The COVID-19 pandemic has encouraged the use of remote technological tools that allow a doctor–patient relationship to be maintained as closely as possible despite the distance. This is why a VR application for telerehabilitation has been developed, allowing interaction without any travel and replacing video conferences with a virtual environment. By doing so, the users will be able to gain access to rehabilitation programs in a safe manner and from the comfort of their homes.

3. Materials and Methods

3.1. Problem Statement

Telerehabilitation allows remote interaction between the patient and the medical specialist in order to share information and perform activities that improve the patient’s condition through clinical therapy. Telecommunications and the Internet are vital in providing this service. Within the framework of the COVID-19 pandemic, it is important to reduce the contact between people as much as possible, creating room for the study of new proposals for telerehabilitation. Traditionally, telerehabilitation is carried out by using mobile communication technology or video call programs installed on personal computers. Although these tools allow for rapid interaction between users, there are limitations when sharing information and in data collection [53]. In neuropsychology, the specialist must present acoustic and visual information in the execution of cognitive activities, creating the need for customized applications that include appropriate tools for remote rehabilitation. Specifically, in memory rehabilitation, the information presented must be retained by the patient according to the therapy needs, requiring applications with automated processes and visual richness that favor the patient’s learning. In relation to the latter, VR offers flexible environments to execute neuropsychological rehabilitation tasks, which combined with multiplayer platforms allow users to interact online. The implementation of VR systems requires a display screen and input and output peripherals. Although there are several options available, as well as multiplatform video game engines, it is important to bear in mind the sustainability of the proposal. Thus, head-mounted display (HMD) devices are not appropriate because they are not common household devices, while mobile devices are limited by their screen dimensions. On the other hand, computers are still the most widely used large-screen devices at home, allowing for rapid installation of the proposed system. Figure 1 depicts the system proposed in this research, showing two opposite sides of the application; on one hand, the patient who must perform the cognitive exercises, while on the other hand, the therapist who must analyze the data generated in the cognitive therapy, with both users connected through computers. The application used provides different configurations for each user, so that the neuropsychologist is the administrator and has more privileges than the patient has as a user. This is because the master controls the information to be shown and decides the activities to be performed, although both users are in the same virtual environment. Network events and audio are transmitted using the cloud in a multiplayer platform that allows synchronized interaction.

The proposal offers a VR multi-user application developed in a game engine, providing users with interactive elements for the process of cognitive rehabilitation by using the technique known as spaced retrieval. Through this technique, the neuropsychologist presents the information to be memorized and immediately asks the patient to identify the memorized elements (initial assessment). After this, the patient is distracted for a certain time with another activity, then again asked to retrieve the memorized elements (final assessment). These activities are performed within a virtual office, where the patient moves at will using a first-person view to provide an immersive experience and the administrator moves from a third-person view, taking greater control of the virtual environment. The movements take place using 3D characters (avatars), with animations programmed for the actions performed by the user. The users are provided with voice communication to allow dialogue during the induction, rehabilitation and results processes. In addition, the patient’s scores and response times are recorded in a file for later analysis. Finally, two groups of participants without neurodegenerative diseases are used to evaluate this system. The control group uses the conventional rehabilitation technique (face-to-face therapy) and the experimental group used the VR-based telerehabilitation application.

3.2. System Components

The system was designed for easy implementation in a standard home, so that users do not require specialized equipment or computer accessories that are difficult to install. In this context, personal computers (portable or non-portable) with Windows operating systems are the most used in the field of telerehabilitation. These devices generally include large screens and input–output devices, which are useful for interacting with the system. Regarding the development software, there are two highly popular options, namely Unity (San Francisco, CA, USA) and Unreal (Cary, NC, USA), both offering easy-to-learn engines and large support communities for application development. Based on these criteria, the system components for cognitive rehabilitation were defined.

Figure 2 shows the components required for the implementation of the system, whereby the computer monitor shows the virtual interface to the user and the audio is emitted through external or built-in speakers. The input elements (mouse, keyboard and microphone) allow interaction with the system. The selected game engine was Unity (Unity Technology, San Francisco, CA, USA), which allows the development of an application with both flexibility and multiplatform features. In addition, the proposal required three-dimensional animated characters or avatars, for which Mixamo was chosen. Mixano is a web application that allows one to download prefabricated characters and avatar animations. As far as the multi-user platform is concerned, Unity works with Photon Cloud, a real-time interaction service that allows the synchronization of actions Photon Unity Networking (PUN) and communication by voice (Photon Voice).

Internet connection is an essential requirement for the execution of the system, because the multi-user platform constantly exchanges information over the network to synchronize virtual environments and communicate with users. In addition, the results of the process are recorded in an open format file with CSV extension and then visualized in a simple spreadsheet. In the implementation of this proposal, no further software add-ons are required. On the other hand, the requirements regarding the user’s knowledge include handling of screen devices and computer applications in general, and optionally some experience in VR or video games. Finally, although the patient experiences the virtual environment in first person, it is not necessary to restrict the system to users without motion sickness, because the immersion is minimal and the movements are gentle.

3.3. VR Application Design

There are several options to be implemented in the VR environment in the cognitive telerehabilitation process, but it is important to consider the simplicity of the tools for ease of use. The cognitive rehabilitation technique to be implemented was spaced retrieval [18], the strategy for which requires the presented information to be retained in the memory to observe the response to be observed immediately after (first assessment). After this, there is a sub-stage of error correction and reminder of the correct elements in the story, with the purpose of restarting the elements in the participant’s memory. Then, the patient has to do extra activities for a certain period, aimed at distracting the participant. The time taken to perform this activity varies according to the patient and the decision of the specialist. For example, in healthy patients, it is common to wait between 10 and 20 min. After this period, the patient must again remember the elements recovered in the initial assessment. Each correctly remembered element is considered a success and each element not remembered is an error; selecting an incorrect element also results in an error. The entire procedure is performed within the virtual environment, using interactive tools to present information, distractions and response options, and with the control of the neuropsychologist at each stage.

The multiplayer system was designed for a maximum of two users at a time, with two independent roles. The patient role allows the virtual environment to be visualized in first person and the therapist role presents the environment in third person. A different configuration was established depending on the user role, so that the patient has less control of the system. Figure 3 summarizes the characteristics of each type of user role within the virtual environment. The first-person experience of the patient allows for a minimum level of immersion, while the user has the view from the player’s position. In this way, the movements of the avatar in the VR environment are the main camera movements, although the patient can observe its avatar (patient role) for short periods of time when the movements are initiated. On the other hand, the neuropsychologist has a wider view of the virtual environment, visualizing the actions of both avatars and the elements that are incorporated into the scene in the rehabilitation process. Regarding the patient’s attributes, the access to the main scene is free and the user can control the avatar’s movements within the virtual environment, interact with the menus, visualize the images and use the games that are enabled. The privileges of the administrator are different, since this role has restricted access, which is secured by username and password. Once inside the main scene, they can freely control the movements of their avatar and can also activate and deactivate the menus, manage the visual information and the games within the scene experienced by the patient and of course visualize the results obtained by the patients.

The VR application consists of two user interaction scenes. The initial scene contains the start and configuration menus in two-dimensional format, where the player defines the access parameters before entering the virtual rehabilitation office. Then, the main scene contains the three-dimensional VR environment simulating a patient care office, where the telerehabilitation activities are performed and where users interact. Figure 4 shows the algorithm to be followed for the initial scene. It starts with a menu to select two available avatar choices: male or female. After this, the user selects their particular role (patient or neuropsychologist). If required, the user can return to the previous menu. Each role leads to a different access menu; in the patient role, the user only has to enter their name to access the next scene, whereas in the neuropsychologist role, the user must enter their name and password to gain access to the main scene. Whenever the input data are not correct, the user is prompted and cannot move on to the next scene until this is corrected. Once the user name is entered by the patient this is saved in a database.

Figure 5 depicts the main algorithm scene. Once the VR environment has started up, the operating configurations are made according to the stored input data in the previous scene, the 3D VR models are loaded by applying the parameters according to the user role and subsequently avatars are instantiated. If the user enters when the scene is empty, only the local avatar is instantiated, otherwise both (local and remote) are instantiated. Once within the main scene, users can interact with the VR environment by moving around and talking to each other in order to become familiar with the system’s features. When the rehabilitation process is about to start, both must be placed in the rest positions, sitting in their pre-established places, and then the neuropsychologist decides when to start. The information to be memorized is presented through audio and images and in video format played on a virtual screen within the environment. The video tells a short cartoon story with a duration of 2 min. The evaluation is performed through an option menu that contains 14 animals (7 of them appearing in the story), where the patient must select the right answers using the mouse. After this, the patient is distracted by a 3D game, where the user must solve mazes with different levels of difficulty; the number of mazes to be solved and their complexity depend on the therapist’s criteria, which is set up beforehand. At this point a second evaluation is performed (final assessment). Both response accuracy and reaction times are saved in a database in a compatible spreadsheet format and the neuropsychologist can open the file when required. At any time, the administrator (neuropsychologist role) can stop, restart or repeat any of these rehabilitation processes if they consider it necessary.

3.4. Development of the VR Environment

To develop the application, two computer software programs were used—Mixamo for the avatar animations and Unity to create the VR environment and execute the routines designed for the system. Figure 6 shows the general diagram developed in the Unity game engine, highlighting three main components: game objects, scripts and hardware. Unity’s scenes mainly contain the elements that make VR possible, namely game objects that allow the creation of lights, cameras, 3D models, avatars and other objects. In this application, the game objects are the user avatars, 3D models, menus and audio components. To complete the VR rehabilitation office, some other elements were created with the basic tools in Unity and certain 3D models imported from the Asset Store. On the other hand, the scripts are the software that control the actions within Unity’s scenes, which allow animations, activating menus, save files and many more functions to be shown. Within the system, the scripts are in charge of linking the hardware inputs with the avatars’ movements and the application tasks; they enable game objects, play audio and video and generate the results. In addition, the scripts contain the sentences of the multiplayer functions, updating the actions for both user roles and sending the voice audio. The hardware connected to the application is the computer monitor, the input peripherals (mouse and keyboard) and input and output audio devices (speaker and microphone).

The avatars and multiuser features were developed in stages, as shown in Figure 7. The first stage involved downloading the avatars, animations and textures for the 3D model assembly via the Mixamo website. The objects were dragged into the game engine for the next stage. In the second phase, the avatars and animations were incorporated into the Unity environment, where the avatar game object was inserted into the scene and an animator controller was built. The animation controller allows combinations of movements to be added in the form of blocks of states connected through transitions. The control inputs were configured using parameters and a tree of combinations that regulate the type and speed of the animation. In addition, a special transition was inserted to adopt the sitting position (used to perform rehabilitation), imitating a real therapy session. The final stage of development included the programming of the scripts (C#) and the configurations of the objects in scene. In order to interact realistically with the environment, a capsule collider and a rigid body were added to the avatar, with the rest of objects maintained with colliders. Two scripts (for the avatar) are in charge of locomotion and camera tracking. The locomotion is performed kinematically using the transform.Rotate and transform.Translate commands and is controlled with the keyboard arrow keys (Input.GetAxis). Camera tracking is performed by modifying the states of the main camera in the scene through the camera.position and camera.LookAt commands, and the parameters for this script depend on the selected user role (first-person and third-person options available). In addition, pressing the space bar on the keyboard activates the transition so that the user takes a seat in the preset furniture according to the type of user role. Then, regardless of the position of the avatar, they walk to the destination and take a seat.

The scripts also contain the multiplayer actions implemented using the Photon Unity Networking (PUN) library. To load the main scene in the local and remote stations, a room in the Photon network is created using the command PhotonNetwork.JoinOrCreateRoom. The avatar is then instantiated using PhotonNetwork.Instantiate, so that all connected users appear in the application. The multiplayer configuration requires all game objects to be updated by PhotonNetwork with the script component called Photon View, which assigns an in-game ID and allows the basic states in all running applications to be modified. Additionally, each avatar must contain the Photon Transform View component, which synchronizes the movements performed by the user, so that they are displayed in all open applications. The update of the states via the menus, avatar animations and the maze game is done through remote procedure calls (RPCs). On the other hand, the movements controlled by the user only trigger the corresponding avatar by means of the photonView.IsMine command (a true value guarantees the execution only on the assigned avatar). Finally, the voice chat is installed for the avatar by means of the Photon Voice View and Speaker script components of the Photon Voice add-on, together with an audio source (within the Unity engine). This allows the remote user’s audio to be played for verbal communication.

The scenes of the application were implemented and evaluated by the developers in search of improvements related to usability and errors, so as to improve system function. Figure 8 shows screenshots of the user interface in the initial scene, which is presented prior to the start of the main scene and allows basic user information to be collected, such as the avatar gender and the user role. In the patient role menu, there is only one input field, which allows the entered name to be registered in the system via keyboard. In the case of the neuropsychologist role, there are two input fields to be completed, the name and password. Finally, the “back” button allows the user to go back to the different menus and the “start” button loads the main VR scene.

Figure 9 depicts the implemented VR environment from the patient’s view. The images show the first-person screenshots with the elements of the virtual office: the video screen tells the story, the maze game is to be solved by the patient and the answers option menu is used for the memory assessment. The VR space contains armchairs, vases and wall paintings, with a large area of 9 × 7 square meters for the avatars to move around. A label appears above the avatar, whose text is taken from the initial scene (to facilitate communication when talking to another person). The audio and video playback screen appears only when the neuropsychologist enables it and disappears in the same way or at the end of the playback. The maze game is solved by taking out the sphere through the only existing exit, for which the user uses the directional keys on the keyboard. The evaluation menu is completed using the mouse to select the answers that the patient considers correct; once the evaluation is completed, the administrator (neuropsychologist role) disables this menu, saving the answers. On the other hand, Figure 10 shows the VR environment from the neuropsychologist’s view, observing the avatars in third person and with the presence of the administrator menu, which is shown and hidden by pressing the letter “m” on the keyboard and is only available for this user role. The available options are: video playback, generation of the maze with the desired level of difficulty and patient evaluation. Should the results not be generated, the option to open results does not appear. In this case, the spreadsheet (.cvs format) containing the following results is opened: number of successes, number of failures and response times of the first assessment; the same also occurs for the second assessment, which is performed after the distraction period.

3.5. Statistical Analysis

Statistical analysis was carried out using SPSS 16. Student t-tests were used to check for differences in age and years of education between the groups at baseline, while Pearson’s Chi-squared test was used to verify the homogeneity between groups in terms of gender. The analysis of the obtained results after the intervention was performed using repeated measures ANOVA with between-subject (intervention: control vs. experimental) and within-subject factors (accuracy response: initial vs. final assessment). A paired t-test was performed to check for differences in reaction times in the experimental group, whereas Pearson’s correlation was used to infer any association between reaction times and age, gender and years of education, respectively. Finally, the experimental group completed a usability test to measure the level of acceptance of the system’s features. The statistical significance level was set to 0.05.

3.6. Participants

Two groups of participants were used for this research. The experimental group performed the tasks via the VR application and the control group performed the tasks following a conventional therapy approach by using printed materials in face-to-face mode. All participants signed an informed consent form prior to their participation in the research. The activities in the experimental group started with the installation of the VR application in the computer, followed by a phase to gain familiarity with the application (via video calls with the researchers), but without performing any rehabilitation activity. Participants were only recruited into the experimental group if they had some experience in handling computer applications.

Table 1. Participant demographics.

Variable		Experimental Group	Control Group
Gender	Male	9	10
	Female	11	10
Age	Mean	32	35
	SD	11	13
Experience VR	Yes	10	0
	No	10	20
Education (in years)	Mean	16	14
	SD	3.6	4.9
Family history	Yes	5	4
	No	15	16
Cognitive exercise	Yes	12	11
	No	8	9

details the demographics of the participants, consisting of 20 people in each group with members of both genders and with ages ranging between 18 and 65 years. Regarding the experimental group participants, 45% of this group were women, 50% had experience with VR or video games, 25% had a family history of neurodegenerative diseases and 60% used to performed cognitive activities on a regular basis. The average age was 32 years, with a standard deviation (SD) of 11 years, while the average level of education was 16 years, with an SD of 3.6 years. On the other hand, the control group contained 50% women, 20% of the participants had a family history of neurodegenerative diseases and 55% of the participants performed cognitive activities on a regular basis. The average age of the group was 35 years, with an SD of 13 years, while the average level of education was 14 years, with an SD of 4.9 years. No significant differences were found by gender (p = 0.525), age (p = 0.543) or years of education (p = 0.094) between the groups.

4. Results

4.1. Experimentation

The data were obtained from the participants belonging to both groups: the control group and experimental group. The latter group also undertook usability tests after the completion of the task. Figure 11 shows two users interacting with the application. The patient is solving a maze (Figure 11a) and the neuropsychologist is interacting with the main menu (Figure 11b). The system was used by 20 healthy patients and a neuropsychologist who conducted the cognitive rehabilitation procedure.

4.1.1. Face-to-Face Mode

The control group performs only face-to-face rehabilitation using the spaced recovery technique, with physical materials such as printed sheets, a pencil and an eraser. The neuropsychologist begun by requesting written consent and explaining the processes to be performed. Then, a story was told with images and verbal descriptions immediately after the initial evaluation was performed. Figure 12a shows the results of the initial evaluation (immediate memory) for the 20 participants, with an average accuracy rate of 5.6/7 (80%) and an average error rate of 1.7/7 (24.3%), the latter corresponding to forgotten or erroneous elements. The next step dealt with the distraction stage, where the patient performed cognitive activities, solving printed mazes for a period of 20 min. The final evaluation dealt with the identification of the elements remembered after this period of time, with an average accuracy rate of 4.8/7 (68.6%) and an average error rate of 1.3/7 (18.6%). The results are shown in Figure 12b. A decrease of 11.4% in the accuracy response rates between the initial and final evaluations can be seen.

4.1.2. Telerehabilitation Mode

Experimentation in the VR application had one more stage in comparison to the face-to-face modality because the 20 participants had to learn to use the system prior to telerehabilitation. The usability of the application was subsequently evaluated. The results of the application were automatically determined, showing a 6.55/7 (94%) success rate and 0.55/7 (8%) error rate in the initial evaluation (Figure 13a), with better scores than those obtained in the face-to-face modality. The system also offers the reaction and response times for each correct element of the evaluation menu. Taking into account that only seven answers were correct, the median times (seconds) obtained were 7.07, 2.65, 2.65, 1.88, 1.53, 2.25, and 2.35, respectively, as shown in Figure 13b. The average response time was 3.74 s, with some outliers.

The results of the final assessment (spaced retrieval) only showed a few mistakes (2% in all tests), with an accuracy response rate of 98% (Figure 14a). Therefore, the performance was better than that obtained for the face-to-face modality (on average). As far as the final evaluation response times were concerned, the following median times (seconds) were obtained: 3.3, 1.69, 1.75, 1.32, 1.43, 1.65, and 2.16 for correct items, with an average response time of 2.21 s. These values were lower (on average) than those obtained in the initial evaluation and there were also outliers.

4.2. Comparative Analyses between and within Groups

The results for the accuracy response rates showed significant differences between groups (F (1, 38) = 12.95, p = 0.001) in favor of the experimental group (telerehab group). In the comparisons within groups, the results did not show a significant difference over time (p = 0.061) or in the interaction effect between factors (p = 0.701). Figure 15 depicts the accuracy response rates (mean scores) for both the initial and final assessments and the group variable. Concerning the response times taken to accomplish the tasks within the experimental group, the average time spent in the final evaluation of the test was significantly lower (p = 0.001) (M = 15.28, SD = 4.61 for the final evaluation versus M = 25.87, SD = 12.05 for the initial evaluation). On the other hand, there was no association (p > 0.05) between response times and age (p = 0.175, initial assessment, p = 0.184, final assessment), gender (p = 0.352, initial assessment, p = 0.376, final assessment) or years of education (p = 0.361, initial assessment, p = 0.406, final assessment) in any of the assessments.

4.3. Usability Analysis

The usability analysis for any application is very important, and in this case even more so since a potential target group for cognitive rehabilitation is the elderly population. The system must have a sufficient degree of usability to avoid interaction difficulties, especially in this elderly group. The designed application, besides being as simple as possible, must also have a high degree of usability. To evaluate the usability of our system, we have used the suitability evaluation questionnaire (SEQ) test described in [54]. The instrument as designed to evaluate VR applications in rehabilitation activities. The test consists of 14 questions, comprising 13 closed-response items (Q1–Q13) and one item with an open-response option (Q14: “If you felt uncomfortable during the task, please indicate the reasons”). The closed-answer questions were assigned based on Likert scales, and the scores obtained for questions Q1, Q2, Q3, Q4, Q5, Q6 and Q11 were directly used for the overall score, whereas the scores obtained for questions Q7, Q8, Q9, Q10, Q12 and Q13 were processed by subtracting them from 6 (6-Qi). The questionnaire was given to the 20 participants after use of the VR application, the results for which are shown in

Table 2. Usability test results (n = 20).

Question	Mean	SD
Q1. How much did you enjoy your experience with the system?	4.65	0.45
Q2. How much did you feel you were in the environment of the system?	3.55	0.57
Q3. How successful were you in the system?	4.70	0.46
Q4. To what extent were you able to control the system?	4.40	0.80
Q5. How real is the virtual environment of the system?	4.20	0.87
Q6. Is the information provided by the system clear?	5.00	0.00
Q7. Did you feel discomfort during your experience with the system?	1.10	0.30
Q8. Did you experience dizziness or nausea during your practice with the system?	1.00	0.00
Q9. Did you experience eye discomfort during your practice with the system?	1.10	0.30
Q10. Did you feel confused or disoriented during your experience with the system?	1.40	0.49
Q11. Do you think that this system will be helpful for your rehabilitation?	4.65	0.63
Q12. Did you find the task difficult?	1.40	0.66
Q13. Did you find the devices of the system difficult to use?	1.50	0.67
Total	59.65	0.48

. The overall score of 59.65/65 (91.77%) indicated the high usability of the system, with very low standard deviation (SD), denoting minimal difference in criteria. On the other hand, the lowest score was obtained for Q2, showing that immersion in the virtual environment was limited (it is a non-immersive system). With respect to Q14 (related to task discomfort), no participant reported any related discomfort.

4.4. Bandwidth Requirements

The online multi-user application may be limited by network requirements, so we analyzed the bandwidth requirements for a suitable level of function for the VR application. The communication between users involves the exchange of two types of data: the synchronization of movements and actions within the VR environment and conversation through voice. Figure 16 depicts the transmission rates in a common SR-based rehabilitation session that lasted 27 min and 30 s. A sampling rate of 15 s was used and transmission rates are shown in kilobytes per second (Kb/s). The download rate showed a minimum of 15.8 Kb/s, a maximum of 48.9 Kb/s and an average of 28.8 Kb/s. For upload rates, there was a minimum of 12.6 Kb/s, a maximum of 31.8 Kb/s and an average of 17.5 Kb/s. In general, very low transmission speeds were observed during normal operation due to the optimized performance of Photon Cloud, which synchronizes the two virtual environments through state updates, such as for positions, rotations, instructions and activations (without images), together with the exchange of voice data. In summary, the bandwidth requirements are very low and transmission rates ensure that this cognitive telerehabilitation approach can be applied in all households with a simple internet connection.

5. Discussion and Conclusions

The objective of cognitive rehabilitation is to compensate for or repair certain cognitive functions that have been impaired by either age-related changes or a specific medical condition. The use of technology has made it possible to migrate those processes that were previously performed manually into computer-developed interfaces. The implementation of VR environments in the treatment of cognitive diseases has been taking place in recent years as a new alternative to conventional procedures [55,56]. VR therapy has been tested to improve cognitive abilities in people suffering from Alzheimer’s disease, Parkinson’s disease, trauma or after a stroke, among others [6,49,57]. Similarly, other psychiatric pathologies have been treated, as well as dementia, eating disorders and phobias [46,52,58].

Undoubtedly, VR-based interventions have been especially numerous in the cognitive rehabilitation of Alzheimer’s patients, providing satisfactory results, although the long-term effects are not yet fully known [57,59,60]. However, these sessions are usually administered face-to-face and the use of telerehabilitation is relatively new for the treatment of this disease [49]. Something similar occurs in post-stroke treatment, whereby the application of VR has already been widely developed [37,40,42], however remote assessment is more recent [9,15,50]. This shows that the application of VR in telerehabilitation is a new trend, the implementation of which has been accelerated by the COVID-19 pandemic. It is important to highlight that in synchronous sessions (real-time) a high interaction is achieved, while in asynchronous sessions this depends on the patient’s commitment [58].

In this work, we have developed a VR-based system that allows effective interaction between the patient and the specialist physician for cognitive rehabilitation purposes. We have implemented the spaced retrieval method for cognitive rehabilitation, a technique that had previously only been applied manually [20,25,26], but has now been digitized through the use of immersive environments. The system makes it possible to evaluate the patient’s ability to retain information over time. The results of our study, validated with healthy subjects aged 18 to 65 years and without diagnosed cognitive problems, show that it is possible to perform telerehabilitation or cognitive training with the help of virtual-reality-based applications. In fact, according to the assessments in the designed task, it should be noted that in both the initial and final assessments, there were significant differences in favor of the experimental group (telerehab group). In line with our study, Meltzer et al. compared a tablet-based telerehabilitation treatment with a face-to-face treatment for a total of 44 adults with communication disorders after stroke. They obtained statistically equivalent gains between the face-to-face and telerehabilitation groups for most of the tests used [61]. We can find another example in the study by van der Linden et al. [62], who developed an iPad-based cognitive telerehabilitation program for patients with primary brain tumors based on previously evaluated face-to-face cognitive programs. They demonstrated its feasibility through measures such as adherence, attrition, accrual and patient experience.

This leads us to believe that not only are both cognitive training systems equivalent (face-to-face versus remote), but that the participants achieved better success rates in the two evaluations of the spaced retrieval task. This slight improvement in scores in the experimental group may have been due to greater focus on the activity and the memorized items being presented in an engaging environment within the VR setting. The adoption of a telerehabilitation system such as the one developed in this work promotes a series of clearly positive factors in cognitive rehabilitation, such as the possibility of more intense treatments, repeatability, the promotion of engagement, as well as the ability to transfer cognitive skills to the real world, as pointed out by Mantovani et al [6]. In addition, the proposed system acquires the user’s response times, information that is relevant for the patient’s cognitive analysis.

According to the results obtained in the usability test, we can state that the application helps in performing pleasant rehabilitation sessions in the same way that conventional therapies do. However, the results need to be validated with additional tasks that demand greater attention and cognitive effort. Although our study was a quasi-experimental and controlled design with a reasonable number of subjects, it would be desirable to validate the system with patients suffering from certain types of neurodegenerative disease, such as Alzheimer’s disease (AD). In our favor, recent studies have pointed out important improvements in the cognitive rehabilitation of this population, such as the study conducted by Small and Cochrane [63], where interventions with SR helped to maintain the functioning of recent episodic memory. The study by Hawley and Cherry [64] showed that the training with the SR technique had a positive influence in the recall of simple face–name associations.

We have created a framework with which to meet future needs in the field and that can be used by healthcare staff to promote cognitive telerehabilitation. Within this framework processes can be automated, therefore allowing data collection in real time for subsequent analysis and decision-making. At the same time, social distancing measures due to the COVID-19 pandemic are promoted, increasing the safety of patients who can access rehabilitation services from the comfort of their homes. The advantages that our system provides (simplicity of use, high usability and cost-effectiveness, among others) are clearly in line with making healthcare systems more efficient and overcoming geographical limitations (for examples in areas that are difficult to access or areas with limited health services) and temporal limitations (reduction of waiting times). In the same way, it allows patients with reduced mobility to access healthcare professionals. For the above reasons, and due to the resource savings it offers, we can state that our approach is also environmentally friendly.

The design of this system was influenced by the great restrictions and difficulties inherent to the COVID-19 pandemic, such as reduced patient mobility, overrun health services and difficulties in carrying out community-based interventions. The fact is that the lessons learned from the COVID-19 pandemic have served to develop new technological advances, which are here to stay and will certainly be used in the future, even after the end of the pandemic [6]. In anticipation of future pandemics that may significantly affect our everyday life, the implementation of such approaches in the field of cognitive rehabilitation will undoubtedly help health systems to improve their efficiency in terms of the use of resources and time, to increase their productivity, and above all, patients may benefit from these telerehabilitation programs.

In future work, the authors of this research propose several lines of action. First, the developed system should be validated with patients with both mild and moderate cognitive impairment, for example patients suffering from Alzheimer’s disease. Evidently, it will be necessary to include spaced retrieval tasks that demand a little more attention and effort in order to find contrastable evidence. Similarly, a usability evaluation will have to be taken into account, as this is critical when working with users who have specific needs [65]. This evaluation should be performed in the design stages of the application, considering the execution time, level of immersion, comfort and possible adverse side effects on the participant. On the other hand, certain autonomous rehabilitation activities could also be added to the application that would allow users to train memory function without the need for a specialist physician and without removing the multiuser features that characterize this proposal. Finally, an important aspect that should be addressed in the case of discharged COVID-19 patients is the study of the different recovery trajectories through multidisciplinary telerehabilitation programs, in line with what Salawu et al. pointed out in their study [66]. In fact, the developed tool allows the implementation of a multidisciplinary telerehabilitation program that encompasses the different post-COVID 19 sequelae, such as psychological issues, cognitive disorders, physical problems and speech and language disorders.