Virtual Reality Environment with Haptic Feedback Thimble for Post Spinal Cord Injury Upper-Limb Rehabilitation

Abstract

Cervical spinal cord injury is damage to the spinal cord that causes temporary or permanent changes in body functions below the site of the injury. In particular, the impairment of the upper limbs limits the patient’s autonomy in the execution of activities of daily living. This paper illustrates the use of a low-cost robot with a virtual reality platform for upper limb rehabilitation of cervical spinal cord injury patients. Vibration and pressure haptic feedback sensations are provided thanks to a custom-made thimble feedback device. The virtual reality platform consists of three different virtual rehabilitation games developed in Unity. They provide the user with the opportunity to interact with the virtual scene using free hands thanks to the data collected by a hand tracking system. During the therapy session, quantitative data about the motor performance are collected. Each virtual reality environment can be modified in settings according to the patients’ needs. A proof of concept was performed with both healthy subjects and spinal cord injured patients to evaluate the platform and its usability. The data saved during the sessions are analyzed to validate the importance of haptic feedback and stored both for patients and therapists to control the performance and the recovery process.

1. Introduction

Spinal cord injury (SCI) is mechanical damage to the spinal cord. It can be traumatic when the damage results from physical traumas (car accidents, work accidents, gunshots, falls, sports injuries, etc.) or non-traumatic when it is caused by an underlying pathology (infection, tumors, musculoskeletal disease, etc.) [1]. SCI causes an alteration of sensory and motor function since there is an interruption of nervous tissue that performs the communication between the brain and the rest of the body [2]. According to the level of the injury, there are different types of paralysis that may be grouped as tetraplegia (arms, trunk, legs, and pelvic organs are affected) and paraplegia (trunk, legs, and pelvic organs are affected). Moreover, SCI can be classified according to the severity in incomplete injury when nerve communications are partially interrupted (some sensory and/or motor activity is retained) and in complete injury when the damage completely impedes nerve communications (no functions retained). Therefore, determining the cord segments affected by the spinal cord injury appears to be crucial in order to understand the gravity of the situation and to adapt the rehabilitation plan [3].

The worldwide incidence of SCI lies between 10 and 83 per million inhabitants per year [4]. In Spain, approximately 1000 new cases of SCI per year occur due to trauma, which is the main cause, half due to traffic accidents, and the rest due to falls, blows, sport accidents, or other injuries [5]. Most of these people have permanent damage. It is observed that people with SCI have difficulties and limitations in mobility (96.9% of cases), in self-care (81.1% of cases), and in carrying out tasks of domestic life (84.3% of cases). Regarding the economic aspect, the lifetime financial burden brought on by SCI conditions can be massive for patients, insurance companies, and hospital systems. The costs associated are greatly affected by both the patient’s extent of injury and the subsequent degree of disability, regarding the acute and post-acute phases as well as the chronic phase within the context of longer-term rehabilitation [6]. Reducing therapy cost by including lower cost technologies to improve the quality of life for SCI injured patients is suggested.

It is estimated that cervical SCI accounts for approximately more than one-third of all people living with SCI [4]. The loss of upper limb (UL) function and in particular the use of the hands is one of the most devastating losses an individual can experience [7]. In contrast with lower limbs, upper limbs comprise different joints that allow execute several fine movements thanks to complex neuromuscular control. This results in a considerable and crucial functionality of these elements. Consequently, upper limb and trunk motion impairment caused by SCI has a high impact on the independence and quality of life of the affected person [8,9]. Each small improvement in upper limb motor control is fundamental and translates to important amelioration in the patient’s independence and confidence in everyday life [10]. In particular, the role of the hands appears to be fundamental in everyday life and could reduce dependencies on the caregiver: Even with few movements of the finger, a tetraplegic person could accomplish different useful tasks such as pressing a switch, interacting with a keyboard to communicate, or holding an object [11]. Among others, pinching and grasping movements are essential gestures in everyday life situations.

In rehabilitation, functional neuro-recovery is emphasized, which aim is promoting the restoration of function through use of the affected limbs [12]. The goal is to promote plasticity, that is, the ability of neurons to rearrange their anatomical and functional connectivity in response to environmental input, thereby achieving new or modified outputs, namely, behaviors. The process of plasticity consists of changes in the activation pattern either in structure or function (neuronal excitability and inhibition, conduction velocity, and synaptic efficacy) that involves alteration of the strength of old and new connections [13,14]. To achieve these effects, a fundamental aspect is the repetition of exercises to strengthen the neural circuit and to start the process of recovery. In addition, the results depend on the effort required, so SCI individuals need to maintain a high degree of participation and involvement to facilitate motor learning [15]. Nowadays, technology is typically involved in rehabilitation since it represents a motivating solution to these needs.

In this scenario, there is an increasing interest in the use of robotics and virtual reality (VR) in neurorehabilitation therapies. Combined with traditional rehabilitation, they can improve the efficacy of training sessions and ameliorate patients’ quality of life. The use of technologies increases the number of training sessions with consistent repetitions, delivering a high volume of high-quality movement, promoting functional recovery, and enhancing adaptive plasticity [16]. Moreover, it enables the performance evaluation of patients during therapy, and objective and quantitative assessment after therapy [17]. With robotics, therapy programs can be automated or semiautomated and do not require close supervision or support from physical or occupational therapists. In this way, these could be potentially delivered as home therapy programs with the condition of smaller, more portable, and less expensive devices [18]. VR rehabilitation introduces interaction and immersion, allowing the patient to interact and participate in real time with the virtual world while having an inclusive, extensive, surrounding, and vivid illusion of reality [19]. In addition, it is possible to establish countless different fun and motivating environments and to develop plenty of skills and task-based techniques that can stimulate and enhance participants’ interest and motivation while maintaining a safe environment [20]. When used in conjunction with computer-based virtual reality, robotics allows haptic feedback to be given to the patient while they are freely moving in the VR environment, improving somatosensory recovery. Somatosensory-motor integration has been demonstrated to be fundamental and thereby ameliorating the somatosensory system would improve motor recovery. Consequently, different solutions have been studied in order to enhance rehabilitation using both virtual platforms and wearable actuated haptic devices.

The devices used for UL rehabilitation can be grouped, depending on the part of the somatosensory system they stimulate, in kinaesthetic/proprioceptive devices and tactile/cutaneous devices. The first type provides forces that guide, resist, or interfere with movement and deliver force feedback in response to physical properties or movements of virtual objects, allowing the patient to sense the object’s weight and size. The tactile devices transmit information about various physical parameters mostly related to the surface properties of objects, including temperature, texture, shape, and roughness [21]. This haptic thimble presented in this paper belongs to cutaneous devices.

Gloves with external exoskeletons and endpoint devices are typical examples of kinaesthetic devices [22]. Viau et al. [23] compared the performances achieved in terms of movements done in physical and virtual environments in adults with motor deficits (hemiparesis after stroke) with respect to those in healthy individuals. In this case, the external exoskeleton CyberGrasp (CyberGloveSystems) mounted on a sensorized glove delivered prehension force feedback in the form of extension forces to the distal phalanxes of the thumb and each finger to simulate the interaction with a virtual ball. More recently, Connelly et al. [24] developed the “PneuGlove,” a pneumatic glove that can provide independent extension assistance to each digit while still allowing full arm movement. It was proved in combination with a virtual environment where different objects were presented with the aim of performing grasp-and-release tasks in a given time. The same principle has been used and improved in recent years in the device developed by Cappello [25], where finger flexion and extension are obtained thanks to three fabric layers and two air-tight bladders placed between each fabric pocket. Examples of endpoint devices are PHANToM by Geomagic and the Novint Falcon [22]. Different studies used the Novint Falcon in combination with other devices to build up a virtual reality environment for patients with diseases of mobility in the upper extremities [26,27]. However, these technologies are bulky and expensive, so they are not the best choice for a low-cost platform for SCI patients.

Tactile stimulation has been exploited with greater emphasis on vibrotactile or electro-tactile stimulation [21]. Dimbwadyo-Terrer et al. [8] combined CyberTouch, a system composed of a glove with a vibro-tactile stimulator on each finger and the palm, with a VR environment for therapeutic training of reaching movements after spinal cord injury. Nonetheless, wearing a glove can sometimes be difficult and that is why researchers have shifted the focus to thimbles, smaller devices that are easier to wear and more “realistic” in the feedback that they can provide. Yem and Kajimoto [28] developed the FinGAR (Finger Glove for Augmented Reality), a tactile thimble device that combines electrical and mechanical stimulation to selectively stimulate skin sensory mechanoreceptors. Pacchierotti et al. [29] presented a wearable 3-DOF (degrees of freedom) fingertip device for interaction with virtual and remote environments. It consists of two platforms connected by three wires controlled by three small electrical motors and one force sensor placed in contact with the fingertip at the center of the platform for closed-loop control. Based on the same principles, the 3RRS (Revolute-Revolute-Spherical) wearable fingertip device [30] is composed of two parallel platforms that are instead connected by three articulated legs, actuated by three motors in order to make and break contact with the skin and move or re-angle the mobile platform toward the fingertip. Recently, [31] developed a soft, flexible artificial skin made of silicone and electrodes. The system allows the artificial skin to adapt itself to the user and provide haptic feedback in the form of pressure and vibration. Strain sensors continuously measure the skin’s deformation so that the haptic feedback can be adjusted in real time thanks to soft pneumatic actuators that form a membrane layer that can be inflated by pumping air into it. All these devices can be used for upper limb rehabilitation in spinal cord injury patients if used alongside computer-displayed virtual reality environments.

The aim of this paper is to develop a set of serious games based on UL functional tasks that allow neurorehabilitation in SCI patients. Haptic feedback should be provided to the user by means of a non-detachable device to prevent the patient from becoming fatigued simply by holding it [26]. However, issues such as donning and doffing haptic gloves should be overcome. Therefore, a low-cost haptic thimble, such as the one presented in [32], should be built to simplify the interaction with the patient and the serious games. The haptic device must be thin enough so that a hand tracking system can extract finger parameters. Two haptic feedback effects are selected for their importance in serious game interaction: vibration and pressure. Vibration produces molecular mechanisms that mediate signal transduction at the tissue level [33]. The current general consensus about vibration is that the Pacinian corpuscles sense vibration with a frequency range from 40 to 800 Hz and a peak from 200 to 300 Hz. Therefore, the haptic feedback should provide a vibration frequency of at least 200 Hz. Moreover, one single contact point is enough for the perception of the material hardness [22], and this hardness can be provided to the user by means of pressure. A moderate pressure of 10 N is considered enough to feel the hardness of the most common materials [34].

The rest of the paper is organized as follows: Section 2 describes the methodology in relation to the design and implementation of the feedback device, the integration of the hand tracking system into a virtual reality platform, the development of the serious games, and the validation proof. Section 3 presents the results of the implemented proof of concept. Section 4 includes the conclusions.

2. Materials and Methods

2.1. UL Rehabilitation Platform

The complete platform is composed of the haptic thimble connected to a commercial microcontroller (i.e., Arduino board). The microcontroller receives information from the VR games and controls the thimble motor to provide vibration and pressure sensations. A commercial LeapMotion device acquires information from the hands’ movements to interact with the game. The designed games are run in a standard PC and are developed in the Unity 3D commercial environment (see Figure 1).

2.1.1. Haptic Thimble

The haptic thimble (see Figure 1 right) is a prototype device designed to provide vibration and pressure haptic feedback. The device consists of three main mechanical components (base, platform, and reel) assembled together with a direct current motor and designed to provide feedback thanks to a traction system based on cables (see Figure 2).

The main objective is to take advantage of the same feedback principles of tactile gloves but to restrict them from the whole hand to the fingertips in order to solve the problem of discomfort of the patient when donning and doffing the device. A second objective is to reduce the cost of manufacturing and the complexity of the thimble, to be able to build a device that could be acquired, together with the software, as a low-cost platform for its use in home-rehabilitation sessions.

The mechanical parts were designed using a computer application for 3D mechanical design and then 3D printed (files for replicating the thimble can be found at: https://github.com/Robolabo/softHapticThimble, accessed on 9 March 2021). The main piece, the base, is placed on the back of the fingertip and has the functions of securing the whole structure on the finger and holding the motor in its position. The second part is a parallel platform that is pulled through a cable to the first part, so it is the one that produces the feedback touching the user’s fingertip. The third one, a reel, is directly connected to the motor axis and allows the platform to be pulled or released towards the base when it rotates. The transmission system is implemented by a steel cable 0.8 mm in diameter; this allows the platform to be pulled towards the user finger. However, for the platform to be released, the thimble includes three compression springs attached vertically. These springs allow the base and the platform to be parallel and stable thanks to the three contact points. Moreover, they provide a soft compression depending on the user’s finger characteristics with three degrees of freedom. The springs selected are 25 mm in length and 4 mm in diameter, have 45 N/m of stiffness, and have a maximum compression of 13 mm. A maximum distance of 20 mm between the platform and the base was selected for the resting position. It allows a user with an average finger size to wear the thimble without having the parallel platform in contact with the tip of the finger, thus avoiding incorrect feedback.

Two Velcro strips are assembled on the base in order to secure the device to the user’s finger. The motor used is a 298:1 micro metal gearmotor 12 V equipped with an extended motor shaft that allows a magnetic encoder of 12 quadrature counts per revolution (Pololu, 3056) to be mounted.

The motor is controlled using a vibration function and a PWM (Pulse Width Modulation) implemented on a proportional and derivative control implemented on the Arduino and triggered by the data received by the games. Moreover, the firmware allows the patient to modify the distance of the platform from the base, thus calibrating the device to different needs. The Arduino Due board is connected to the PC through a USB cable and is integrated with the X-NUCLEO-IHM04A1 board (STMicroelectronics), a power stage that allows the direction and the velocity of the motor to be controlled through an H-bridge circuit. This last board is mounted on top of the Arduino. The electronics are incapsulated in a 3D printed box and connected through connectors and a 114 cm-long cable to the thimble, where the cables are welded to the encoder board.

The device is lightweight to avoid additional weight that would increase the inertia that in turn would affect the hand movements, resulting in erroneous gestures. It has a small design to not interfere with hand gesture recognition. Moreover, thanks to the calibration option, it is adaptable to different-sized fingers and it does not interfere with the haptic perception thanks to the connection on the second phalanx of the user’s finger. The solution adopted by placing the electronics on the table and connecting the thimble with a long cable makes the device wearable and the connectors add an easy interchangeability to the whole system.

The thimble provides vibration up to 400 Hz and force feedback up to 16.17 N. However, the most relevant aspects are related to the response of the thimble to the contraction values sent by the microcontroller and therefore by the games. This is useful for understanding how the thimble will behave during the game session based on the movements of the user. To carry out this evaluation, the distance between the base and the parallel platform after sending the serial data was measured.

For each measurement, the thimble was connected to the Arduino. The same firmware implemented for the platform was used to ensure a consistent evaluation, running at a sampling period of 1 ms. The measurement procedure consisted of sending 8 bits of data from the serial port and measuring the distance between the base and the platform in its steady state. Values from 0 to 240 in steps of 20 were sent. An extra value of 255 was also sent, being the highest value for the position controller.

The results of the distance between the base and the parallel platform in millimeters are shown in Figure 3. As can be observed, the thimble exhibited an almost linear behavior. The initial stall was because of the beginning of the movement of the motor, to which the springs were opposed. The thimble went from a maximum distance of 20 mm, corresponding to the minimum value sent (0), up to a minimum distance of 12.80 mm, corresponding to the maximum value sent (255).

2.1.2. Leap Motion

The Leap Motion controller [35] is a commercial USB peripheral device designed for hand tracking. The heart of the device consists of two monochromatic infrared (IR) cameras and three infrared LEDs. These LEDs generate patternless IR light with a wavelength of 850 nanometers, which is outside the visible light spectrum. The light is reflected by the objects and reaches the two cameras, which generate almost 200 frames per second while capturing these data. The data received by the camera are read and treated with necessary resolution adjustments in the memory included in the device. At this point, the obtained data are sent through a USB to the host computer. Here, the Leap Motion service software processes the images using heavy mathematical tools. It first compensates for background objects and ambient environmental lighting. Then it reconstructs a 3D representation of what the device sees. Moreover, the tracking algorithms extract information relative to the positions of the hand and fingers. After a filtering process, the information is managed by the software and expressed as a series of frames, or snapshots, containing all of the tracking data.

In this paper, the Leap Motion controller was selected because it is a low-cost device with a small observation area and high resolution that make it suitable for upper limb and hand rehabilitation. It is used to track index finger and thumb movement and to provide position information from the serious games. Moreover, it allows quantitative information to be collected about the patients’ movements.

2.2. Serious Game Development

For the serious game development Unity 3D was used, a commercial environment designed for the development of 2D and 3D applications, games, and programs. The first step to create the rehabilitation platform was to integrate the Unity 3D environment and Leap Motion. It was achieved thanks the Leap Motion Unity plugin. Leap Motion service software analyzes images produced by the hardware and sends tracking information to the applications. The Leap Motion Unity plugin connects to this service to get data. Scripts included with the plugin translate Leap Motion coordinates to the Unity coordinate system.

Subsequently, games with a therapeutic sense were implemented. Multiple variables related not only to the physical condition, but also to the emotional and psychological state of the patients were considered in developing games for rehabilitation purposes. The design of the games needed to ensure that the games’ characteristics did not compromise the rehabilitation effort; thus the exercises conceived for rehabilitation need to be simple and have clear goals and limitations. A motivating and encouraging feedback system increases the involvement of the player with the game and reduces the risk of abandonment. Since these games were intended for SCI patients, they needed to account for different capability levels that depend on the type, the level, and the seriousness of injury. The games included the possibility to use the haptic device and a “calibration” option. In addition, one design consideration was to provide the therapist with patient identification and performance during sessions in order to conduct future evaluations.

The goal of all of the games was to achieve as many objectives as possible in a limited time frame. The games differed from each other by the scene presented, but more importantly by the objective of rehabilitation and the type of haptic feedback provided. A summary of the main characteristic of each game is shown in

Table 1. Summary with the main characteristics of each game.

Game	Exercise Type	Feedback	Therapeutic Objective
Maze	Path guidance	Vibration	Arm movement, accuracy
Path	Path guidance	Pressure	Arm movement, accuracy
Grab	Reaching objects, grasping and releasing	Pressure	Coordination, hand-grasp improvement

.

2.2.1. Maze

The first game developed is entitled “Maze.” The main idea is that the user controls the position of a squirrel with the index fingertip position (see Figure 4a). The objective is to guide the squirrel to collect the acorns positioned in different places in the scene, thus gaining points. When the squirrel touches the walls, an error counter increases and vibration feedback is provided to the user through the haptic device. The game continues until the user exits the maze or the time ends. In this game, there is not a selection of the number of objectives. The path is the same in all configurations.

This game was implemented in 2D (x- and y-axes) because the depth of the scene could cause trajectory disturbance in patients.

Therapeutic Objective

The therapeutic objective is to improve the precision in the UL movement made by the patient and to recover fine motor control. The game scene is implemented to force the patient to move their entire arm. In fact, to reach the acorn in the upper part of the maze or just to travel the path while avoiding errors, the patient is forced to raise their arm. Moreover, the therapeutic effect implies an improvement of accuracy because the user sees the objectives (acorns) and tries to reach them, paying attention to not touching the walls. The haptic feedback makes them aware of the errors, thus trying to reduce them.

In this training, the errors, the points, and the trajectory performed by the patient acquire relevant importance. For this reason, the trajectory made by the patient was collected together with velocities for each frame and the serial data sent to the thimble as a function of time.

2.2.2. Path

The environment in this scene is also in 2D and is composed of a blackboard inserted as a background and a figure drawn on it. This figure is a shape composed of different circles colored in red connected by blue lines (see Figure 4b). The main idea in this case is that the user controls the position of a white dot with the index fingertip position. After setting the desired parameters (time and level), the user touches a red circle and the game starts. The different levels allow the user to decide the intensity of the pressure feedback received: the higher the value, the higher the pressure applied when leaving the path. The objective is to guide the dot and to travel the line between the circles to gain points. When the dot leaves the path an error counter increases. In this game, the feedback implemented is different than the vibration in the Maze game. If the user moves the white point outside of the path, the haptic device provides pressure feedback by squeezing the finger in proportion to the distance from the path. The user must travel all the lines before the time ends.

Therapeutic Objective

This rehabilitation game is based on a path guidance exercise. The therapeutic objective is to improve the precision of the UL movement made by the patient when traveling the lines and to recover fine motor control. It strengthens the accuracy of the movement by encouraging the user to remain on the path.

The principle on which this feedback is based is Hooke’s law (see Equation (1)). This states that the force F exerted by a spring is equal to a constant that depends on the spring, times the displacement or change in length.

F (kT) = kv ∆X(kT),

(1)

where T is the sampling time between frames, kT is a specific instant in the discrete domain (k ∈ [0, 1, 2, …]), ∆X(kT) is the distance to the edge of the path, and kv is the stiffness or elasticity simulated on the edge of the path and set with the chosen level.

The game scene is implemented to force the patient to move the entire arm. In fact, the patient has to raise their arm to reach the red circles in the upper part. The haptic feedback tries to make the user aware of their errors and redirect them to the path by applying a force proportional to the distance.

2.2.3. Grab

The third game developed is entitled “Grab.” This game is completely different from the other two both in terms of scenario and rehabilitation purposes. It is a 3D game in which the user can freely move and see their hand in 3 dimensions interacting with solid objects.

In this game, the user is situated in a virtual kitchen located precisely in front of the cooking hob. The surface of the kitchen shows a wooden tray and a pot from which steam comes out. The main task of this game is that after setting the initial options, the user picks up the objects that appear on the tray and puts them in the pot. Each time an object enters the pot, it is hidden, the point counter increases, and a new object appears on the tray (see Figure 5).

When the user grasps an object, pressure feedback is applied to the index fingertip. The sensation is different for different elements. The objective of this game is to leave as many objects as possible inside the pot in a limited time. When the time ends, the results are shown and stored.

Therapeutic Objective

The rehabilitative objective of this game is to improve object reaching capability, and thus coordination. Moreover, it aims to strengthen the hand-grasp movement by forcing the patient to repeat the same gesture several times. Thanks to the possibility of providing different hardness sensations, the objective of this game is to improve the sensorial perception of objects.

The feedback is proportional to the distance between the thumb and the index finger. The stronger the grab, the higher the pressure exerted on the fingertip. Moreover, different hardness is added to objects so that the user can perceive the difference between them. The script is made by adapting a script in the Leap Motion Unity plugin. If an object is close, it is grabbed when the fingers approach each other beyond a certain value named “grabbing distance.” In this moment, the object passes its hardness value to the feedback proportioning script that, from the beginning of the grip, continuously checks the distance between the index finger and the thumb. The distance value is multiplied by the hardness of the object. This value is remapped to a value between 0 and 255 so that smaller distances are matched by larger values to be sent. The formula used in this case is

Fij = (kmax − ki)*(ϑ − dj),

(2)

where kmax represents the maximum value that k can acquire, ki is the hardness of the grabbed object, ϑ is the grabbing distance (i.e., the threshold distance that the user needs to pass to grab an object), and dj is the distant between the index finger and the thumb.

2.3. Participants

Thirteen people participated in the platform validation, eight healthy subjects and five cervical SCI patients all between 23 and 45 years of age.

Table 2. Spinal Cord injured patients’ characteristics.

Patient	Age	Gender	Injury Level	AIS *
1	40	F	C4	D
2	33	M	C7	C
3	43	M	C8	A
4	40	M	C5	B
5	43	M	C6	A

* AIS (ASIA Impairment Scale).

shows the clinical characteristics for all SCI patients. Because of the reduced size of the participant sample, the results could not be considered a statistical evaluation; to achieve this, a pre-clinical study with a higher sample of SCI patients should be carried out. Nonetheless, in order to test the haptic thimble and the virtual reality platform, a proof of concept was implemented with the objective of providing a technical validation of the complete system. The results are shown in Section 3.

The arithmetic mean of the values of the two populations was considered for the comparisons. The patients were recruited from the inpatient populations at Hospital Nacional de Parapléjicos in Toledo (Spain). They were asked to participate in the trials voluntarily. All patients involved in the study needed to have a minimum control of their UL, including shoulder and wrist movement allowed. All the participants had to fill in the corresponding informed consent and follow the inclusion criteria. The study was conducted in accordance with the Declaration of Helsinki and the protocol was approved by the Ethics Committee of the Complejo Hospitalario de Toledo (N° 372 (30/04/2019)).

2.4. Experimental Setup

The setup was exactly the same for the healthy subject and the patients. The only difference was in the structure on which the platform elements were placed. For the patients that were in a wheelchair, in fact, the normal table used for healthy subjects was not suitable since they assumed uncomfortable positions. In their case, a height-adjustable table was used to allow the test to be carried out in the same position. The setup was mounted by placing the Leap Motion controller 10 cm in front of the computer. The box containing the boards was wired to the computer and positioned to its right in order to insert the thimble on the index fingertip of the right hand of the user. The user placed their elbow about 20 cm in front of the Leap Motion controller (see Figure 6). Before the session started, the user was introduced to the platform with a brief explanation of its characteristics. The main functioning and objectives of the three games were explained, along with the feedback haptic thimble device and the Leap Motion controller. Once the introduction part was complete, the game session started. First, the initial windows were navigated and the data relative to the user were saved. The first game accessed was Maze, then Path and at the end Grab. In the setting window the user was asked to move their hand in order to get used to the virtual depiction of the hand movements.

Two sessions for each game were completed. In the first one, the user was asked to wear the thimble, but no haptic feedback was applied. In the second one, the system provided haptic feedback. Keeping the thimble, even if it was not activated, was useful for imposing the same conditions in both sessions. Therefore, it was possible to make a meaningful comparison in the posterior analysis.

2.5. Data Processing

In the Maze and Path games, the X, Y, and Z position and velocities of the index fingertip were acquired along with the minutes and seconds from the beginning of the game. Moreover, the signal sent to the thimble was saved. These data were inserted in a file named with the game name, the date, and the exact time of the session. In the Grab exercise, the important parameter saved was the distance between the thumb and index. Moreover, to have an idea of the performance, the hardness value of each picked object was saved along with the serial data sent to the thimble and the time for each frame.

3. Results

We implemented a proof of concept with eight healthy subjects and five SCI patients to envision the capabilities of the system. As stated in Section 2, the proof of concept provides information about the system functionality to prepare it for a future pre-clinical study. Nonetheless, average and standard deviation results in relation to the system performance for each relevant variable within the games are shown in

Table 3. Average and standard deviation results for the three games with healthy and SCI groups.

	Healthy Subjects (n = 8)		SCI Patients (n = 5)
	Average	Standard Deviation	Average	Standard Deviation
Maze
ROM X (mm)	362.86	14.93	369.33	23.52
ROM Y (mm)	231.79	18.10	277.34	40.84
Duration (s)	36.62	9.44	49.50	14.47
Points	8.50	0.76	6.90	1.82
Errors	10.69	7.33	16.70	8.58
Error no feedback	12.12	7.61	22.80	12.93
Error feedback	9.25	7.09	10.60	5.64
Path
ROM X (mm)	288.65	22.61	282.86	9.72
ROM Y (mm)	330.04	17.52	319.98	10.94
Duration (s)	38.62	11.39	54.40	6.57
Points	13.94	2.76	14.1	3.38
Max value sent	114.31	58.86	90.3	36.34
Errors	7.75	4.77	10.10	4.05
E. NO feedback	7.85	5.54	11.00	4.64
E. feedback	7.62	5.40	9.20	3.63
Grab
ROM (mm)	95.92	12.10	106.80	13.08
Points	10.88	3.09	6.40	2.99

.

The results of the arithmetic mean between the data obtained by each user in both trials (with and without feedback) are shown.

3.1. Maze

The first value analyzed for each user was the range of motion (ROM), defined in this work as the maximum distance reached in the game. For this reason, this was calculated as the difference between the maximum and the minimum value of the position reached with the index both on X and Y. On the X-axis there was no significant difference between the two groups. However, a strong difference (46 mm in average) was found on the Y-axis. This suggests that the patients did not reach the upper and/or lower extremes of the maze. This was due to the less controlled movement of the limbs: Once the movement started, it was more difficult for the patients to stop. This was also reflected in the higher average error rate for patients (16.70 for SCI patients vs. 10.69 for healthy subjects). The average of the points obtained shows that it was more difficult for the patients to reach all acorns before time ended. Alternatively, this could mean that they could not get to the places where the acorns were positioned. The results show that, on average, patients took longer to complete the exercise (49.50 s for patients vs. 36.62 s for healthy subjects). This suggests slower movements. In the data collected, an attempt was made to evaluate the differences between the sessions completed with and without haptic feedback. The number of errors was considered the most significant for this evaluation. This consideration derives from the fact that the objective of the feedback was to make the user more aware of the interactions with the virtual environment (in this case, the errors). Figure 7 compares the results between the two groups for the experiments with and without haptic feedback. As expected, we observed that on average, healthy subjects made fewer mistakes than SCI patients. However, more important is that in both groups, the average of errors without feedback (12.12 for healthy subjects and 22.80 for patients) was higher than the average of errors made with the use of the haptic device (9.25 for healthy subjects and 10.60 for patients). No statistical difference evidence, shown by the Pearson correlation coefficient, (p = 0.52 between in the healthy group and p = 0.09 in the SCI patient group) could be claimed in the proof of concept. However, we observed that six out of eight individuals in the healthy group and all individuals in the patient group made fewer mistakes when the haptic feedback was activated. These results provide good expectations for the pre-clinical study.

3.2. Path

In this game, the results showed no significant differences in ROM in either the X- or Y-axis. The slight difference in the Y-axis was attributed to the fact that the patients had difficulty in reaching the highest red circle. As for the previous game, the time needed by the patients to complete the exercise (54.40 s) was higher than the one needed by healthy subjects (38.61 s).

No important difference could be highlighted in the points gained by both groups because they just completed the path without passing more times on the same line. An interesting aspect was represented by the number of errors and the maximum value sent to the feedback device. Even if, as for the previous game, the average of errors was higher for the SCI patients, the healthy subjects had a maximum value higher than patients. The explanation consists of the fact that, guided by curiosity, all the healthy subjects wanted to try a high value of force provided by the thimble. This was observed during trials but could also be inferred from the fact that for the healthy patient there was almost no difference between the error made with and without feedback. This was because even if they made more errors without feedback, in the session with the device enabled they induced the feedback sensation by leaving the path, thus committing errors. The results were different for SCI patients. In this case as well, the number of errors made without thimble (11) was higher than the ones made with the thimble enabled (9.20). Figure 8 compares the results between the two groups for the experiments with and without haptic feedback. As mentioned before, on average, healthy subjects made fewer mistakes than SCI patients. However, there was no appreciable difference between the use or not of the haptic feedback (p = 0.93). As in the Maze example, there was no statistical difference in the SCI group (p = 0.51). However, we observed a reduction in the variability and an improvement in the average of the behavior of this group.

3.3. Grab

The time imposed to complete the task was one minute. During the performance, data completely different from those of the other two games were collected. In particular, the values of distance between index and thumb, the value sent to the haptic device, the stiffness value of the object grabbed and the time of playing were saved. If the distance between fingertips was less than the threshold level at about 6 cm, the object was grabbed and the proportional signal was sent to the thimble. As the distance between the fingers decreased, force feedback was greater. If the distance was above the threshold value, the object was not grasped. The value of the threshold was imposed considering that the user was wearing the thimble, so the parallel platform was placed between fingertips. If the value was lower, it would be possible to touch the parallel platform with the thumb even before grasping the virtual object, thus generating incorrect and incomprehensible sensory information. By imposing the value at 6 cm, there was enough space to perform the grasping movement without touching the platform with the thumb.

The biggest difference that could be noted between healthy subjects and patients was the number of times an object was taken. The trend of the distance value was also more stable and rhythmic in a healthy subject than in a patient. A significant element related to the performance of the users was the range of motion (ROM), intended as the difference between the maximum and the minimum distance between fingers reached. The average value of all patients was very similar to that of healthy subjects. The average number of points obtained by the patients was lower than that of the healthy subjects. From all this information it can be deduced that even if the movement carried out by the patients is correct, the time they need to complete the task is much higher.

4. Conclusions

In the context of the rehabilitation of the upper limbs of patients affected by spinal cord injury, VR technologies facilitate the performance of the necessary movement repetitions. By means of devices such as Leap Motion, it is possible to work on the motor aspects of the upper limbs, enabling tasks such as reaching, grasping, manipulating, and transporting virtual objects by executing certain hand movements and gestures. However, in those cases there is no sensitive information about the objects being manipulated. Therefore, VR needs to be coordinated with robotics devices. Unfortunately, these technologies are often expensive and cumbersome. Some devices are uncomfortable to wear, and some rehabilitation platforms do not provide any feedback to the affected limb. This often leads to an abandonment of rehabilitation platforms. For these reasons, they are not suitable for domestic use, which would allow the amount of therapy results to be increased further.

On the contrary, the work presented in this manuscript took advantage of low-cost VR systems and a custom-made haptic device to enhance the involvement and engagement of patients, provide a congruent afferent feedback during motor exercises, and benefit from the flexibility of virtual scenarios. Since this research was carried out in patients with cervical spinal cord injury, it is of special interest to integrating sensory functions because they are altered and/or modified after injury. In this way, through the haptic feedback delivered by the thimble developed in this manuscript together with the use of Leap Motion, the therapeutic objective in neurological patients is enriched. It is not possible to generate detailed sensory information about the size and texture of the virtual object but it is possible to recreate the sensation of holding an object in the hand. From a rehabilitation point of view, this is crucial for patients with neurological diseases.

Three games were designed and implemented for SCI patients, meeting the needed requirements for adequate rehabilitation. The user can interact with the games just by using their hand, thus focusing only on moving the arm and the hand in the correct way. To achieve this, the platform takes advantage of the Leap Motion device to represent the hand movements and access the data relative to the positions and velocity of the user’s index fingertip. The platform was developed to provide the user with the ability to receive haptic feedback that allows them to be more aware of the interaction with the virtual environment.

The work also focused on the development of an easily wearable device that can provide two different types of tactile feedback: vibration and pressure. Vibration was chosen because of its significant effect in neurorehabilitation. Several previous studies reported improvements in movements and a reduction in spasticity [36]. Pressure was selected as a direct association regarding the distance to the correct path and to feel the hardness of the materials [34]. The design and implementation of the mechanical parts, the connections, and the control were oriented to allow the patient to interact with the virtual environment without interfering with performance.

A proof of concept to check the functionality of the platform with both SCI patients and healthy subjects was conducted. The results obtained in terms of movement, error, and time showed some small differences between the groups. A statistical analysis with a larger sample must be carried out to clinically validate the system and to obtain statistical conclusions related to the degree of motor impairment in SCI patients. Nonetheless, the proof of concept provided glimpses into some benefits of the system, which are exposed hereafter:

• All the devices used to build the platform are low-cost and easily adaptable for different people, with a short preparation for the session.
• The configuration options used in this project allows different virtual scenarios and training situations personalized for different patients to be created.
• The final design of the thimble and of the electronic connections ensures good mobility and generates a good haptic response.
• The data storage makes the platform a useful tool for monitoring progress in rehabilitation.

For further research, extending the sample analyzed would be interesting to determine the possible effectiveness of this therapeutic modality.