Development and Studies of VR-Assisted Hand Therapy Using a Customized Biomechatronic 3D Printed Orthosis

Abstract

This article presents the process of development, testing, and use of wrist–hand orthosis in the hand therapy of a teen patient with congenital paresis disease. A regular 3D-printed anatomically adjusted orthosis is modified with a set of sensors, to work as motion and interaction controller in virtual reality (VR). As the patient with this condition cannot operate VR controllers due to wrist and hand defects, the corrective orthosis was converted to a VR controller, by introducing custom-made electronics and commercially available motion trackers, linking them to the orthosis. A VR game scenario, with typical input from the VR controllers replaced by input from the custom-made controllers is then designed. The VR game scenario is prepared with involvement of physiotherapists, to incorporate the most important exercises for patients with the same condition. The scenario is tested with a group of human patients and assessed by an expert physiotherapist, for determining its efficiency, as well as to determine a set of necessary improvements for future development of the orthosis.

1. Introduction

1.1. Biomechatronics

Biomechatronics is an interdisciplinary field that combines mechanical, electronic, and biological elements to create innovative devices and systems. It is a subfield of biomedical engineering that focuses on the interaction between mechanics and electronics with human biology [1]. Biomechatronic systems can interact with the human skeleton, muscle, and other biological tissues, such as the nervous system, to enhance or replace their functions. It is a key area of study in biomedical engineering. Examples of biomechatronic devices include prosthetics that can mimic the movements of natural limbs [2], exoskeletons that can help people with physical disabilities to walk or stand upright [3], and neuroprosthetic implants that can restore motor function to people with paralysis [4]. The components of a biomechatronic system typically include a human subject, which represents the biological aspect of the system, and a stimulus, which is the input delivered to the system [5]. Biomechatronics devices have plenty of medical applications, such as biointerfaces for diagnostics and control, passive and active prosthetic limbs and joints, sensing and biofeedback, rehabilitation systems [6], and many others. One of the key areas are wearable sensors and biofeedback systems, which allow users to monitor their physiological parameters and use that feedback to control various devices or systems, e.g., through EMG [7]. The concept of biofeedback and virtual reality systems are frequently connected together in the available literature for therapeutic uses [8,9].

Biomechatronics is not a new discipline and it has been widely explored by researchers throughout the world. For instance, a publication by Bencsik and Lendvay [10] discusses current and emerging trends in biomechatronics, exploring advancements in the fusion of mechanical engineering, electronics, and biology. The focus is on interdisciplinary developments shaping the field. Junaid et al., in their publication [11], concentrate on the design aspects and control mechanisms of an artificial robotic hand. It covers topics related to robotics, mechatronics, and control systems, exploring the functionalities and engineering behind creating such a system. Another review publication by Yang and Burdet [12] provides an overview of biomedical robotics and biomechatronics systems and their applications. It covers various applications and advancements in the intersection of robotics and biomedical systems. It is noteworthy and evident from the available literature that one of prominent focal points of biomechatronics is the fusion between robotics, prosthetics, and orthotics, especially in the scope of the upper human limbs.

1.2. 3D Printing Methods for Customized Orthotics

The evolution of 3D printing technologies has deeply impacted a myriad of sectors, with healthcare standing out prominently. In the realm of orthopedics, 3D printing has pioneered advancements, particularly in the domain of customized orthotics [13,14]. Current cutting-edge orthotics emphasize biocompatible polymers (according to the class I of devices by the European Union—materials and devices not placed inside of a human body, possibly in contact with the skin), such as thermoplastic polyurethane (TPU) and polylactic acid (PLA), due to their flexibility, durability, and lightweight nature [15]. There is also a surge in research towards composite materials, integrating nanomaterials or fibers, for enhanced mechanical properties [16]. The latest 3D-printing technologies, such as PolyJet, facilitate the simultaneous deposition of multiple materials. This allows for the creation of orthotics with varying degrees of rigidity in targeted regions, catering to patient-specific needs [17]. The structure design and facilities that CAD and 3D printing software programs are using nowadays allow the design and manufacture of intricate openwork structures. These not only reduce the weight of orthotics but also offer improved ventilation and can be tailored to provide differential support across the orthotic [18].

The integration of 3D scanners in the process of realizing the orthoses has revolutionized the customization process. Patients’ limbs can be digitally scanned, ensuring a precise and tailored fit. These scans are then converted into 3D models, which serve as the blueprint for the orthotic device, thus ensuring a high degree of personalization [19].

The convergence of 3D printing with wearable tech has opened avenues for orthotics equipped with sensors. These can monitor pressure distribution, gait analysis, and offer real-time feedback, potentially revolutionizing rehabilitation processes [20]. The synergy of 3D-printing technologies and orthotics production has brought forth unprecedented advancements in the healthcare sector. The focus has shifted from generic, one-size-fits-all solutions to highly individualized and optimized devices that prioritize patient comfort, functionality, and aesthetics [21]. As research in this domain continues to burgeon, it is anticipated that future innovations will further enhance the user experience and the therapeutic efficacy of orthotics.

Various studies show that customized 3D-printed orthoses are better for the course and results of the therapeutic process than the traditional ones. Customized hand orthoses offer tailored solutions, precisely fitting the patient’s hand, enhancing comfort, and potentially improving treatment outcomes [22]. 3D printing enables intricate designs and complex geometries, allowing for novel orthotic structures that might be challenging to create conventionally, if need arises for a given patient [23]. With 3D printing, a wide range of materials can be utilized, enabling orthoses to be more adaptable to patient needs and potentially improving durability [24]. Customized orthoses might also enhance patient compliance and satisfaction due to personalized designs tailored to their preferences and needs, which can be an important aspect deciding whether the therapy will be successfully pursued or not [25].

1.3. Virtual Reality Applications in Therapy

Virtual reality has plenty of medical applications, including educational applications for students [26,27], advanced haptic feedback training solutions [28], as well as all-purpose training simulators [29,30]. It can also be used to improve design and development of customized prosthetic devices [31], as well as in anxiety therapy [8].

One of the prominent VR applications in medicine is carrying out or supporting the rehabilitation process using virtual reality applications. This is a way to not only make exercise more attractive and thus motivate the user [32], but it is also possible to track the course of the session remotely and to easily store and save medical data that would be impossible to measure without digitizing the rehabilitation system. More and more applications supporting the rehabilitation of patients are available on the market. There are also many studies measuring the effectiveness of physiotherapy using VR. One such application is presented in [33], where a system consisting of a controller measuring hand movements, i.e., CyberGlove or Rutgers Master II-ND haptic glove, and an application in which the user was to perform several repetitive actions was tested on a group of four patients The purpose of the application was to restore hand function after a stroke, and more specifically to improve the range of motion in the joints of the upper limb, increase its speed, increase the strength of the finger muscles and independence of action. The Jebsen hand function test was also conducted, which patients with an initial higher degree of impairment completed more than 23% faster after training than initially.

Another application is the one presented in the study [34], where a more extensive program is presented that assumes mirror actions of the hand in which the right or left limb is actively exercising. The activities performed include hitting a ball, where, as part of the progression, adjustments can be made to its speed, the time interval for the next one to appear, or the width of the spacing on which the ball will fly. The results of the conducted research showed that patients who underwent treatment quickly accepted this form of treatment and learned to work with it just as quickly. This proves the potential of technology to support classical therapy in the rehabilitation process.

Another study that tested the effectiveness of the VR system, this time compared to CT (conventional training) [35]. For the duration of the study, 120 patients were assigned to two groups, of which the group testing virtual reality rehabilitation used a system consisting of the YouGrabber application, gloves with position sensors, infrared cameras, a computer, and a screen. The application contains several games that the therapist could match the patient’s mobility abilities. The results of the study showed that training with the help of VR was as effective as CT. In addition, in the case of virtual reality, there is a motivational aspect that can encourage patients to exercise.

To sum it up, it might be stated with certainty coming from multiple sources in the literature, that VR is in general a technology which has large potential to improve physical rehabilitation processes and the use of VR combined with physical exercises brings better results than using purely conventional therapy methods. However, most solutions are still experimental and customized therapies, tailored to specific patients, are rare and largely unavailable.

As is known from the available literature and medical practice, all types of rehabilitation processes in hand therapy require regularity, without which none of the known effective methods will achieve their full effectiveness, if they have any impact at all [36,37]. Many effective rehabilitation techniques are being developed that have a very positive physical effect on the human body, but an often unaddressed problem is motivation and the frequency of the exercises that the user performs. These two characteristics have a significant impact on the patient’s ultimate level of independence [38].

In the project developed by the authors of this paper and preliminarily presented in the further chapters, the plan is to remedy this problem by creating a system that is attractive to the user, while at the same time utilizing the strengths of other concepts. The system should also be able to control the user’s achieved ranges of motion and provide a sense of achievement and independence. An important aspect is the system’s cooperation with other rehabilitation devices such as orthoses.

A rehabilitation application taking the form of a game as a solution that meets the above objectives was therefore considered in the current article. Originally, it was intended to be developed for upper limb exercises. To ensure control of the user’s movement, it was planned to use a virtual reality helmet combined with sensors of the body’s position in space. To additionally gather more precise information about finger and wrist movements an additional was attached to the hand. Both this device and the previously mentioned trackers were considered in the process system to be built using a customized orthosis for a particular case study of one patient.

2. Materials and Methods

2.1. Research Main Concept and Plan

The main concept of the research presented in this paper is the use of a corrective customized wrist–hand orthosis (WHO) in connection with a special, dedicated virtual reality application in order to improve hand therapy in persons with disabilities (resulting from injuries or long-term diseases such as congenital paresis or cerebral palsy). As such persons, due to defects and deficits in hand functioning, are often unable to use VR controllers properly; a concept that is presented in this paper was to convert a mechanical orthosis into an electronic device with motion and interaction sensors, effectively replacing standard VR controllers. As a result, a prototype VR system was created, consisting of a therapeutic orthosis, experimental electronic game controller, and a VR application, to aid in the performance of rehabilitation exercises.

After analyzing the current state of the art concerning the rehabilitation requirements, several general requirements were set that the system must meet. Initially, the basic requirements for each application must be met:

• Intuitiveness,
• Clear and easy interface (without much text),
• Quick loading time,
• Encouragement for the user to engage appropriately,
• Carry high-quality mechanics, graphics, and sounds, to be compelling mostly to young people (who are main target of the system).

The application should include movements that will imitate functional movements performed in everyday activities, such as grasping and throwing objects, hitting, cutting etc. The application should also give a sense of purpose, having a specific scenario and style, to be more engaging for the users.

The requirements regarding the experimental orthosis-based controller were defined as follows:

• Ensuring accurate and up-to-date recording of hand movements, including independent finger movements simulating grasping,
• Possibility of simultaneous cooperation with another state-of-the-art movement registration system,
• Ensuring the mobility of the device and user,
• Cooperation of the device with the selected VR system,
• Low weight and ergonomic use,
• Independent power source,
• Basic aesthetics,
• External housing, with the option of attaching to the orthosis or switching between orthoses for various hand sizes,
• Low-cost build with available, cheap sensors.

Another element of the system is the orthosis, or rather the design of the parts for attaching the controller, which should be strong and at the same time not hinder the movements performed in everyday life without electronics installed.

Based on these requirements, the stages of development were set. These are shown in the schematic presented in Figure 1.

2.2. AutoMedPrint System

For materializing the concept of the customized wrist orthosis, the AutoMedPrint system was used. The AutoMedPrint system is a system for rapid, automated design of selected, 3D-printable orthopedic and prosthetic supplies. It is mostly focused on producing wrist–hand orthoses [39] and upper limb prostheses [40]. A prototype system was built with a mechanized 3D scanning station, containing automated algorithms of scan processing and data extraction, as well as intelligent CAD models of orthopedic products. The system has gathered many awards, including being named the Polish Product of the Future in 2022.

Figure 2 presents the prototype of AutoMedPrint, while in Figure 3, the methodology for working with the system is given. Virtual reality technology was also used extensively in the development of the system and the products contained within it [41].

2.3. Preliminary Studies—Patient Case and Customized Orthosis Description

The patient whose case was the basic inspiration for the research presented in this paper was a 13-year-old boy with congenital paresis that requires special therapy treatment. Congenital paresis refers to partial or complete muscle weakness that is present at birth. It arises due to developmental anomalies or injuries to the nervous system during fetal growth. This condition can affect any part of the body, limiting movement and function in the affected area. Early diagnosis and intervention are crucial to enhance mobility and improve the quality of life for individuals with congenital paresis.

The patient exhibits restricted mobility in various joints of the upper limb. In the shoulder joint, there is limited extension, abduction, and external rotation, with movement restricted to approximately 80 degrees for extension and abduction, and approximately 45 degrees for external rotation. Within the elbow joint, there is minimal flexion movement. At the distal radioulnar joint, the hand is positioned in pronation, with constrained suppositional movement limited to around 40 degrees and the absence of radial or ulnar movement. Furthermore, in the ulnar position of the wrist joint, there is an absence of active extension of the distal radiocarpal joint, resulting in limited hand movement. The combined effects of these limitations may significantly impact the patient’s ability to perform various daily activities and require tailored rehabilitation strategies to address their specific impairments and restore functional mobility.

The main concept of the therapeutic orthosis in this case was based on 3D anatomical scans of both arms of the patient. Both limbs were scanned; the healthy left arm was scanned in an anatomically correct position at the AutoMedPrint scanning station using David SLS-3 scanner (HP Inc., Palo Alto, CA, USA) (Figure 4a), while the right arm was scanned manually, using EinScan Pro 3D scanner (Shining 3D, Hangzhou, China), with a wrist position comfortable to the patient (Figure 4b). Both scanned meshes were split at the wrist; healthy left hand was digitally “transplanted” (and mirrored) to the right forearm, and properly scaled using main dimensions from the scan of the right hand.

On the resulting hand model, data extraction procedure was performed, using a set of automated MeshLab and MS Excel macros present in the AutoMedPrint system. Design table was generated in form of MS Excel sheet, containing point data extracted at 11 planes of the arm, 16 points at each plane. Also, other settings were applied, with offset between arm and orthosis set at 3 mm (to compensate for the lining) and orthosis thickness left at default 4 mm to maintain its strength and durability. Moreover, additional cutouts were added in the CAD model of the orthosis for the patient to be able to perform daily activities and therapeutic exercises.

The designed orthosis was manufactured using FlashForge Creator Pro 3D printers working in the FFF (FDM) technology. The orthosis was 3D printed using standard methodology in AutoMedPrint system [39], with layer thickness of 0.25 mm and 30% infill. Using PLA material, it took approximately 4 h per one part. Full post processing was applied, in form of overall grinding and the internal parts of the orthosis were lined up with foam. Such an orthosis was tried by patient (in several iterations) and then used for rehabilitation for 9 months. After 9 months of continued use (1–2 h daily), slight improvement was observed in motion functions, as well as muscle development and wrist position. At that time, the orthosis was no longer usable due to growth. The patient was 3D scanned again and another orthosis was prepared. Two orthoses made and implemented for use by the patient are shown in Figure 5a,b. The whole patient case was a subject of previous studies by the authors and described in previous publications [42].

The orthosis use was largely successful, allowing the patient to improve in functioning. After gathering feedback from the patient and working with another number of patients with similar conditions, it was decided to try making the therapy more compelling and interesting, by introducing gamification in virtual reality, as stated in the research concept. However, to make it easier for building and testing the prototype of the system, it was decided to make another orthosis for one of the creators of the system. The whole customized design procedure was repeated for one of the authors, and a new orthosis was designed and printed, as shown in Figure 6, using the same printing parameters, materials, and devices. This orthosis was subjected to further development of electronics and VR application, described in further chapters.

2.4. Design and Prototyping of a Motion Controller

After the 3D-printed orthosis was obtained, the work was divided into four main parts that were meant to create an entire system designed to support rehabilitation process. The first issue was to form an application pad that is able to track hand movement without excluding the motion of the fingers. The device was supposed to allow easy control of the gameplay, without the need to hold anything in user’s hand.

It was decided as follows:

• The controllers replace standard VR controllers for both hands—the healthy one and the one requiring therapy.
• The controllers should be easily attached and detached to/from the orthosis and user’s fingers.
• The controllers must detect rotational hand movement, as well as movements of two fingers (thumb and index).
• A button must be present in the palm of the hand to allow detection of grasping gestures.
• Another device should be used for pointing/teleportation inside the application.
• Hand positional tracking should be realized in compatibility with used VR system (e.g., HTC Vive, which was selected to be the main platform).

Taking these assumptions into account, electronic components were selected and planned. Figure 7 shows the main schematic of the controller. It was decided to use the ESP32 chipset and a custom PCB board, as well as a number of components marked in Figure 7, to fulfill the requirements set in the initial phase of the research.

For easy mounting of the controller at users’ hands, it was decided to build an external housing, easily attachable and detachable from the orthosis main shell. The housing, due to increased demand for accurate dimensions, was manufactured using PolyJet technology. Stratasys MediJet J5 (Stratasys Co., Waltham, MA, USA) machine was used with a standard DraftWhite material (UV resin), with layer thickness of 18 µm. Design of this housing and its manufacturing is shown in Figure 8.

The development of the controller resulted in a device capable of measuring the angular position of the hand, thanks to a module with a gyroscope and an accelerometer mounted on the outside of the limb. In addition, the flexion of the index finger and thumb is recorded thanks to flexible ties that move the joysticks. A selection button is located inside the hand. In addition, there is another analog resistive joystick on the thumb (giving the impression of gliding, and, due to its design, limiting the height of the element), which is used to activate the indicator and teleporting around the VR scene.

2.5. Conceiving the VR-Assisted Hand Therapy Application

The next stage was to design the VR application for therapeutic purpose itself. To do this, it was necessary to determine the type of game and create a suitable scenario in VR for it. The type chosen was specified as an escape room. For this, it was necessary to consult a specialist (physiotherapist) to choose the appropriate exercises and design their implementation in the game. The created scenario for the game assumed the making of four rooms, in which the user would focus on movements within a specific joint—successively, the shoulder, elbow, wrist, and finger joints (an idea diagram of the initial space can be seen in Figure 9). Even before exercising, a warm-up of the whole body would be performed by a simple game forcing the user to avoid flying obstacles by leaning, crouching and jumping.

At this stage of research, it was decided to create four rooms, each for exercise pertaining to different joints in the upper limb (shoulder, elbow, wrist, fingers). According to the expert physiotherapist opinion, there should be a fifth room with compound exercises. It was decided that this room will be added once the initial concept is verified positively in practice.

The next issue consisted in the choice of virtual reality device and the type of motion tracking system. The assumption here was to use the HTC Vive Pro helmet (HTC Corp., Taipei, Taiwan) and the body position sensors dedicated to it (Vive Tracker 3.0 devices). The next step was to determine the program in which the application will be carried out, which in the current case is the Unity environment, version 2021.3 (Unity Technologies Inc., Austin, TX, USA). Once the scenes are known, it was necessary to find or make the appropriate models of the virtual world, as well as soundtracks or visual effects. The programming part itself was divided into certain sub-steps. The first was to provide the necessary mechanics such as grasping, moving, interaction system, as well as the approximation of the avatar’s body movements based on sensor data. The next tasks were to realize the scenario, or more precisely to provide the basic functionality of each scene, which is individual and different for each case. The created elements was intended to be connected in such a way as to preserve the cause-and-effect sequence, as well as add instructions (in the form of icons, text, and sound), a tutorial and a menu with a language option (Polish and English options are assumed). In the programming, Unity’s own XR toolkit was used, connected with Vive Input Utility (Vive Software, version 1.18.3) and SteamVR SDK (Valve Corporation, version 2.7.3) plugin tools. Mostly free assets were used in building the game, including user’s avatar, objects, textures, sounds, etc.

The fourth stage of work was aimed at intersecting with the other three, and it concerns connecting the entire system and ensuring communication between its components. This requires determining the type of signal to be sent, writing appropriate communication scripts from the application side as well. It was decided to choose UDP protocol for communication between controller and the PC with VR application. Here it was intended to also include the creation of mechanical connections between the orthosis and the housing of the other devices, as well as the adjustable attachment of the controller to the hand without the orthosis and the power supply.

2.6. Testing Methodology

At the initial stage of validation of the rehabilitation methodology proposed with the use of customized mechatronic orthosis and VR application, it was decided to work only with healthy subjects—work with the patients will be realized after concluding the initial tests. As such, a group of 12 persons was selected to take part in the tests. For practical purposes, it was decided that the subjects will use the same orthosis as manufactured for one of the authors. As such, the group consisted of 12 females, in the age group between 20 and 40 (with the exception of two subjects), with hand and forearm anatomical features allowing them to use the orthosis comfortably, which was checked before actual tests occurred. In the case of any discomfort caused by the orthosis not fitting a given user, temporary lining with foam or other soft material was applied to problematic areas, to allow the tests to be realized properly. Users were also informed that the orthosis in the final product would be customized especially for the hand of a given patient and as such they were asked to not report bad fitting as a problem.

The users were tested under laboratory conditions, using HTC Vive Pro headset connected via cable to a PC with the application. The customized controllers were used on both hands, with the right hand equipped in the 3D-printed orthosis and the left hand equipped with the prototype controller itself (auxiliary, to help realize activities requiring two hands). A user during the tests is presented in Figure 10.

All the users were initially asked to familiarize themselves with the VR goggles (if they never used them before), orthosis, controller and it means of operation, as well as basic VR environment. The application contains an in-built tutorial, which was completed by all the participants prior to the actual exercises. Then, participants were asked to go through the full game, completing all the exercises in the way they saw adequate, with slight guidance from the researchers performing the experiment. Approximate completion time was also registered.

After the test subjects finished working with the application, they completed a questionnaire with a number of questions to evaluate the application. The questionnaire was divided into three main parts:

• Part 1—evaluation of VR application and exercises, 8 questions, Likert scale (1–5),
• Part 2—evaluation of custom-made game controllers, 6 questions, Likert scale (1–5),
• Part 3—general assessment (1–5 scale), comments, observations, problems (open questions).

Full contents of the questionnaire are attached at the end of the manuscript (Appendix A).

Apart from that, the application was tested and reviewed by a physiotherapist—expert with a degree in game therapy of upper limbs. The expert was also able to test the solution (thanks to similar anatomy) and apart from filling in the questionnaire, gave a number of opinions and feedback towards further development of the application and the whole solution.

3. Results

3.1. Orthosis and Controller Manufacturing and Assembly

The mechanical part of the design was realized without disturbances, using already established capabilities of the AutoMedPrint system, known from previous studies and implementations. The orthosis was successfully manufactured out of PLA material in a total time of approximately 10 h (approximately 5 h per part). Other mechanical parts—the housing of the controller and its cover—were also manufactured without any errors and successfully assembled.

For proper assembly of the controller to the user’s upper limb, additional straps and pockets were added, to secure the orthosis and controller in place, as well as provide the possibility of mounting the powerbank.

The electronic part was also manufactured without significant problems. Several iterations were required before all the controller components started to work properly. Considerable problems were encountered in the operation of the wireless module (losing connection), inertial measurement unit (frequent drifts in recorded rotation values), and small touch joystick (improper functioning). However, these problems were solved in several attempts and they did not affect the performance of the controller in tests or the operation of the VR application. The fully assembled orthosis with a controller is shown in Figure 11.

3.2. VR Application Building Results

The created application ultimately consists of several levels. At the very beginning, a tutorial is available, which the user can skip. Then the user moves to the room (Figure 12), where the main part of the game begins. There he is informed about the purpose of the game by a virtual character (named “Damini” in the game) and can move to the first round, which begins with a short warm-up of the whole body by avoiding obstacles approaching the user. Then the patient’s task is to pull down the ropes located in the house in front of him (Figure 13). Points are counted separately for the left and right hands, and the final score is their average.

The elbow joint exercise stage begins in the dungeon (Figure 14). As in the previous stage, the prompts for completing the round are located on the walls. Here the user, making a move similar to throwing a bowling ball, must knock down enemies approaching him. Points are counted analogously to shoulder exercises.

Wrist exercises are performed in a small garden (Figure 15). In this case, the patient’s task is to pick fruit from bushes and trees by rotating their wrists within them, and then throw them into a box. Both fruit picking and successful hits are scored. Finally, a burning table appears, whose flames must be extinguished with a flick of the wrist.

The last of the rounds—finger exercises—takes place in a museum (Figure 16). Forty-one chaotically moving candles appear in front of the user, which must be caught and crushed. The last candle breaks a glass with hammers, which must be used to break a platter on the other side of the room.

3.3. User Test Results

Almost all the users were able to complete the application and realize all the activities, to some extent. One of the 12 users was not feeling well before the test and needed to cease it after the second room, but was however able to complete the survey and evaluate the application. The controllers (on both hands) operated properly, and the users were able to quickly learn about their operation, to properly realize the exercises contained in the specific rooms.

The results of completing the exercises (time in minutes), together with final evaluation of the complete VR system by its test users is presented in Figure 17.

Detailed results of the survey are presented in

Table 1. Survey results—part 1—VR application evaluation.

ID	Question	Avg.	Std. Dev.
VR1	I feel my muscles warming up.	4.08	0.51
VR2	I feel muscle fatigue.	2.42	1.08
VR3	The application is more interesting than classic training.	4.67	0.65
VR4	I feel that I would exercise more often if I had constant access to the presented system.	4.33	0.98
VR5	Using the application was pleasant and simple.	3.83	1.03
VR6	The exercises were engaging but possible to perform correctly.	4.25	0.62
VR7	The level of graphic quality was sufficient.	4.75	0.45
VR8	Interacting with objects was intuitive and easy to learn.	3.50	1.00

and

Table 2. Survey results—part 2—controller evaluation.

ID	Question	Average	Deviation
C1	The controller is comfortable.	3.42	0.90
C2	I like this controller solution.	4.08	1.00
C3	I believe this controller is better for this specific use than the classic one.	4.17	0.72
C4	The response speed to signals from the controller was sufficient.	4.00	0.74
C5	Using the controller was pleasant and simple.	3.50	1.24
C6	Interacting with objects using the controller was intuitive and easy to learn.	3.75	1.06
	Total average	3.82

, as well as in diagrams in Figure 18 and Figure 19.

The main observations of the testing results can be summarized as below:

• The average evaluation of the whole VR system by all test users is 4.45 in 5-point scale. Most assessments were 4 or 5, confirmed by positive opinions expressed by test participants, regardless of age and background. This means that the system was generally met with a favorable reception.
• The set of exercises were finished on average in a little above 20 min, some persons taking 16 min, and the longest time being 30 min. Including the tutorial and setting up the system, whole sets of exercises can be set to take one full hour of a physiotherapeutic session, which is acceptable from the point of view of hand therapy practice.
• All users agreed that this form of training is more interesting than the classical one and that they would be more frequently engaged in the training if it had that form.
• Graphics quality was assessed as acceptable or high, there was no problem with this aspect, although some users noted minor glitches, especially in behavior of the virtual avatar (deformation of hand and wrist during some movements).
• Interaction was generally evaluated positively, although most users had some minor problems with getting used to interacting with objects, mostly due to the steep learning curve of becoming familiar with the designed game controller.
• Most users observed their muscles to be warmed up during the exercises, but rarely fatigued, which is understandable considering the total length of the game.
• In the survey part related to the design of a custom orthosis-based controller, the evaluation was generally positive, but there are problem indicators. The controller was not very comfortable for all persons, and also not always intuitive and easy to use. However, most users liked it and of all questions, the highest answer and certainty was that it is better for the rehabilitation game than a typical VR controller.

The following observations were made by the researchers while supervising the tests:

• Test subjects of younger age tended to adapt to the controller and the game mechanics more rapidly (in general).
• Most users has problems with understanding pictures with instructions contained in the rooms and they had to obtain spoken hints on what to do in a given exercise.
• Limited times for certain exercises did not allow some subjects to complete them.
• The assumed course of realizing the exercises was not clearly followed by the users and as such the game would require plenty of improvements, to “forbid” certain behaviors (such as dragging the rope with both hands at once).
• There was a problem with IMU in the controller—it frequently noted a rotational drift, which accumulated over the course of the game. It vanished after the device was reset, but it was clearly disturbing the work.
• Teleporting around the scene was initially problematic for most test subjects.
• The thumb joystick also was not fully intuitive and required plenty of tries from most subjects to operate properly.
• Wired VR goggles hinder the movements of test subjects, especially during warmup. A wireless solution or special cable harnesses should be used in future development.

4. Discussion

The realized customized VR system, using a custom-built game controller based on a biomechatronic orthosis and motion tracking, was found to be a working solution in the initial tests. The greatest problems were with the controller interaction—test subjects slightly complained about its low intuitiveness and discomfort of use along with the orthosis. This is clearly understandable, as the orthosis was not one-size-fits-all, but rather a customized solution, tailored for a single person. This is a clear limitation of the study, of which the authors were aware since the beginning, but the tests needed to be performed as early as possible to test if the concept is viable. As it was found to be working, testing methods will be improved in further iterations, building different orthoses for different subjects (both healthy and with certain hand conditions) and testing them again in the improved application.

It is important to note that due to the customized nature of the game controller built and programmed especially for the purpose of the game, standard VR controllers cannot be used.

There were also remarks about the application itself. Its graphics level was unequivocally evaluated as high and sufficient for the purpose. However, colors and style were not praised by everyone taking part in the tests. This is a matter of taste, but as motivation is an important factor in the efficiency of physical rehabilitation, in future studies and development, various versions of the application might be created, with different color themes and styles for different age and gender groups of potential patients.

As for now, the most important improvement would be to make the instructions clearer and allow for a more easygoing approach to the exercises. Some subjects, including the physiotherapist, complained about the time limit for some exercises as too low, not allowing them to fully complete the tasks—for other participants the difficulty level was just right and they assessed it positively. Here, an improvement could be to introduce various modes of operation (i.e., game difficulty levels), with an initial, “learning” mode without any timer and with the possibility of free restarting for each exercise. The user avatar could also be configurable, to adjust it to patient preferences, which could further increase motivation for rehabilitation exercises.

The expert physiotherapist noted several things to improve while assessing the application. First of all, the application contains exercises with wrist movements that are not fully possible to realize in the 3D-printed orthosis, as it reduces mobility in that joint. These exercises should be improved to not include these motions. Moreover, there were some problems with the player avatar, showing a deformed hand and wrist after making certain movements, which could potentially affect patients psychologically. The aspect of the warmup and activization of the whole body was assessed positively, and also the requirement to perform all exercises while standing (not sitting). To sum up, the physiotherapist noted that the game can be accepted for use with real patients with congenital paresis and cerebral palsy.

As such, it is important to state that during the tests and evaluation of the application it was observed that this method of therapy should be a supportive one to the traditional methods. In other words, VR with customized orthotics has a potential to greatly complement the existing therapeutic methods, not replace them, at least at a current (generally understood) level of development. The novelty of the solution comes with the integration of two concepts, previously not joined in the known literature—a corrective orthosis and a VR game. Separately, these concepts are quite well known, but together it is still a novel idea, enabling certain patients to operate VR applications on a level they were previously not able to—cerebral palsy or Erb’s palsy sufferers sometimes cannot even grasp traditional VR controllers, let alone utilize them in a game-like experience. Use of a customized orthosis that corrects the wrist joint position and enables patients to operate the VR application enables new possibilities in therapy not possible with regular VR. An alternative approach could be AI-aided hand and finger tracking (such as that available in mixed reality devices such as HoloLens 2), which could be an approach explored simultaneously, either with or without use of a customized orthosis.

After summarizing the test phase, it became clear that another set of experiments is needed to determine the full scope of the usefulness and advantages of the proposed therapeutic method. An experiment with a test group and a control group of real patients with disabilities is required and will be subsequently planned and realized in the further studies by the authors to answer if the proposed method is superior to the currently used hand therapy methods.

5. Conclusions

Regardless of the rehabilitation method used, one of the more important issues for its success is the patient’s commitment and regularity of exercise or participation in treatments. For this reason, a rehabilitation system in the form of a game was proposed to increase the user’s motivation to exercise. The application was designed for upper limb exercises and would use virtual reality combined with sensors that track movement and a dedicated sensor that would record more accurate hand positions. As a result of the work, a bilingual escape room-type application was created, containing four rooms, and in each of them the user must perform exercises focused on the movement of a specific joint. The game and the whole system was evaluated as a viable idea, both by a number of test participants and an expert physiotherapist and it was accepted for further development and use with real patients.

The first stage of further development will include working with the original patient. Then, the tests will continue with other patients, along with improving both electronic parts and the software part of the system. Further development of the application is aimed at the possible creation of additional practice rounds, which would be randomly assigned to the user. In addition, it is planned to expand the menu and allow the user to rejoin unsuccessful approaches. An additional idea is to create non-isolated exercises as an end game. All these will be explored in further work. Also, problems with existing controller prototypes will be solved by testing other classes of components for angular position tracking and communication, to prevent rotational drift and other communication problems.

What is also important to mention is that VR is just one of three XR technologies, with the other ones being mixed and augmented reality. The authors plan to study the use of the other technologies in hand therapy as well, especially utilizing hand and finger tracking of devices such as holographic goggles (HoloLens and similar) and compare it with the immersive VR approach—probably all XR technologies could be useful in hand therapy, but to what degree should be answered in carefully planned experimental studies.

In the authors’ opinion, the form of customized, tailored therapy using dedicated biomechatronic orthotics and VR games is a therapy of the future, which could potentially significantly improve current standard therapeutic methods in certain cases. For that to happen, a number of further studies must be realized, to which the authors will contribute in the near future.

6. Patents

The orthosis–VR game controller concept, design and operation is subject to a patent application, submitted to the Polish Patent Office.