Immersive VR (Virtual Reality) Simulator for Vein Blood Sampling

Abstract

Vein blood sampling is a method of mass blood sampling that involves drawing blood from a vein for blood type discrimination, confirmation of various physiological indicators, disease diagnosis, etc.; it is the most commonly used blood sampling method. An important aspect of vein blood sampling is the search for the exact location of the vein for insertion of the syringe to draw blood. This is influenced by obesity as well as skin and blood vessel conditions in the patient and the experience of the clinical technologist, nurse, and resident who performs the blood sampling. Frequent practice is required to effectively perform blood sampling techniques. However, due to the many limitations of the practice room or laboratory, there is a problem of using only a limited environment and model for clinical practice. As a result, many medical educational institutions have situations in which only fragmentary clinical practices are performed, and it is difficult to practice many blood sampling skills, so they do not provide enough experience to understand the actual skill field. In this paper, we propose a virtual-reality-based vein blood sampling simulator that allows the practice of blood sampling techniques without limitation. The proposed vein blood sampling simulator can operate a 3D model related to vein blood sampling using an HMD controller and a haptic device in a virtual space for vein blood sampling practice by wearing an HMD (head-mounted display). Vein blood sampling can also be practiced through interaction with the patient 3D model. In addition, the effectiveness of a simulator developed for dental students was verified, and as a result of the verification, the potential of the proposed vein blood sampling simulator was confirmed.

1. Introduction

Clinical practice is one of the most essential parts of medical education. Clinical practice plays a very important role in nurturing high-quality medical personnel by learning through practice how the theoretical contents of the medical education course are used in the actual clinical field [1,2]. For this reason, equipment for clinical practice is being developed and advanced. However, although various pieces of equipment and materials are used for clinical practice, there is a problem with using only a limited environment and model for clinical practice due to the many limitations of the practice room and laboratory. Accordingly, many medical educational institutions perform only fragmentary clinical practice, and experience to understand the actual clinical field is not sufficiently provided. This situation in which the provision of experience is not sufficient makes it difficult for trainees to understand the anatomical features and may cause problems in clinical practice [1,3,4]. To solve the problem of clinical practice, medical educational institutions or trainees use various methods. Typically, mutual practice is performed between the patient and the trainee or between trainees, or a high-quality anatomical model is separately prepared and used for practice. However, the method of conducting mutual practice between the patient and the trainee or between trainees has disadvantages such as ethical or stability problems and difficulty in securing patients suitable for clinical practice, and the method of performing practice through a high-quality anatomical model requires a lot of time and money to search and purchase a high-quality model suitable for clinical practice, and it is difficult to respond to unexpected situations that occur in actual clinical practice. To solve these shortcomings, simulators using virtual reality (VR) or augmented reality (AR) technology are being used for clinical practice in many medical education fields [5,6]. At Northampton University, in order to improve nursing students’ ability to respond to clinical situations, a virtual reality simulation product that integrates four HMD (head-mounted display) devices and a large screen was developed. Through this, nursing students can experience various clinical scenarios in a virtual environment and can learn and verify clinical practice and communication skills [7]. Oxford University developed the OMS (Oxford Medical Simulation) to improve the clinical practice of nurses, doctor and medical students working at John Radcliffe Hospital. Through this simulation, clinical practice in nursing education, medical emergencies, pediatrics, internal medicine, mental health care, and physical examination can be learned and verified [7,8]. Falah et al. [9] developed a virtual-reality-based cardiac anatomy education system using stereo 3D glasses to solve the problems of existing visualization-based anatomy education methods and confirmed its usefulness through a group of medical experts. Schild et al. [10] developed an emergency response training simulation for anaphylactic shock, and improved the training efficiency by adding a module so that two trainees and one educator can access the simulation at the same time. Pulijala et al. [11] developed a virtual reality education tool for loft type 1 fracture using HMD and Leap Motion (Ultraleap, San Francisco, CA, USA) [12] and verified the usefulness of the virtual reality education tool through a surgeon to confirm the potential of VR surgery for orthodontic surgery training. Ropelato et al. [13] produced an augmented-reality-based microsurgery simulator that works with AR glasses for micromanipulation training in ophthalmic surgery and verified the effectiveness of the simulator with 50 trainees. As a result of the verification, it was confirmed that the training method through the simulator improved the fine manipulation ability compared to the existing training method. Rhienmora et al. [14] developed a dental practice simulator using a haptic device and an augmented reality algorithm AR Toolkit [15], and several educators and experts identified the advantages and disadvantages of the technical practice simulator to confirm the potential of augmented reality technology in the field of dental practice. Si et al. [16] enhanced CT-data-based brain 3D models to 3D-printed skull models, developed an educational simulator that can train brain tumor resection through Hololens [17] and two haptic devices, and confirmed the usefulness and potential of augmented reality technology in neurosurgery training. Many medical educational institutions and hospitals are developing and utilizing virtual-reality- or augmented-reality-based clinical practice tools for training in various medical fields.

Aligned with this research direction, we would like to develop a virtual-reality-based vein blood sampling practice tool that can be used to practice vein blood sampling related to blood tests used in various medical fields. The reason for developing a vein blood sampling training tool using virtual reality technology instead of augmented reality is due to hardware constraints of augmented reality devices, reduced learning concentration caused by the surrounding environment, and decreased immersion, efficiency, and accuracy of tracking sensitive to the surrounding environment. Vein blood sampling is the most commonly used method of blood sampling, and as one of the methods of mass blood sampling, it is the act of extracting blood from veins. It is used to distinguish blood types, check various physiological indicators, and diagnose diseases. The important point in vein blood sampling is to find the exact location of the vein and insert the syringe. Incorrect vein blood sampling methods cause problems such as platelet destruction, failure to control the amount of blood sampling, damage to blood vessels, and reduced reliability of blood tests. To avoid incorrect blood sampling, it is necessary to be familiar with appropriate vein search and syringe insertion methods according to the blood sampler’s condition (obesity, skin and blood vessel conditions, etc.). This ability varies depending on the experience of the medical personnel (clinical pathologist, nurse, resident) performing vein blood sampling. Therefore, practicing vein blood sampling frequently is highly important. However, as with other medical practices, repetitive vein blood sampling practice is difficult due to issues such as space constraints, ethical or safety issues, and the high cost associated with the practice. Therefore, incorporating virtual reality or augmented reality technology to enable smooth repetitive vein blood sampling practice is needed. However, studies on the development of simulators based on virtual reality or augmented reality related to vein blood sampling practice are still incomplete.

Therefore, in this paper, we develop an immersive vein blood sampling VR simulator using virtual reality technology for smooth vein blood sampling practice. The immersive vein blood sampling VR simulator allows users to experience a virtual vein blood sampling space through a head-mounted display (HMD) and perform the vein blood sampling process using interactive equipment. Essentially, the VR simulator can be used on a PC, and it can also be utilized on HMD devices or smart mobile devices using limited interactive devices (excluding haptic devices). In detail, for the development of an immersive VR simulator for vein blood sampling, a virtual vein blood sampling space is constructed by creating environmental 3D models, interactive 3D models that respond to interactive device, and vein blood sampling 3D models related to vein blood sampling tools. Then, interaction methods are developed to perform the vein blood sampling process through the manipulation of interactive 3D models using interactive devices. Finally, we would like to evaluate the utility of the simulator by performing a usefulness test on the vein blood sampling simulator implemented for educators in medical education institutions. Figure 1 shows a schematic diagram of the development of an immersive simulator for vein blood sampling practice.

2. Materials and Methods

2.1. Production of Virtual Space for Vein Blood Sampling

In order to create a simulator for learning vein blood sampling, information on the actual environment where vein blood sampling is performed was collected, and based on this, a 3D model of the space where vein blood sampling was performed, related tools, and a patient model was directly created using 3D CAD software (3Ds Max 2022.3 and Cinema4D R23.110). Three-dimensional models include geometry, color, and material information, and were saved in a format (DXF, OBK, FBX, etc.) usable in Unity (Unity, San Francisco, CA, USA), a piece of virtual environment production software. Figure 2 shows the configured virtual space for vein blood sampling learning.

The 3D models constituting the virtual space are classified into three types: an environment model expressing the space where vein blood sampling is performed, a 3D model that interacts with the user’s (educator’s) HMD controller(Oculus Quest 2, Meta Platforms, Inc., Menlo Park, CA, USA) or haptic device(Touch, 3D systems, Rock Hill, SC, USA) manipulation, and a 3D model of a vacuum tube containing blood for vein sampling.

Table 1. Classification of 3D models according to utilization plan.

Environment Model	Interaction Model	Blood Sampling Model
Ward	data3D hand model	Vacuum tube
Medical bed	Syringe	Tray
3D patient model	Tourniquet
	Alcohol swab

shows the classification of 3D models according to the utilization plan.

Among the models classified under the environmental model, in the case of ward and medical beds, they were created to closely resemble actual environmental information to enhance user immersion. Rigid body properties were applied to prevent overlapping or passing through other 3D models. In the case of the patient 3D model, the left arm was exposed to allow vein blood sampling to be performed on the left arm. Additionally, clothing 3D models were created to attach to the patient 3D model, reducing rendering resources for body parts other than the left arm. Figure 3 shows 3D models included in the environment model.

In addition, to enhance realism during vein blood sampling and improve vein search skills, an anatomical 3D model of the patient’s left arm was created. To achieve this, we investigated the bone, muscle, artery, and vein structures within the parts of the left arm (upper arm, forearm, wrist, hand). We also obtained CT data from various real patients. Based on the CT data, we created 3D volume models, classified them by structure, and compiled three-dimensional geometric information for each structure. Subsequently, based on the compiled information, we created 3D models for each structure within the parts of the left arm. To enable interaction through interactive devices, collision information was added to each structure’s 3D model. Figure 4 shows the left arm anatomy 3D model.

Among the models classified under the interaction model, the 3D model of the hand is integrated with the user’s controller or haptic device movements. The left hand is utilized for the manipulation of the UI and the left arm of the 3D patient model, including the selection of vacuum tube, while the right hand is used for the operation of tourniquets, alcohol swabs, and syringes for vein blood sampling. For smooth selection and operation, collision (collider) information was added to the 3D hand model, tourniquet, alcohol swab, and syringe models. The interaction methods for interaction devices and the interaction 3D models are explained in Section 2.2. Figure 5 indicates 3D models included in the interaction model.

In the case of hand 3D models, hand motion animation was produced to express natural hand shapes when interacting with tourniquets, alcohol swab, syringes, etc. To represent hand gestures, we utilized software that allows the creation and editing of human motion animations, as well as the depth-sensing device Leap Motion, which can recognize a user’s hand movements. Using the hand skeleton data obtained through Leap Motion, we acquired keyframe data for significant hand gestures. Subsequently, keyframe interpolation was applied to generate hand gesture animations. Figure 6 shows object utilization animation using Leap Motion (Ultraleap, San Francisco, CA, USA).

Among the models classified under the blood sampling model, in the case of the vacuum tube model, we created six vacuum tubes (Solium, SST, Plain, Heparin, PST, EDTA) corresponding to each vein blood sampling order and adjusted them to the appropriate volumes (2 mL, 3 mL, 4.5 mL, 6 mL, 10 mL). They were positioned on the tray 3D model after adjusting their locations. To prevent overlapping or passing between the vacuum tube 3D model and the tray 3D model, rigid body properties were added to each model, and collision information was added to the vacuum tube 3D model for vacuum tube selection through the 3D left hand model. Figure 7 shows the 3D models included in the blood sampling model.

2.2. Implementation of Vein Blood Sampling Interaction

HMD controllers (Oculus Quest 2, Meta Platforms, Inc., Menlo Park, CA, USA) and haptic devices (Touch, 3D systems, Rock Hill, SC, USA) are used as a way to interact between users, vein blood samplings, and 3D models in virtual space in a vein simulator. In the case of HMD controllers, inter-actions with virtual objects are performed using raycast techniques. When the object is in contact with the virtual object for more than a certain period of time by projecting a ray in the direction indicated by stopping the hand 3D model linked to the HMD controller, the object is determined to be selected. Based on this, manipulation of the syringe, UI selection, manipulation of the left arm of the 3D model of the patient, selection of the vacuum tube, and selection and manipulation of the tourniquet or alcohol swab are performed. Figure 8 shows the interaction method used in the HMD controller.

In the case of a haptic device, interactions are performed using physical collisions. The movement of the haptic device is synchronized with the 3D hand model, and when the 3D hand model directly collides with virtual objects through the haptic device, the respective object is considered to be selected. Depending on the object’s shape, it performs actions such as selecting and operating tourniquets or alcohol swabs and selecting and operating syringes. Furthermore, when the selected objects are touched in the 3D patient model through the haptic device, it provides tactile feedback to the user as a physical response. This enhances realism in interactions and allows for processes such as tying tourniquets and applying alcohol swabs. Also, the angle between the blood sampling area of the patient 3D model and the syringe 3D model is calculated using a vector, and when the syringe insertion angle range (about 15–25 degrees) is reached during actual vein blood sampling, the syringe 3D model is inserted into the patient 3D model’s blood sampling area to perform vein blood sampling. Figure 9 shows the interaction method using the haptic device, and Figure 10 shows the method of calculating the angle between the blood sampling area and the syringe 3D model.

2.3. Evaluation of Vein Blood Sampling Simulator

This study’s protocol was approved by the institutional ethics committee (CUDHIRB 2202005). In order to verify the effectiveness of the implemented vein blood sampling simulator, a usability evaluation of the virtual-environment-based vein blood sampling simulator was conducted on 40 dental college students who had no experience in performing within a virtual-environment-based platform. For the objectivity of the usability evaluation results, students who had no experience in actual vein blood sampling were selected, and education on vein blood for actual patients was conducted through textbooks and practice books. In addition, the learning method for the vein blood sampling process was regenerated in the vein blood sampling simulator, and subjects performed vein blood sampling simulations according to the learning method. Figure 11 shows the sequence for performing vein blood sampling using the VR simulator, and Figure 12 shows the execution of the VR simulator.

After performing the vein blood sampling VR simulator, a questionnaire consisting of five items (simulator resolution, simulator accuracy, simulator immersion, simulator usability, simulator educational utility) was provided. Each item is composed of 4–5 questions related to the item’s theme. Additionally, a 5-point scale (very good (5), good (4), normal (3), bad (2), very bad (1)) was used to assign scores for each question.

3. Results

3.1. Implementation of Vein Blood Sampling Simulator

Based on the proposed method, a vein blood sampling simulator was implemented. Unity was used to implement the virtual environment of the vein simulator, and virtual objects for the environmental model, interaction model, and blood sampling model that make up the virtual environment were created using the 3D CAD software 3Ds Max (2022.3) [18] and Cinema4D (R23.110) [19]. Furthermore, for experiencing the vein blood sampling VR simulator’s virtual environment, the HMD used was Oculus Quest 2. To ensure smooth integration with the Oculus Quest 2 in Unity, Android SDK, Android NDK, and Oculus SDK were employed. The interactive devices used during the vein blood sampling simulator execution were the Oculus Quest 2 Controller and Touch Haptic Device, and OpenHaptics was used for seamless integration with the haptic device. In addition, the module for interaction between the controller of Oculus Quest 2 and the virtual object through the Haptic device was self-produced based on the C# script. And a simple GUI allows users to select necessary functions in real time in a virtual environment, train vein blood sampling, and check the current status. A guide to performing vein blood sampling, skin transparency and strength of patient 3D models, and recording vein blood sampling can be used through GUI buttons, and injector angles for vein blood sampling, and current vein names are displayed through GUI labels. Figure 13 shows the execution scene of the vein blood sampling simulator implemented in the Unity environment.

3.2. Application and Efficacy Evaluation of Vein Blood Sampling Simulator

Table 2. Simulator evaluation result.

Item	Question	Avg.	SD.
Resolution	Is the representation of the human anatomy 3D model clear?	4.125	0.81202
	Is the human anatomy 3D model similar to the picture in the book?	4.1	0.86023
	Is the human anatomy 3D model representation clear when interacting with it?	4	0.86603
	Does the human anatomy 3D model hold up well during the vein blood sampling?	4.025	0.8511
Accuracy	Is the vein blood sampling virtual environment implemented well?	3.975	0.90795
	Are the viewpoints and hand positions consistent?	3.05	1.13908
	Does the movement of the virtual object match the movement of the educator?	3.425	1.11552
	Does the virtual object reflect minute movements?	3.45	1.11692
	Is the vein blood sampling step going well?	4.05	0.80467
Immersion	Is there a sense of tactile reality?	3.425	1.13771
	Is it visually realistic?	3.725	1.09516
	Are the controls user-friendly?	3.525	1.04851
	Are the manipulations as intended by the educator	3.225	1.03652
Usability	Are the simulators professional?	3.775	1.6037
	Can it be used for practical training?	3.875	0.95361
	Can it be used for patient explanation?	4.175	0.77096
	Are you thinking of using simulators for other medical procedures?	3.975	1.01211
Educational utility	Is it helpful in understanding actual skills compared to books?	4.1	0.91652
	Can I adapt to the actual vein blood sampling process?	3.925	1.05801
	Can you explain vein blood sampling to other medical personnel after learning with the simulator?	3.95	0.9734
	Can you explain the vein blood sampling process to the patient after learning with the simulator?	4.075	0.81815

shows the usability evaluation items and result for vein blood sampling simulation. Figure 14 shows a graph of average results of usability evaluation.

According to the evaluation results, the average scores for all items were above 3.807. This being close to 4 (good) on the 5-point scale, it can be considered that the overall utility of the vein blood sampling practice VR simulator is good, and with some improvements, it appears that results above 4 could be achieved. In the case of resolution, an average of 4.06 points with a standard deviation of 0.847 points was obtained for the four questions. These results suggest that the anatomical information of the patient 3D model in the vein blood sampling practice VR simulator is similar to actual information. In the case of accuracy, an average of 3.59 points with a standard deviation of 1.017 points was calculated across the five questions. However, upon closer examination of the specific scores, it can be noted that while the items related to the functionality of the VR simulator received high scores, the interaction aspect received lower scores. In the case of immersion, an average of 3.475 points with a standard deviation of 1.079 points was calculated for the four questions. However, upon examining the detailed scores, it can be observed that, similar to accuracy, while the items related to the functionality of the VR simulator (visual-based realism) received scores close to 4, the interaction aspect received lower scores. In terms of usefulness, an average of 4.0125 points with a standard deviation of 0.941 points was calculated for the four questions. This can be considered to represent results better than ‘good’. Based on this, it was confirmed that by further improving the VR simulator, the proposed vein blood sampling practice VR simulator could be used as a substitute for actual practical training. Compared to other items, the scores for items related to interaction in the accuracy and immersion categories are lower, which can be attributed to a lack of experience with VR-related content (difficulty in navigating the connected hand model, operating virtual objects, and lack of proficiency in using interactive equipment). It is expected that these aspects will improve through repetitive exposure to various virtual-environment-based simulators.

4. Discussion

In this paper, a method for developing a vein blood sampling simulator based on a virtual environment was proposed, and the effectiveness of the developed vein blood sampling simulator was confirmed. The environmental model, interaction model, and blood sampling models used in the vein blood sampling simulator were created using 3D CAD software, and a patient 3D model including an anatomical 3D model was created and used to improve the realism of vein blood sampling, and information such as rigid body and collision was created for smooth interaction of each model using Unity. And, for interaction with virtual objects in the Unity environment, an interaction method using HMD-controller-based ray-cast and an interaction method using haptic-device-based collision were implemented and applied to the simulator. Then, in order to verify the effectiveness of the vein blood sampling simulator, usability evaluation was conducted on 40 dental students, and as a result, the validity of the vein blood sampling simulator was verified. As a result, it was confirmed that the practice of vein blood sampling is possible through the proposed vein blood sampling simulator. Ultimately, it has been confirmed that vein blood sampling practice is achievable through the proposed vein blood sampling VR simulator. This VR simulator utilizes only operational and interaction devices, making it a cost-effective tool for repetitive vein blood sampling practice. Therefore, it is expected to be an efficient educational tool. Furthermore, the vein blood sampling VR simulator can be used on various operating devices (PC, HMD Device, Smart Mobile), allowing customization for different clinic operators by selecting suitable interaction devices for each device.

However, it was confirmed that the proposed vein blood sampling simulator method was not suitable for application to the evaluation of vein blood sampling skills because it was difficult to evaluate the user’s vein blood sampling practice process objectively and in real time. In the future, we will study the method for applying the proposed vein blood sampling simulator to the evaluation of vein blood sampling skills. To this end, we plan to develop a multi-access module that allows a large number of people to access the simulator so that the vein blood sampling practice process can be checked in real time, as well as an artificial intelligence model for evaluation of vein blood sampling techniques to objectively evaluate the user’s vein blood sampling technique process. Furthermore, we also plan to undertake advanced work to enhance the educational efficiency of the implemented vein blood sampling VR simulator. These advanced tasks include improving the realism of the VR simulator by enhancing the quality of 3D objects (such as textures and grids), adding various vein and patient shapes to better adapt to real vein blood sampling scenarios, refining interaction methods based on interaction devices (haptic device, HMD controller) to enhance the immersion and accessibility of the VR simulator, and exploring the application of augmented reality technology to create various vein sampling education methods.