An Immersive Virtual Reality Simulator for Echocardiography Examination

Abstract

Echocardiography is a medical examination that uses ultrasound to assess and diagnose the structure and function of the cardiac. Through the use of ultrasound waves, this examination allows medical professionals to create visualizations of the cardiac muscle, enabling them to diagnose and monitor conditions such as cardiac diseases, abnormalities, and functional disorders. An echocardiogram plays a crucial role in the early detection and diagnosis of various cardiac issues such as hypertension, myocardial infarction, valvular cardiac disease, and myocardial hypertrophy. It significantly contributes to determining treatment and management strategies. To achieve accurate disease diagnosis and develop appropriate treatment plans through echocardiography, it is essential to have a thorough understanding of proper probe usage, the precise acquisition of echocardiographic images, and the ability to interpret various echocardiographic examinations such as two-dimensional, M-mode, Doppler, etc. To enhance the skills required for echocardiography, medical educational institutions conduct theoretical classes, practical sessions using patient models, and clinical practice sessions with actual patients. However, issues such as inadequate practical adaptation due to theory-centric education, limitations in practical opportunities due to insufficient practice equipment, ethical or safety concerns arising during clinical practice, and a lack of educators leading to insufficient feedback, are currently being encountered. Hence, there is a need for new educational methods that can address the existing challenges in echocardiography education. In this paper, as part of these efforts, we propose a virtual reality-based immersive simulator for practical echocardiography training. The proposed echocardiography simulator allows users to explore a virtual echocardiography examination space by wearing a head-mounted display (HMD). This simulator consists of 3D virtual space models, interactive models manipulated by interaction devices, and 3D patient models containing normal or abnormal anatomical cardiac models. Using interactive devices such as HMD controllers and haptic devices, users can manipulate 3D models related to echocardiography within the simulator and interact with 3D patient models containing normal or abnormal anatomical cardiac models, allowing for the practice of echocardiography examinations. Ultimately, a performance evaluation of the developed immersive virtual reality simulator and usability validation targeting medical university students were conducted. The evaluation and validation results confirmed the potential efficacy of the proposed echocardiography VR simulator.

1. Introduction

Clinical practice is a crucial aspect of medical education, complementing theoretical learning by providing an opportunity to acquire the skills and experience necessary in real clinical situations or to learn how theoretical knowledge is applied in practice. Through clinical practice, learners can gain practical experience, enhance teamwork skills, boost confidence in clinical situations, and improve problem-solving abilities [1,2]. Clinical practice, being highly important in conjunction with theoretical learning for the development of competent healthcare professionals, is actively conducted in many medical education institutions. However, traditional clinical practice has several drawbacks. Some of the typical issues include limited environments and models, insufficient practical opportunities, and a lack of patient diversity. The frequent use of limited environments and models in clinical practice makes it challenging to fully understand a variety of real clinical situations. Due to a lack of time or resources for clinical practice, learners often face challenges in securing a sufficient number of practice opportunities. The limited diversity in patient models used for clinical practice can result in a diminished ability to handle specific types of patients or situations, making it challenging to enhance interaction skills with patient populations [1,3,4]. Due to these issues, many medical education institutions are currently making efforts to enhance traditional clinical practice. Additionally, companies and research institutions are conducting studies on alternatives to replace conventional practical education.

In particular, there is a significant amount of research being conducted on incorporating virtual reality (VR) or augmented reality (AR) technologies into clinical practice [5,6]. Virtual reality or augmented reality technologies can replicate various clinical practice environments and scenarios at low cost. They enable the creation of 3D anatomical models and 3D surgical equipment models, allowing for clinical practice in a manner that closely resembles real clinical situations. Furthermore, by manipulating 3D models of surgical equipment using various interactive devices, learners can experience not only visual but also auditory, tactile, and kinesthetic sensations. This immersive experience, achieved through the integration of virtual reality and augmented reality technologies into clinical practice, enhances educational effectiveness.

Firstly, there exist various research cases on educational training using VR. At Northampton University, in order to improve nursing students’ ability to respond to clinical situations, a virtual reality simulation product that integrates four head-mounted display (HMD) devices and a large screen was developed. Through this, nursing students can experience various clinical scenarios in a virtual environment and can learn and verify clinical practice and communication skills [7]. Oxford University developed Oxford Medical Simulation to enhance the clinical skills of nurses, specialists, and medical students working at the John Radcliffe Hospital. Through this simulation, learners can practice and assess clinical skills in areas such as nursing education, medical emergencies, pediatrics, internal medicine, mental health care, and physical examinations [7,8]. Falah et al. [9] developed a virtual reality-based cardiac anatomy education system using stereo 3D glasses to address the shortcomings of traditional visualization-based anatomy education methods. They confirmed the utility of the virtual reality cardiac anatomy education system through a group of medical experts. Schild et al. [10] developed an emergency response training simulation for anaphylactic shock. They improved training efficiency by adding a module that allows two learners and one instructor to connect simultaneously to the simulation. Pulijala et al. [11] developed a virtual reality education tool for loft type 1 fracture using HMD and Leap Motion (Ultraleap, San Francisco, CA, USA) [12] and verified the usefulness of the virtual reality education tool through a surgeon to confirm the potential of VR surgery for orthodontic surgery training. Mckinney et al. [13] compared the educational outcomes of a group trained on a VR-based total knee arthroplasty surgical simulator with a group guided by traditional surgical technique manuals, involving a total of 22 orthopedic surgery trainees. The results of the comparison confirmed that VR surgery was superior to education based on traditional surgical technique manuals. Andersen et al. [14] conducted education with fourth-year medical students, dividing them into a group that received VR-based self-directed ultrasound training and a group that received instructor-led training. The educational effectiveness was then compared using an objective structured ultrasound examination tool. The comparison revealed that VR-based self-directed learning was as effective as instructor-led education. Kennedy et al. [15] conducted education on arterial blood gas with a group trained using VR-based methods and a group undergoing traditional education. The study then compared the error rates between the two groups during simulated practice. The comparison revealed that the VR-based education group had 40% fewer errors compared to the traditional education group. Arango et al. [16] developed a VR-based echocardiography simulator that allows practice of echocardiographic examinations on nine types of cardiac models using a head-mounted display (HMD) and controllers. Bard et al. [17] implemented a virtual reality-based representation of Alzheimer’s patients to enhance understanding of the needs of Alzheimer’s disease patients and to serve as a curriculum tool. They evaluated the utility of the Alzheimer’s VR patient among 150 medical students with relevance to Alzheimer’s patients. Kiyozumi et al. [18] developed virtual reality content for the initial assessment segment of a face-to-face-based Japanese prehospital trauma evaluation and care education program. They evaluated the educational effectiveness with a group of 14 medical students. Yang et al. [19] validated the effectiveness of a virtual reality-based neonatal resuscitation education program aimed at enhancing responses to high-risk neonatal emergency situations. The study involved 29 nursing students and assessed outcomes related to neonatal resuscitation nursing knowledge, problem-solving and clinical reasoning abilities, confidence in practical performance, anxiety levels, and learning motivation.

Additionally, there are research cases on educational training and surgical guides using AR. Ropelato et al. [20] developed an augmented reality-based microsurgery simulator operating with AR glasses for fine manipulation training in ophthalmic surgery. They validated the effectiveness of the simulator with 50 trainees, confirming that the training method using the simulator improved fine manipulation skills compared to traditional training methods. Rhienmora et al. [21] developed a dental surgery simulator using haptic devices and the augmented reality algorithm ARToolkit [22]. They assessed the strengths and weaknesses of the surgical simulator through feedback from some trainees and experts, confirming the potential of augmented reality technology in the field of dental surgery training. Si et al. [23] augmented a 3D model of the brain based on CT data onto a skull model created by a 3D printer. They developed an educational simulator for training brain tumor resection surgery using a Hololens (Microsoft, Redmond, WA, USA) [24] and two haptic devices. Through usability validation, they confirmed the utility and potential of augmented reality technology in neurosurgical training. Schott et al. [25] developed a system that allows for multi-user connectivity in anatomy education using VR, AR, and multiple user connections. Aebersold et al. [26] conducted a study with 69 nursing students, where they learned the skill of placing a nasogastric tube using either traditional teaching methods or an AR-based anatomy simulation module. Through a survey, it was confirmed that learning with the AR-based module was more beneficial than traditional teaching methods. Dennler et al. [27] confirmed that AR technology-based surgical guidance improved the precision of pilot hole drilling for pedicle screws and enhanced surgical competency in a novice physician group. Zhu et al. [28] utilized an AR-based surgical guidance system for catheter insertion and hematoma removal during hypertensive intracerebral hemorrhage surgery, confirming its high accuracy and feasibility. Hess et al. [29] developed advanced cardiovascular life support simulation scenarios and validated the effectiveness of communication skills during scenarios when cardiac arrest occurred using AR headsets. The validation was confirmed through interviews with 18 medical assistant students. Mai et al. [30] compared the efficiency of a system guiding implant placement using augmented reality in dental implant surgery with traditional implant placement methods (the freehand method and template-based static placement).

Many medical education institutions and hospitals are developing and utilizing virtual reality- or augmented reality-based clinical practice tools for training in various medical fields. However, research on the development of virtual reality- or augmented reality-based simulators for echocardiography training is still relatively scarce. Echocardiography is a diagnostic medical test that utilizes ultrasound technology to visualize the structure and function of the cardiac muscle. It involves creating images of the cardiac muscle to diagnose various cardiac conditions. This type of examination aids in the early detection and diagnosis of various cardiac issues, providing valuable support for treatment and management decisions. Echocardiography allows for the identification of various conditions such as hypertension, myocardial infarction, valvular cardiac disease, myocardial hypertrophy, abnormalities in ventricular cells and valves, cardiac failure, atrial fibrillation, and aneurysms. To accurately assess the cardiac condition of individuals undergoing such diverse examinations, it is essential to have precise probe contact at the correct locations, obtain appropriate echocardiographic images, and possess the ability to interpret the acquired ultrasound images. To enhance echocardiography skills, repetitive practice of echocardiographic examinations is essential, complementing theoretical learning. However, existing echocardiography practice faces challenges in skill improvement due to a lack of practice opportunities caused by insufficient practice equipment, inadequate adaptation to real clinical environments due to the technical limitations of equipment, difficulties in interaction with actual patients due to practicing in consistent scenarios, and discrepancies with real clinical environments. Therefore, similar to other medical practices, there is a high demand for virtual reality- or augmented reality-based practice tools in echocardiography training. Research on virtual or augmented reality-based simulators for echocardiography training has only been investigated in the study by Arango et al. [16]. However, in this study, the virtual space for cardiac ultrasound examination was found to be simple, leading to a lack of immersion, and the use of only controllers for interaction made it difficult to provide learners with various sensations, resulting in a lack of realism. Additionally, it lacks multi-user connectivity, making it challenging for group education, collaborative practice, and real-time feedback within the virtual space.

The objective of this paper is to develop a VR-based simulator that allows multi-user access, supports various interactive devices, and enables the practice of echocardiography examinations in diverse cardiac scenarios. We intend to describe the development approach, performance evaluation, and validation methods for a simulator designed to practice echocardiographic examinations on various cardiac models using VR, interactive, and multi-user connectivity technologies. Subsequently, we aim to evaluate the performance of the developed VR echocardiography simulator, validate its effectiveness among actual medical university students, and present the results.

2. Materials and Methods

In this section, we propose a development approach for an echocardiography simulator utilizing virtual reality and interactive devices. The proposed simulator primarily utilizes a PC with an HMD for a virtual reality experience. The simulator allows users to wear an HMD and practice echocardiography examinations using haptic or HMD controllers. Moreover, with limited interactive devices (excluding haptic devices), the simulator can perform echocardiography practice on an HMD or a smart device with available storage media. The internal components of the simulator include a virtual space similar to the actual echocardiography examination space, interactive 3D models related to echocardiography tools, and a 3D patient model containing normal and abnormal anatomical cardiac models. Furthermore, interaction methods were developed to perform the echocardiography examination process through interactive devices. Additionally, a multi-user connection module was developed to allow multiple individuals to participate in the same educational and practical sessions. Figure 1 shows an overview of the development of the echocardiography simulator.

2.1. Production of Virtual Space for Echocardiography Examination Practice

Information about the actual environment where echocardiography examinations take place was collected. We visited medical facilities outside of clinic hours to directly observe the echocardiography examination space. We collected information about the actual environment where echocardiography examinations take place by seeking advice from specialists in cardiology. The gathered information includes the layout of the echocardiography examination space, the procedures involved, the types of equipment used, and the methods of interaction related to echocardiography examinations. Based on this information, we created 3D models of the space where echocardiography examinations are performed, the equipment used for echocardiography examination, and the 3D models of normal and abnormal anatomical cardiac muscle. These models were developed using Universal Render Pipeline (URP) shaders and 3D CAD software (3Ds Max 2022.3 and Cinema4D R23.110). The 3D models include geometric shapes, color, and material information and are saved in formats such as DXF, OBJ, and FBX that are compatible with virtual environment creation software like Unity (Unity, San Francisco, CA, USA). Subsequently, to mitigate resource consumption during rendering, we utilized Unity’s Light Mapping feature. Figure 2 shows the configured virtual space for echocardiography practice.

The 3D models constituting the virtual space are categorized into environmental models representing the space where echocardiography examination is performed, models interacting with the HMD controller (Oculus Quest 2, Meta Platforms, Inc., Menlo Park, CA, USA) or haptic device (Touch, 3D systems, Rock Hill, SC, USA) manipulation, and patient models incorporating anatomical 3D models of the cardiac muscle.

Table 1. Classification of the 3D models according to the utilization plan.

Environment Model	Interaction Model	Blood Sampling Model
Ward	3D hand model	3D patient model
Medical bed	Probe	3D cardiac model
Echocardiography equipment

shows the classification of the 3D models according to the utilization plan.

The 3D models categorized under the environmental model, such as ward and medical beds, were meticulously designed to closely resemble real environmental information for an enhanced sense of immersion. These 3D models are fixed in the virtual space, and each model interacts with models belonging to different categories. The ward 3D model represents a space where the trainee wearing the HMD can perform activities and learn, while the 3D medical bed model represents the place where the posture of the 3D patient model is portrayed and changes. When directly utilized in the virtual environment, the ward and medical bed models created in 3D CAD software (3Ds Max 2022.3 and Cinema4D R23.110) can lead to various issues. Problems such as the trainee passing through ward walls during activities and learning in the ward model and the medical bed model experiencing overlap or pose changes when the 3D patient model passes through or changes posture could occur. To prevent these issues, rigid body properties were assigned to the ward and medical bed models. Rigid body properties provide physical information (shape invariance, volume, mass, constraints on external forces, etc.) to the models, allowing 3D models in the virtual space to exhibit realistic movements. To ensure the stability and invariance of the hospital and medical bed models, mass and constraint information were assigned as rigid body properties. Figure 3 shows the ward and medical bed models.

Furthermore, the echocardiography equipment model was created based on the actual echocardiography examination equipment (RS85, Samsung, Suwon, Gyeonggi, KR) used in clinical settings to enhance the immersion of the VR simulator. Separate 3D models were generated for the probe, monitor, panel, and main body, mimicking the real echocardiography examination equipment. These models were then assembled, and rigid body properties, including mass and constraint information, were assigned for fixed placement similar to other models in the environmental model category. Figure 4 shows the actual echocardiography examination equipment and the created 3D model of the echocardiography equipment.

In the echocardiography equipment model, the probe is a model that interacts with the movement of the interaction device through coordinate transformations. It consists of a probe orientation marker and a probe touch part. The probe touch part is capable of contacting specific areas of the 3D patient model, and after contact, it plays the appropriate ultrasound image on the monitor based on the direction of the probe orientation marker. The monitor includes a plane texture for displaying echocardiography images, and the appropriate image is output based on the contact position of the probe and the angle (the direction of the probe orientation marker). To address the inconvenience of having to turn the neck frequently for confirmation during practice, a large plane texture object linked to the equipment monitor was additionally created on the upper wall of the medical bed. The detailed interaction methods of the probe and the output results on the monitor based on interactions are discussed in detail in Section 2.2.

The panel consists of two buttons that can change the visualization mode of echocardiography examination images. Collision properties were assigned to each button to enable interaction with the 3D hand model. Collision properties refer to characteristics related to collisions between models existing in virtual space, allowing realistic interaction through collision detection and responses. The visualization mode of the inspection images was altered based on the properties of the collided button with the 3D hand model. The results of the visualization mode change based on the interaction between the 3D hand model and the panel buttons are discussed in detail in Section 2.2.

The model of the hand, classified under the interaction model category, is linked to the HMD controller or haptic device movement. The left hand is used for UI (user interface) manipulation to facilitate the smooth utilization of the echocardiography examination simulator. The right hand is used to manipulate the probe model, contacting the 3D patient model and displaying echocardiography examination images for the corresponding location. Considering the scenario where a left-handed individual conducts an echocardiography examination, we designed the interlocking mechanism and roles of the right and left hands to be interchangeable. And to represent a natural hand shape during UI and probe manipulation, we created hand motion animations. To generate hand motion animations, we utilized a Leap Motion device that can recognize hand gestures through a depth camera and software that enables the creation and editing of motion animations. We utilized the hand skeletal data obtained through Leap Motion to generate key hand motion keyframe data. Through animation editing software, we applied interpolation between keyframe data to create hand gesture animations. Figure 5 shows the 3D hand model with the included skeletal data and the applied animation.

The probe model is depicted as being held in either the left- or right-hand model by the trainee, depending on the hand using the HMD controllers or haptics. Through coordinate transformation, the movement and rotation of the probe model are synchronized with the trainee’s hand movements. And, upon contact of the probe model with the cardiac region of the patient’s 3D model, collision information is incorporated into the probe contact area to generate echocardiographic examination images for the corresponding location. Figure 6 shows the grasp of the 3D probe model by each hand.

For the 3D patient models included in the patient model classification, they possess 3D anatomical cardiac models. To achieve realistic representations of 3D patient models, a plug-in utilizing actual facial images was employed to create virtual human models. Additionally, skeletal data and Mesh Deform were utilized for a natural representation of echocardiographic examination positions. The created examination positions include the supine position (lying straight), the lateral position (lying on the side), and the prone position (lying face down with the chest elevated). Keyframes were created for each posture, and keyframe interpolation was utilized to smoothly represent transitions between different positions. Figure 7 shows the postures of the 3D patient model.

For the 3D cardiac model, it was created as an anatomical model, realistically represented based on CT volume images. A total of 10 steps (dicom file read → 3D model generation → select segmentation → masking → volume rendering → mesh remove → mesh detach → mesh reconstruction → surface mesh retopology → texture and diffuse map) were followed to produce the basic 3D cardiac model. Next, anatomical information for the basic 3D cardiac model was generated by referencing actual cardiac images and analyzing the internal structure of the cardiac muscle. To achieve a realistic depiction of the 3D anatomical cardiac model, normal map textures and diffuse maps were applied, and animation synchronized with actual cardiac contractions was created using the mesh deformer feature. Figure 8 shows the process for generating the 3D cardiac model, and Figure 9 shows the produced basic 3D anatomical cardiac model.

Cardiac diseases arise from factors such as hypertension, atherosclerosis, smoking, hyperlipidemia, diabetes, genetics, and obesity. Depending on the type of disease, cardiac diseases manifest abnormalities in the shape of the cardiac muscle, including damage, expansion, and contraction in the myocardium, ventricles, atria, and valves. Seven cardiac diseases (myocardial infarction, left ventricular hypertrophy, dilated cardiomyopathy, atrial septal defect, mitral valve stenosis, aortic coarctation, and pulmonary atresia) with prominent abnormal cardiac structural forms were selected, and the abnormal structural forms associated with each disease were investigated. Subsequently, based on the investigated shapes, various processes were performed, including texture generation and application, as well as the generation and application of abnormal shapes. These processes were conducted to apply abnormal cardiac forms to the internal or external parts of the normal 3D anatomical cardiac model.

Table 2. Three-dimensional model generation methods for each cardiac disease.

View	Generation Method Based on Abnormal Cardiac Feature
Myocardial infarction	Feature: The presence of necrotic areas due to a lack of blood supply to the cardiac muscle. Method: To represent the necrotic areas, the diffuse map and normal map of the normal cardiac model were modified and applied.
Left ventricular hypertrophy	Feature: An increase in the thickness of the left ventricular wall resulting in reduced size of the ventricle. Method: The size of the left ventricle was physically reduced using sculpting, a method of manipulating the shape of the 3D model and adding details.
Dilated cardiomyopathy	Feature: A decrease in the thickness of the left ventricular wall resulting in enlarged size of the ventricle. Method: The size of the left ventricle was physically enlarged, a method of manipulating the shape of the 3D model and adding details.
Atrial septal defect	Feature: The creation of a perforation in the septum between the left atrium and right atrium. Method: The perforation was represented using the difference function of the Boolean technique, and the created perforation was smoothly rendered using smoothing.
Mitral valve stenosis	Feature: Abnormal closure of the mitral valve due to bacterial infection in the mitral valve. Method: A modified diffuse map and a normal map were used to represent bacterial infection in the mitral valve. Abnormal closure movements of the mitral valve were depicted using keyframe interpolation.
Aortic coarctation	Feature: Abnormal closure of the aortic valve due to bacterial infection in the aortic valve. Method: A modified diffuse map and a normal map were used to represent bacterial infection in the aortic valve. Abnormal closure movements of the aortic valve were depicted using keyframe interpolation.
Pulmonary atresia	Feature: The presence of a blood clot in the artery connecting the cardiac muscle to the lungs, resulting in the blockage of the pulmonary artery. Method: Additional modeling of a blood clot and its placement in the pulmonary artery.

shows the abnormal shapes of cardiac structures associated with each cardiac disease and the methods used to create 3D models, while Figure 10 shows the differences between the normal 3D anatomical cardiac model and the abnormal 3D anatomical cardiac model.

Normal and abnormal anatomical cardiac muscles consist of a total of 8 variations, and accordingly, 8 3D patient models were created. The 3D patient models do not have any specific correlation with normal or abnormal cardiac muscles. To facilitate the smooth differentiation of normal and abnormal echocardiography processes selected during the use of the VR simulator, different 3D patient models were applied to each cardiac model. To achieve this, various 3D modeling techniques were used to create 3D patient models for each cardiac model by altering features such as hairstyle, skin, and body size. For the hairstyle, we used Bezier surfaces to shape the hair and mapped color textures of the hair onto surface control points using a diffuse map. Additionally, an alpha map was used to remove specific parts, resulting in the creation of various hairstyles. Figure 11 shows the before and after application of an alpha map to the hair with a diffuse map.

Additionally, to create diverse age groups for the 3D patient models, normal maps were used to represent skin wrinkles and textures, providing three-dimensionality. Color information from diffuse maps was employed to express various skin tones. And, by manipulating the distances between joints in the skeletal information composed of joints and bones, we adjusted the size of the patient’s 3D model. Using a morphing technique [31] to manipulate the shape of the reference model (including a normal cardiac muscle), we adjusted the muscles of the patient’s 3D model. Figure 12 shows the generated 3D patient models.

2.2. Approaches to Echocardiography Examination Interaction

In the echocardiography examination simulator, interaction between the 3D models is facilitated through HMD controllers and haptic devices. In the 3D hand model, the left hand is linked with the HMD controller, while the right hand is linked with the haptic device to perform interactions with virtual objects. If the user is left-handed, the interaction devices connected to the left and right hands will be switched. In the case of HMD controllers, the interaction is based on ray casting, involving casting a ray in the direction pointed by the index finger of the hand model and pressing the buttons on the controller. If the projected ray comes into contact with a virtual object for a certain duration, the object is determined to be selected. Through interaction with the HMD controller, it is possible to manipulate UI objects, utilize UI buttons for performing echocardiographic examinations, and select buttons on the echocardiography equipment body.

For the haptic device, interaction is based on both ray casting and physical collision techniques, and different functionalities are performed based on each interaction technique. Interaction based on ray casting involves projecting a ray in the direction pointed by the index finger of the hand model when a button on the haptic device is pressed. If the projected ray comes into contact with the probe model for a certain duration, the hand model grasps the corresponding probe model. Then, the probe 3D model is synchronized with the hand 3D model, which is responsive to the movement of the haptic device through coordinate transformation technology. Interaction based on physical collision techniques involves manipulating the haptic device, and when the haptic device, containing collision information, directly collides with the cardiac region of the patient’s 3D model, that area is determined to be selected. If the contacted area corresponds to the actual echocardiography position, the appropriate echocardiographic examination image is displayed in the UI box at the contacted area. Furthermore, by providing tactile feedback through the haptic device when it comes into contact with the patient’s 3D model, the sense of immersion in the interaction is enhanced. Figure 13 shows the developed interaction method.

Through the manipulation of a virtual probe or the control of the ultrasound equipment panel using interactive methods, adjustments to the echocardiographic images were made. In an actual echocardiogram, the probe is adjusted to examine eight different views. These views include PLAX (parasternal long axis view), PSAX MV (parasternal short axis view—mitral valve level), PSAX AV (parasternal short axis view—aortic valve level), PSAX PM (parasternal short axis view—papillary muscles level), PSAX Apical (parasternal short axis apical view), A4C (apical 4-chamber view), A3C (apical 3-chamber view), and A2C (apical 2-chamber View). In the echocardiography VR simulator, the echocardiographic image view changes based on the virtual probe’s contact position on the patient’s 3D model and the orientation of the probe orientation marker. Figure 14 visually depicts the virtual probe positions and the direction of the probe orientation marker for each type of view, while

Table 3. Echocardiography views based on probe 3D model positions and direction.

View	View area based on probe position and direction
PLAX	Probe: Place the probe in the left parasternal region of the chest, with the orientation marker facing the right shoulder of the patient’s 3D model. View area: The aortic valve, left atrium and ventricle, and aortic arch are observed.
PSAX MV	Probe: Place the probe in the left parasternal region of the chest, with the orientation marker facing the left shoulder of the patient’s 3D model. View area: The lower left part of the cardiac muscle, specifically the mitral valve and surrounding structures, is visualized.
PSAX AV	Probe: Aligned in the same position and orientation as PSAX MV, tilting the probe contact surface downward. View area: Anatomical structures related to the aortic valve, specifically the open state of the aortic valve, are observed.
PSAX PM	Probe: Aligned in the same position and orientation as PSAX MV, tilting the probe contact surface upward. View area: The muscles known as the papillary muscles, which are involved in the contraction of the ventricles, are observed.
PSAX Apical	Probe: Move down approximately one to two intercostal spaces from the position of the PSAX MV and the orientation marker facing below the right shoulder of the patient’s 3D model. View area: The structures near the top of the ventricles and around the chordae tendineae (chordae tendons) are observed.
A4C	Probe: Positioned in the fifth to sixth intercostal space in the lower left chest area, with the orientation marker facing the upper arm of the patient’s 3D model. View area: The right and left atria, as well as the upper and lower ventricles, are simultaneously visualized.
A3C	Probe: The contact position for A4C is the same, with the orientation marker facing the right shoulder of the patient’s 3D model. View area: The left atrium, left upper ventricle, and left lower ventricle are observed.
A2C	Probe: The contact position for A4C is the same, with the orientation marker facing the head of the patient’s 3D model. View area: The left atrium and left upper ventricle are predominantly observed.

shows the virtual probe positions and the direction of the probe orientation marker for each type of echocardiographic image view.

The echocardiography examination equipment can change modes for each view. Typically, it supports B-mode (brightness mode), M-mode (motion mode), and color Doppler mode. In the echocardiography VR simulator, it can represent B-mode and M-color mode (a combination of M-mode and color Doppler mode). By interacting with the ray casting or directly colliding with the index finger of the 3D hand model with the buttons on the panel of the echocardiography equipment model, the mode of the echocardiography view changes. Figure 15 shows the B-mode and M-color mode displayed in the echocardiography VR simulator.

2.3. Multi-User Connection Module

The echocardiography examination simulator was developed and integrated with a multi-user connectivity module, allowing multiple educators to simultaneously conduct practical training and exercises. This was designed to address limitations in traditional medical education settings, prevent continuous expenses on consumable materials, and resolve ethical issues arising from clinical practice on actual patients. For the development of a multi-user connectivity module, the Normcore library with features such as smooth mutual transformation synchronization, efficient serialization for congestion control, and stable implementation of multiple connections by blocking the unreliable transmission protocol UDP (user datagram protocol) was utilized. Additionally, the Unity XR Interaction Toolkit was used to support the development of interaction among multiple connected users. The app key issued by Normcore was registered to develop a concurrent connection module, and avatars for participants in the echocardiography examination simulator were created. Multiple participants can freely explore the virtual space of the echocardiography examination simulator, and the participant performing the echocardiography can observe the perspectives of other participants, allowing for a collaborative experience. In addition, to facilitate smooth communication among participants, a voice chat feature was added, and for identification of the participant performing the echocardiography examination among the participants, avatar color variations were introduced. This allows educators to assess the practical echocardiography situation of the learners. Through a multiuser connectivity module, the echocardiography examination simulator can accommodate up to 10 participants. Figure 16 shows the simulator environment with multiple participants connected.

2.4. Implementation Method of the Echocardiography Examination Simulator

Based on the proposed approach, the echocardiography examination simulator was implemented. Unity was utilized to create the virtual environment of the echocardiography examination simulator, and 3D CAD software, including 3Ds Max (2022.3) [32] and Cinema4D (R23.110) [33], was employed to design the 3D models included in the environmental, interactive, and patient models. In addition, the Oculus Quest 2 HMD was utilized for an immersive experience in the echocardiography examination simulator. For seamless integration with Oculus Quest 2 in Unity, Android SDK, Android NDK, and Oculus SDK were employed. For the interaction in the echocardiography examination simulator, Oculus Quest 2 controllers and the Touch Haptic Device were utilized. OpenHaptics (open source) was employed to ensure smooth integration with the haptic device. Furthermore, the interaction between Oculus Quest 2 controllers and the haptic device with virtual objects was custom-made based on C# scripts. Through the Normcore (open source) and Unity XR Interaction Toolkit libraries (open source), we developed a multi-user connectivity module, participant avatar creation, and real-time communication and interaction methods among participants. Furthermore, through a user-friendly GUI, the simulator can select and perform necessary functions in real time within the virtual environment. This includes initiating and concluding echocardiography examinations, as well as checking the current educational status. Additionally, various functions related to echocardiography, such as providing guidance for the examination, adjusting the transparency of the 3D patient model’s skin, and changing the patient’s position, can be carried out using GUI buttons. Figure 17 shows the execution scene of the implemented echocardiography examination simulator in the Unity environment.

2.5. Performance Evaluation of the Echocardiography Examination Simulator

Before validating the implemented echocardiography VR simulator, the performance of the VR simulator was assessed to ensure that smooth echocardiography practice was possible. To evaluate the performance of the echocardiography VR simulator, appropriate performance metrics were selected from Unity’s profiler indicators. The profiler is a performance analysis support tool in Unity for applications and simulations. It provides performance information for frames per second (FPS), CPU usage, GPU usage, memory usage, rendering statistics, physics, audio, network, and UI. Among the performance information provided by the profiler, FPS, CPU usage, GPU usage, and memory usage, which are relevant to the smooth utilization of the VR simulator, were selected for the performance evaluation of the echocardiography VR simulator. Additionally, since the echocardiography VR simulator utilizes a haptic device to manipulate the probe and perform echocardiography examinations, haptic-related metrics such as haptic frame rate and the accuracy of virtual probe contact positions were also utilized in the performance evaluation. Establishing methods to evaluate the selected performance metrics, two PCs with identical specifications were prepared for accurate performance assessment. The echocardiography VR simulator was executed on each PC based on the performance metric evaluation methods.

Table 4. PC specifications used for performance evaluation.

Hardware	Specification
CPU	Intel i9-12900 K Core 16, Thread 24, 5.20 GHz
GPU	NVIDIA GeForce RTX 3080 12 GB
Memory	Samsung DDR4-3200(16 GB), Total 32.0 GB
Storage	Samsung 870 EVO (2 TB) Total 4 TB
OS	Window 10 Education
Display	GBLED GX2802 UHD 4 K, Inch 27, 240 Hz, 16.7 M Color

shows the specifications of the PCs used for performance evaluation, and

Table 5. Evaluation methods for the selected items.

Item	Evaluation
FPS	Method: With two PCs connected simultaneously, the VR simulation was run for 5 min, recording FPS every second and then calculating the average.
CPU usage	Method: With two PCs connected simultaneously, the VR simulation was run for 5 min, recording CPU usage every 30 s and then calculating the average.
GPU usage	Method: With two PCs connected simultaneously, the VR simulation was run for 5 min, recording CPU usage every 30 s and then calculating the average.
Memory usage	Method: With two PCs connected simultaneously, the VR simulation was run for 5 min, recording Memory usage every 30 s and then calculating the average.
Haptic frame	Method: With two PCs connected simultaneously, haptic feedback was used for 5 min, recording haptic frames per second (FPS) every second and then calculating the average.
Contact point accuracy	Method: Calculating the average distance error between the probe position (total of eight) of the specialist-designated 3D patient model and the virtual probe contact position where the echocardiography examination image is displayed (based on every 10 trials).

outlines the selected performance evaluation items and methods.

2.6. Validation of the Echocardiography Examination Simulator

This study’s protocol was approved by the institutional ethics committee (CUDHIRB 2202002Q02). To validate the utility of the implemented echocardiography examination simulator, a validity assessment was conducted using a questionnaire method targeting medical university students. The questionnaire method was chosen as the evaluation approach for validating the utility of the implemented echocardiography examination simulator. This method requires less time, effort, and cost compared to other validity assessment methods. It can easily be applied to diverse samples, and the likelihood of subjective thoughts or biases from the questionnaire provider influencing the results is low. A validity assessment was conducted with a total of 58 medical university students. To enhance the objectivity of the questionnaire results, students without experience in performing virtual environment content tasks, interacting with devices such as HMD controllers and haptic devices, and having limited practical experience in on-site echocardiography procedures were selected. Additionally, the performance evaluation results of the echocardiography VR simulator were described for a meaningful comparison with actual echocardiography equipment. The medical university students participating in the validity assessment were individually instructed to use the echocardiography examination simulator without other participants, followed by the administration of a validity assessment questionnaire. The implemented echocardiography examination simulator follows the sequence of content execution, selection of the desired cardiac condition for practice, starting the simulator, choosing the echocardiography examination, manipulating the probe, and performing the echocardiography examination. During the performance of the echocardiography examination, the displayed examination images are real echocardiographic examination images corresponding to the selected cardiac condition. Figure 18a–f shows the sequence of actions in the echocardiography examination simulator, while Figure 19 shows the execution of the echocardiography examination simulator.

Students independently performed echocardiography examinations on a 3D patient model following the simulator’s execution sequence and completed a questionnaire consisting of five items. The five items in the questionnaire were resolution (performance of transmitting echocardiography examination images), accuracy (precision of model implementation and degree of matching with actual movements within the simulator), usability (realism and ease of use of the simulator), applicability (professionalism and feasibility in medical and educational environments), and educational usefulness (potential for replacing traditional education). Additionally, the students were allowed to provide feedback on any side effects of simulator use and suggestions for improvement. The questionnaire used a 5-point scale (very good (5), good (4), normal (3), bad (2), and very bad (1)) to evaluate responses to questions within each item.

Table 6. Evaluation item for the echocardiography examination simulator.

Item		Question
Resolution	Q (1)	Is the medical imaging displayed on the simulator clear?
	Q (2)	Compared to the images in the book, is the medical image displayed on the simulator clear?
	Q (3)	Is the medical image clear even during actual motion?
	Q (4)	Is the previous medical image well maintained when the phase changes?
Accuracy	Q (1)	Has this simulator implemented the human 3D model well?
	Q (2)	Do the viewpoint and the position of the hand match?
	Q (3)	Does the movement of the equipment match the performer’s movements?
	Q (4)	Does the movement of the equipment reflect subtle movements?
	Q (5)	Does it move to the next step only after the previous step has been clearly carried out?
Usability	Q (1)	Do you think this simulator has tactile realism?
	Q (2)	Do you think this simulator has visual realism?
	Q (3)	Do you think the operation of this simulator is user-friendly?
	Q (4)	Did the operation of this simulator work according to the performer’s will?
Applicability	Q (1)	Do you think this simulator is professional?
	Q (2)	Can this simulator be utilized for actual practice education?
	Q (3)	Can this simulator be used for patient explanation?
	Q (4)	Would you be willing to use a simulator produced based on this simulator?
Educational usefulness	Q (1)	Compared to the book, does this simulator help in understanding the actual skill?
	Q (2)	Can the stages of this simulator be applied to the actual echocardiography process?
	Q (3)	After learning with this simulator, can you explain the skill to other medical personnel?

outlines the usability evaluation items for the echocardiography examination simulator.

Furthermore, to assess the reliability of the evaluation results from medical university students, the kappa statistic, an indicator of the agreement in questionnaire responses among raters, was calculated. The kappa statistic is commonly used, with Cohen’s kappa [34] for assessing agreement between two raters’ response results and Fleiss’ kappa [35] for evaluating agreement among three or more raters. In this study, utilizing Fleiss’ kappa, the questionnaire responses from 58 medical university students were collected, and the statistical program SPSS (29.0.1.0) was employed to calculate the results of the kappa statistic. The kappa statistic results include the kappa value representing the agreement in the response results, the asymptotic standard error indicating the standard error of the kappa value, the Z-value obtained by dividing the kappa value by the standard deviation at the state where the kappa value is 0 (confirming statistical significance), and the confidence interval (CI) bound representing the reliable 95% range of the kappa value. The kappa value ranges from 0 to 1, where values between 0 and 0.2 indicate slight agreement, 0.21 to 0.4 suggest fair agreement, 0.41 to 0.6 indicate moderate agreement, 0.61 to 0.8 signify substantial agreement, and 0.81 to 1.0 represent almost perfect or perfect agreement. The interpretation may be adjusted appropriately based on the evaluated subject or field. The asymptotic standard error reflects the standard error, with values closer to 0 indicating higher reliability of the kappa value. The Z-value, as a standard normal distribution score of the kappa value, indicates statistically significant results with higher scores. The CI bound signifies the confidence range within which the kappa value may exist for the given confidence interval.

3. Results

3.1. Performance Evaluation Results of the Echocardiography Examination Simulator

As mentioned in Section 2.5, the performance evaluation of the echocardiography VR simulator was conducted using two PCs with identical specifications. On one PC, the echocardiography process was performed by connecting both the haptic device and the HMD, while on the other PC, only the HMD was connected to perform concurrent access. Through this, when simultaneous users were present in the echocardiography VR simulator, data for each performance evaluation metric were measured based on the performance evaluation method. The average values for each performance evaluation metric were calculated. Figure 20 represents the graphs for each performance metric, and

Table 7. Average results of the performance evaluation items.

Item	Average	Item	Average
FPS	72.08 (fps)	Memory usage	447.83 (MB)
CPU usage	6.89 (%)	Haptic frame	71.98 (fps)
GPU usage	8.5 (%)	Contact point accuracy	0.39 (mm)

shows the average values for each performance metric.

The results of measuring the FPS of the VR simulator indicate an average of 72.08, with a minimum of 70.78 and a maximum of 73.72. This suggests that the echocardiography VR simulator operates stably, providing a smooth environment for trainees. The CPU usage of the VR simulator was measured, yielding an average of 6.89%, a minimum of 2.00%, and a maximum of 14.0%. The low average CPU usage of less than 10% indicates that the simulator efficiently utilizes resources, minimizing the device’s burden. The GPU usage of the VR simulator was measured, resulting in an average of 8.50%, a minimum of 4.40%, and a maximum of 13.2%. The low average GPU usage of less than 10% indicates that the simulator effectively handles graphics processing, optimizing the use of device graphics resources. The memory usage of the VR simulator was measured, with an average of 447.83 MB, a minimum of 400 MB, and a maximum of 500 MB. The relatively low average memory usage of 447.83 MB indicates that the VR simulator efficiently utilizes memory resources, ensuring stable operation. Overall, the performance metrics selected from the Unity profiler show consistently positive results. In particular, the efficient utilization of resources in performing echocardiography simulations is a highly positive outcome.

Furthermore, the results of measuring the haptic FPS (frames per second) of the VR simulator show an average of 71.98, with a minimum of 70.96 and a maximum of 72.41. The consistently stable average and narrow range of minimum and maximum values for haptic FPS indicate that the simulator provides stable haptic feedback. Measuring the contact point accuracy of the VR simulator resulted in an average error of 0.35 mm, with a minimum error of 0.02 mm and a maximum error of 0.64 mm. This suggests that the VR simulator accurately reproduces the specified probe contact positions by medical professionals, providing an effectively realistic ultrasound probe manipulation experience in the learning and training environment. Performance evaluation items related to interaction also show positive results, indicating that smooth echocardiography learning is achievable through the VR simulator.

3.2. Validation Results of the Echocardiography Examination Simulator

As mentioned in Section 2.6, a validity assessment of the echocardiography examination simulator was conducted using a questionnaire method with 58 medical university students. Based on the questionnaire results, we calculated response rates for each item, as well as the averages and standard deviations of responses for individual questions within each item to evaluate the validity of the simulator. Figure 21 shows the response rates for each item, and Figure 22 shows the averages and standard deviations for individual questions within each item.

In the resolution section, responses were very good (30.17%), good (42.24%), normal (24.14%), bad (1.72%), and very bad (1.72%), indicating that over 72% of the responses were positive. For the four questions in this category, an average of 3.97 and a standard deviation of 0.88 were calculated. These results indicate that the examination images presented in the echocardiography examination simulator are appropriately provided at the probe contact location, and users are satisfied with the sharpness of the provided images. However, the average score for question 4 was lower compared to other questions. This requires restraining the movement of interactive devices to maintain the previous image before medical university students proceed to the next step when displaying echocardiography examination images. However, the medical university students participating in the evaluation have no prior experience with interactive device usage. As a result, precise operations such as restraining the movement of interactive devices were challenging, ultimately resulting in low average scores.

In the accuracy section, very good (31.38%), good (36.55%), normal (24.83%), bad (5.17%), and very bad (2.07%) responses were derived, indicating positive results of 71% or more. For the five questions, an average of 3.90 with a standard deviation of 0.98 was calculated. These results suggest that the representation of models, the movement of objects, and the processes of echocardiography examination within the simulator closely resemble real scenarios. However, the average score for question 3 was lower compared to the other questions. This is because the ratio of haptic device movement to virtual environment movement is different. Representing movement in a 1:1 ratio poses challenges in performing interactions within the virtual environment using the limited movement range of the haptic device. Therefore, the ratio of haptic device movement to virtual environment movement was adjusted to 1:2. Since the validity assessment targeted students with no prior experience in using interaction devices, the low average score was a result of their unfamiliarity with the movement ratio adjustment.

In terms of usability, very good (30.17%), good (38.36%), normal (22.84%), bad (6.47%), and very bad (2.16%) responses were derived. For the four questions, an average of 3.88 with a standard deviation of 0.98 was calculated. In questions related to simulator operation, high average scores were obtained, but lower average scores were obtained in questions related to interaction and visualization. For question 1, the low average score is attributed to the difficulty in sensing haptic feedback when using the controller as an interaction device. Regarding question 2, difficulties such as motion sickness, challenges in adjusting focus, and the inability to wear glasses when using the HMD for the first time resulted in a lower average score due to the challenging visual experience.

In the applicability section, very good (38.79%), good (35.78%), normal (18.10%), bad (4.74%), and very bad (2.59%) were derived. For the four questions, an average of 4.03 with a standard deviation of 1.0 was calculated. The echocardiography examination simulator has been confirmed to be sufficiently utilizable for educational and instructional purposes. Moreover, the potential use of simulation simulators in other medical fields utilizing virtual environment technology for educational and instructional purposes has been verified. However, questions related to professionalism yielded lower average scores compared to other questions. This is due to a slight sense of unfamiliarity with using HMD and interaction devices, causing a slight sense of disparity when compared to the experience of actual echocardiography practice.

In the educational usefulness section, very good (38.51%), good (36.21%), normal (21.26%), bad (1.72%), and very bad (2.30%) responses were derived. For the three questions, an average of 4.07 with a standard deviation of 0.93 was calculated. High average scores were obtained for all questions, confirming that the implemented simulator has educational effectiveness. Finally, feedback on side effects and improvement suggestions included opinions on cyber motion sickness, fatigue related to HMD wear, eye pain, difficulty in exploring the hand model linked to the interaction device, and challenges in object manipulation. These side effects stem from the initial use of virtual reality devices and the unfamiliarity with using interaction devices, which are expected to improve with repeated exposure to various forms of virtual environment-based content.

3.3. Kappa Statistic Results for the Validation Findings

To assess the reliability of the medical university students’ questionnaire responses, which form the basis for the validity evaluation verification, Fleiss’ kappa statistics were calculated.

Table 8. Fleiss’ kappa results by questionnaire item.

Item	Kappa	Asymptotic Standard Error	Z	Lower 95% Asymptotic CI Bound	Upper 95% Asymptotic CI Bound
Resolution	0.508	0.036	14.152	0.438	0.578
Accuracy	0.468	0.026	17.967	0.417	0.519
Usability	0.458	0.033	13.820	0.393	0.523
Applicability	0.547	0.034	15.968	0.480	0.615
Educational usefulness	0.540	0.051	10.666	0.441	0.639
Total	0.419	0.006	68.767	0.407	0.431

presents the Fleiss’ kappa statistical results for each item on the questionnaire and the overall response outcomes.

The kappa values for each item range from a minimum of 0.458 (Usability) to a maximum of 0.547 (applicability). Considering the kappa range results, this indicates substantial agreement, suggesting a reasonable level of consistency in the responses of the medical university students for all items. Additionally, the asymptotic standard error values for each item are below 0.051, indicating that the calculated kappa values are very stable, as they are very close to 0. Furthermore, the Z-values for the statistical significance assessment of the calculated kappa values exceeded 10.666 for each item. This level surpasses the Z-value criterion of 1.96 at a 95% confidence level, indicating that the kappa values are statistically highly significant. Moreover, the size of the 95% confidence interval is below 0.202, suggesting that the item-wise response results of medical university students are reliable in terms of kappa values.

4. Discussion

This paper proposes a development approach for a virtual environment-based echocardiography examination simulator and validates the performance and usability of the implemented simulator. The environmental models, interactive models, and patient models used in the echocardiography examination simulator were created using 3D CAD software (3Ds Max 2022.3 and Cinema4D R23.110). Three-dimensional anatomical models for both normal and abnormal cardiac conditions were created, and distinct three-dimensional patient models were generated for each cardiac model to facilitate clear differentiation. Additionally, using Unity, information such as rigid body and collision was generated for each model to ensure smooth interaction with and between the models. Subsequently, in the Unity environment, interaction methods based on ray casting with HMD controllers, ray casting with haptic devices, and physical collision-based interaction methods were implemented and applied to the simulator, allowing interaction with virtual objects. This enables trainees using the VR simulator to experience various sensations, allowing for realistic performance of echocardiography examinations. Additionally, a multi-user module was developed using available open-source solutions in Unity, facilitating group education, real-time feedback within the virtual space, and collaborative training experiences.

Three-dimensional anatomical models for various cardiac conditions, including normal anatomy, were produced, and distinct 3D patient models were created for each 3D anatomical cardiac model to facilitate easy differentiation. Additionally, using Unity, information for each model, such as rigid body and collision, was generated to ensure smooth interaction. Subsequently, interaction methods based on ray casting with HMD controllers, ray casting with haptic devices, and physical collision-based interaction methods were implemented in the Unity environment and applied to the simulator, allowing for interaction with virtual objects. This enables the simulator to convey various sensations to trainees using VR and enables the realistic performance of echocardiography examinations. Furthermore, a multi-user module was developed using available open-source solutions for multi-user connectivity in Unity. This facilitates group education, real-time feedback within the virtual space, and collaborative training experiences.

To ensure the smooth usability of the implemented echocardiography VR simulator, performance metrics related to the VR simulator were selected in the Unity profiler, and additional metrics related to interaction were defined. Subsequently, performance evaluation methods were established for each metric, and evaluations were conducted based on these methods. The performance evaluation results showed consistently high average results across all metrics. Particularly noteworthy was the minimal difference observed in the average, minimum, and maximum values for all metrics. The narrow range of differences in values for metrics related to the performance of the VR simulator indicates that the implemented echocardiography VR simulator can be utilized reliably with minimal performance fluctuations, providing a stable and high-quality visual experience for trainees. The narrow range of differences in values for metrics related to interaction indicates that the interaction devices within the simulator are effectively functioning, offering users a realistic and consistent interactive experience.

In order to validate the usability of the implemented echocardiography VR simulator, a usability evaluation was conducted with 58 medical university students who had no prior experience with VR content and interaction devices. The usability assessment was performed through a questionnaire on the performance, usability, and applicability of the echocardiography VR simulator. The evaluation results showed high scores for most individual items; however, lower scores were obtained for aspects related to the initial use of VR content and interaction devices. This is attributed to a lack of experience with VR content and interaction devices, and it is anticipated that these aspects will improve with sufficient practice and experience. Furthermore, Fleiss’ kappa statistics were calculated to verify the reliability of the survey responses from medical university students, confirming an appropriate level of agreement in their responses. Ultimately, based on the evaluation results, the echocardiography VR simulator was found to have significant potential, and with slight improvements and regular exposure to users, it is expected to be sufficiently utilized for educational purposes. Future research aims to compare and evaluate the implemented and improved echocardiography VR simulator against traditional teaching methods, intending to confirm the educational efficiency of the VR simulator.

Furthermore, the proposed echocardiography simulator method faces challenges in objectively evaluating the trainee’s echocardiography practice process and providing real-time feedback on echocardiography skills when the instructor is not connected to the simulator. In the future, we plan to explore methods to effectively evaluate the performance of trainees using the echocardiography VR simulator even when the instructor is not actively participating in the simulator. To achieve this, we aim to investigate the integration of cross-platform technology, allowing instructors to remotely monitor the echocardiography practice of trainees across various platforms. Additionally, we will conduct research on the development of an artificial intelligence model for the objective evaluation of the trainee’s echocardiography examination process.