Next Article in Journal
Lower Limb Muscle Activation in Young Adults Walking in Water and on Land
Previous Article in Journal
Sea Buckthorn Leaves as a Potential Source of Antioxidant Substances
Previous Article in Special Issue
Beyond Sight: Enhancing Augmented Reality Interactivity with Audio-Based and Non-Visual Interfaces
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Nuclear Medicine Radiological Hot Laboratory Simulation: A Mixed-Method Intervention Study on Immersive Virtual Reality for Sustainable Education

by
Suphalak Khamruang Marshall
*,
Nantakorn Sirieak
,
Pornchanok Karnkorn
,
Virunyupa Keawtong
,
Awatif Hayeeabdunromae
,
Nadia Noomad
,
Wanita Durawee
and
Jongwat Cheewakul
Molecular Imaging and Cyclotron Center, Department of Radiology, Division of Nuclear Medicine, Faculty of Medicine, Prince of Songkla University, Songkhla 90110, Thailand
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(12), 5041; https://doi.org/10.3390/app14125041
Submission received: 26 April 2024 / Revised: 31 May 2024 / Accepted: 6 June 2024 / Published: 10 June 2024
(This article belongs to the Special Issue Virtual/Augmented Reality and Its Applications)

Abstract

:
The traditional training methods in radiological hot laboratories involve significant challenges, including the risk of radiation exposure and the development of radiophobia among learners. Virtual reality (VR) presents an innovative educational solution by simulating realistic hot lab environments and procedures without associated risks. This mixed-method study investigates the efficacy of VR in enhancing cognitive retention and practical skills and reducing radiophobia among students. All participants (video and VR cohorts) were given a pre-test, same-day training post-test, after 1 month, and after 3 months. In the 3-month test, 13% of the control group scored > 80%, and 87% of the VR group scored > 80% (6.69-fold more significant). VR simulated the real-world hot lab more accurately than training videos, resulting in increased confidence and safety. Resulting in the control group (video training), radiophobia decreased by 1.52-fold; in contrast, the VR training group reduced by 2.42-fold. These reductions indicate that VR training was significantly more effective in reducing radiophobia than traditional video training. VR enhanced knowledge retention, reduced radiophobia, increased safety confidence, and reduced fear about pursuing a career in nuclear medicine. Overall, VR created a safer working environment, and RT students responded more positively than the instruction videos. Consequently, a mixed-method study revealed key codes of engagement, easy understanding, memory, safety, confidence, learning experiences, implementation in the curriculum, and getting ready for clinical practice.

1. Introduction

The utilization of ionizing radiation in the healthcare sector has witnessed a global upsurge due to its wide-ranging applications and advancements in medical technology. These advancements have yielded significant progress, leading to the development of pioneering technologies and enhanced patient care. However, this upsurge is a significant factor contributing to the possibility of increased cumulative radiation doses to medical personnel [1,2,3,4]. Hence, there exists a growing imperative to mitigate the potential hazards associated with radiation, including tissue harm and the development of cancer [5]. A study by Boice et al. determined that 109,019 medical professionals linked with radiation procedures between 1965 and 1994 were at an elevated risk for lung cancer [6]. In the healthcare environment, occupational radiation exposure is well-established as a factor that elevates cancer risk and contributes to premature vascular aging and cataract formation [7].
Furthermore, Hayashi et al. revealed that medical staff lacked adequate knowledge of radiation protection principles [8]. Therefore, it is crucial to enhance training strategies to foster greater awareness of the risks associated with radiological procedures and improve comprehension of radiation safety principles [9]. In healthcare, ionizing radiation is widely used in medical imaging for various purposes, such as diagnosis, treatment, and monitoring. These procedures occur in different healthcare settings, including radiology departments, patient wards, intensive care units, and specialized areas like catheterization labs and operating rooms. Therefore, healthcare professionals must prioritize radiation safety to protect themselves and patients from the potential risks of radiation-induced cancer. Moreover, every institution’s management team is responsible for educating and implementing preventive strategies and operational norms [10].
As a result, digital learning, including virtual reality (VR), is on the rise in radiological technology (RT) education. Since VR provides risk-free, realistic, and highly effective training for medical staff, the healthcare industry is one of the key market sectors to adopt VR technology. Moreover, VR offers a unique and immersive learning experience that can significantly enhance the development of competencies among medical students. Consequently, RT students can benefit from virtual reality training simulating working in a hot radiopharmaceutical lab environment. Students can safely navigate around the hot lab before hands-on training in the hot lab. It is imperative to ensure that students can independently execute a comprehensive radiopharmaceutical preparation without any errors or reliance on external support.
Students must comprehensively understand the various concerns about patients, staff, and environmental safety when handling radioactive materials. Furthermore, they should be capable of effectively applying this knowledge in practical settings, particularly when preparing radiopharmaceuticals. Moreover, VR helps reduce the risk of errors and potential harm from radioactive materials by providing a controlled environment for practicing procedures. This focus on student safety is a significant advantage of incorporating VR into RT and continuing medical education (CME). In addition, VR can also be used for ongoing professional development, allowing medical staff to stay updated with the latest medical advancements and techniques and radiation safety best practices. Various studies have concluded that VR training enhances student engagement and increases their confidence [11,12]. While the promise of VR in RT education is substantial, it is essential to note that successful implementation requires careful design and integration into the curriculum. As technology advances, we can expect even more sophisticated and practical applications of VR in the RT field, ultimately leading to better-trained healthcare providers and skills improvement [13,14].
According to Good Radiopharmaceutical Practice, technetium-99m (6.0058 h half-life gamma emission) is typically prepared in hospital radiopharmacies. It is the most used radiopharmaceutical in nuclear medicine, mainly processed for 99mTc-methylene diphosphonate (99mTc-MDP) for bone scintigraphy. The ALARA (as low as reasonably achievable) principle, emphasizing minimal radiation exposure, is instilled throughout lectures and simulations in laboratory settings, where hands-on training is restricted due to potential health hazards. To overcome this limitation, virtual, augmented, and mixed-reality simulations have been developed to offer a risk-free environment for trainees. Because RT students’ practical training time is limited and restricted by access to hazardous materials, specialist equipment, and limited instructor resources. This study focuses on creating a virtual reality-enhanced, multi-layered meta-cognitive learning module specifically for the preparation and quality control of the radiopharmaceutical 99mTc MDP. RT students must be competent in preparing radiopharmaceuticals such as 99mTc-MDP and receive specialist training in preparing sterile and pyrogen-free pharmaceuticals. Radiation safety is a crucial element within RT training. VR simulation can be vital in educating students on the safety protocols associated with radiopharmaceutical preparation, aiming to decrease trainee radiation exposure.

2. Materials and Methods

2.1. Study Objectives

Create a virtual reality-enhanced, multi-layered meta-cognitive learning module as a teaching aid for RT students’ preparing 99mTc-MDP to Good Radiopharmaceutical Practice.
  • Understand the safety requirements for entering the radiopharmaceutical preparation laboratory (hot lab) and protection from radiation hazards.
  • Gain competency in the simulated preparation of 99mTc-MDP.
  • Gain competency in quality control of 99mTc-MDP.
  • Understand health and safety legislation requirements.

2.2. Study Location

  • Radiopharmaceutical Preparation Laboratory (hot lab), Department of Radiology, Faculty of Medicine, Prince of Songkla University, Thailand.
  • Laboratory 1 and 2, Radiological Technology Program, Department of Radiology, Faculty of Medicine, Prince of Songkla University, Thailand.

2.3. Quantitative Study: Study Design and Participants

In this study, a mixed-method intervention design incorporating both qualitative and quantitative components was employed. Initially, a non-randomized intervention using various teaching methodologies was executed, adopting a quantitative approach within a concurrent integrated design framework. The study’s second phase introduced a qualitative method to delve into significant insights from the observed phenomena. Qualitative data collection occurred after the VR interventions, aiming to uncover underlying mechanisms that might explain the quantitative findings. This included exploring participants’ experiences, effort, motivation, satisfaction, and adherence.
An assessment of radiological technology students preparing 99mTc-MDP radiopharmaceuticals was carried out. The importance of selecting participants without previous experience was highlighted by Karpicke and Blunt, studying retrieval practice highlighted in the study subject to measure the true impact of retrieval practice [15]. Additionally, Roediger and Butler emphasized the importance of accurately assessing prior knowledge to measure the effectiveness of new educational interventions [16]. This study sample size was based on research by Glenn D. Israel’s research, “Determining Sample Size” [17], and consists of 29 second-year and 31 third-year radiological technology students (n = 60). Both cohort groups were selected as having no previous experience of radiopharmaceutical preparation. The student selection criteria are detailed in the Supplementary Materials (Quantitative Study: Study Design and Participants section), the sample was selected using purposive sampling, which involves choosing individuals who meet specific criteria relevant to the research and who represent the studied population. One cohort (n = 30) with no experience preparing radiopharmaceuticals viewed video tutorials preparing 99mTc-MDP radiopharmaceuticals and the quality control procedure (the control group). The second cohort (n = 30) with no experience preparing radiopharmaceuticals viewed an immersive digital twin virtual reality simulation preparing 99mTc-MDP radiopharmaceuticals and the quality control procedure (the VR group). Both groups completed a questionnaire before and after viewing the video tutorial and the immersive digital twin virtual reality simulation. Both groups then completed a written test after viewing the video tutorial and completing the VR training. Furthermore, 1 month after the test, both groups completed the same test. Then, after 3 months, both cohort groups repeated the same test to re-evaluate the knowledge retention of both learning modules.
The first cohort viewed a video tutorial preparing 99mTc-MDP radiopharmaceutical on a computer, and the same tutorial was considered by the second cohort using an immersive digital twin virtual reality simulation (Figure 1). Using photogrammetry [18], numerous photographs of the radiopharmaceutical hot lab and 99mTc-MDP preparation and quality control were superimposed to form a 3D virtual reality model. Photogrammetry is the generation of a digital replica, sometimes called a “digital twin,” of a tangible entity or physical environment using authentic photographic imagery [19]. Therefore, it pertains to reconstructing forms and estimating the spatial location and measurements of objects represented in photographic photographs. In this instance, we utilized an Insta360 X3 Action Camera (Arashi Vision Inc., Irvine, CA, USA) and then edited the filming on a Dell Optiplex 7010SFF Plus i7-13700/16GB/512GB PCIe NVMe/Intel UHD Graphics 770/Windows 11 Pro (Dell Inc., Round Rock, TX, USA), with Adobe Premiere Pro 23.4 (Adobe Inc., San Jose, CA, USA). On completion, the photogrammetric 3D model was then viewed using an Oculus Quest 2 All-In-One VR Headset (128 GB) (Reality Labs, Menlo Park, CA, USA).
For the student evaluation questionnaires, the Likert scale psychometric scale was utilized for students to express their responses and feelings about the video tutorial and the immersive digital twin virtual reality simulation [20]. Likert-scale questionnaires can quickly gather data from large numbers of respondents, provide highly reliable personal ability estimates, establish the validity of interpretations made from the data, and compare, contrast, and combine the data with qualitative data-gathering methods like open-ended questions [21,22].

2.4. Ethical Considerations

Virtual reality training is gaining popularity in healthcare education as it has numerous advantages. However, some ethical concerns must be addressed. Improving the VR learning training experience typically requires acquiring and evaluating user data. Therefore, it is vital to prioritize participants’ privacy, safeguard all sensitive data, and ensure data usage rules are communicated to participants. Furthermore, participants should be able to opt out if any training component causes discomfort freely. Virtual reality can simulate highly immersive and lifelike experiences that can affect participants psychologically. Hence, virtual reality designers must consider the potential for discomfort, anxiety, or other adverse responses and develop modules with the user’s welfare as a primary concern. Brewer-Deluce et al. reported that 3D VR can create eyestrain, although it is not harmful and can be mitigated by enabling students to take short pauses [23]. Hence, this research spent 15 min in the virtual hot lab, split into three 5 min intervals, which avoided eyestrain, although slight confusion is inevitable when switching from virtual to real-world. Furthermore, VR training risk mitigation includes identifying, analyzing, and implementing solutions to minimize or eliminate risks. Virtual world technology requires less physical action, reducing the risk of accidents and enabling participation. Moreover, VR training settings should emphasize user safety, such as preventing real-world collisions, and psychological safety, allowing participants to express their emotions intellectually, emotionally, and physically [24].

2.5. Hot Lab: Assessment of Radiation Dose in Nuclear Medicine Controlled Areas during Preparation and Handling of 99mTc-MDP Radiopharmaceutical

Radiopharmaceuticals in vials, syringes, and sealed quality control sources expose RT staff to radiation. Therefore, radiation dose assessment in nuclear medicine-controlled areas is critical to ensuring the safety of patients and personnel working in these environments. The goal is to monitor and limit radiation exposure to prevent adverse health effects. Frequent radiation surveys and routine mapping of controlled areas are recommended to identify hotspots and ensure that radiation levels are within permissible limits. Also, the effectiveness of shielding should be regularly evaluated through surveys and assessments to provide adequate shielding to minimize radiation exposure to personnel and the public. Furthermore, the occupancy factor, which accounts for the time personnel spend in specific areas, should be considered to determine effective dose equivalents for individuals based on their work habits. All personnel should receive regular comprehensive training on radiation safety protocols and procedures, educating staff on the potential risks associated with radiation exposure and the importance of following established safety guidelines. Nuclear medicine professionals can obtain significant dosages of radiopharmaceuticals during preparation and administration [25,26]. Therefore, basic measures are necessary to reduce exposure, including preparation in controlled areas and purpose-built facilities and wearing personal protective equipment, which can reduce contamination hazards. In particular, the safe production of radiopharmaceuticals requires laminar airflow cabinets and adequately ventilated areas. Moreover, reducing contamination risks and ensuring patient safety requires efficient quality management. Therefore, radiopharmaceutical production must comply with strict adherence to radiation protection rules and pharmaceutical legislation. Abiding by protocols and supported with quality assurance and QC procedures [26,27].
Before use, the dosimeters were calibrated using a traceable 137Cs (662 keV) source, which closely matched the energy range used in nuclear medicine. The calibration process adhered to the guidelines set out by the Japanese Standards Association (JIS) Z 4345-3, which included measuring uncertainty [28].

2.6. Personnel Dose Assessment from Radiation Exposure during Preparation and Handling of 99mTc-MDP Radiopharmaceutical

Continually evaluating the level of radiation exposure in nuclear medicine-restricted areas is essential to guarantee the safety of staff and the general public. Publication 103 of the International Commission on Radiological Protection (ICRP) for nuclear medicine personnel set a limit of 20 mSv/y (averaged over five consecutive years) and 50 mSv/y in any given year [29]. Many countries have included International Atomic Energy Authority (IAEA) safety standards in their national safety regulations. The IAEA publication GSG-7 Occupational Radiation Protection also provides advice on controlling occupational exposure [30]. Furthermore, the IAEA publication Operational Guidance on Hospital Radiopharmacy provides several recommendations [31].
Before the VR study, staff personnel dose measurements were recorded in the hot lab over 3 days to evaluate the radiation exposure of the nuclear medicine scientist preparing 99mTc-MDP (Figure 2). The data were collected from measurements using nanoDot dosimeters calibrated for the personal dose equivalents Hp(10), Hp(0.07), and Hp(3), as well as technetium dosage estimation. The symbol Hp(d) represents the personal dosage equivalent in soft tissue at a specified depth (d), as defined by the International Commission on Radiation Units and Measurements (ICRU) [32]. The whole-body personal dose equivalent, also known as Hp(10), is a quantity used in individual monitoring to measure effective doses at a depth of 10 mm in tissue. The Hp(0.07) dosage equivalent, commonly known as the skin dose, is the dose to the skin, hands, and feet. Hp(3) is the dose to the skin at a depth of 3 mm for monitoring the eye lens [33].
Using a MicroStar reader (Landauer, Ltd., Glenwood, IL, USA), the nanoDot OSL-based medical dosimeter (Nagase Co., Ltd., Bangkok, Thailand), along with a control, were tested three times for Hp(10), Hp(0.07), and Hp(3). Three measurements from the nanoDot dosimeters that were not exposed to radionuclides were averaged and removed from the measurement readings from the exposed dosimeters to obtain one data point. It was found that the coefficients of variation (CV) of the doses measured by the exposed dosimeters were:
C V % =   S D D a v e   × 100
where S D   is the standard deviation of the dose recorded, and D a v e   is the average exposure measured by each dosimeter.
The recording of personnel dose equivalents was conducted using nanoDot [34], optically stimulated luminescence dosimeters (OSLD) (Landauer nanoDot™), and the dose coefficients of variance (CV) were calculated using Equation (2). Additionally, the reading before preparation and handling of 99mTc-MDP radiopharmaceutical was used as the background value.
The mean of three dosimeter readings was used. Doses used the air kerma value:
D o s e S v = O S L D   r e a d i n g   v a l u e   ( c o u n t ) R e a d e r   c a l i b r a t i o n   f a c t o r c o u n t S v ×   D o s i m e t e r   s e n s i t i v i t y   f a c t o r
The correction for the OSLD was calculated using the following equation:
D = D 0 × K s , i × K Q  
where D 0 is the air kerma, K s , i is the relative sensitivity correction factor between the individual elements of OSLD, and K Q is the energy correction factor in CT [35,36].

2.7. Evaluation of Virtual Reality as a Technological Tool

2.7.1. Technology Acceptance Model (TAM) Development of Virtual World Technology Learning Media

The main goal of the technology acceptance model (TAM) is to provide insight into the process of technology acceptance and forecasting. To explain technology’s success to potential practitioners on pre-system implementation measures, in this instance, VR training [37]. Based on social psychology and behavioral science, TAM maintains that perceived utility and simplicity of use impact technology adoption. Among TAM’s essential elements are:
  • Perceived usefulness refers to users’ subjective evaluation of the extent to which technology enhances performance or facilitates job processes. Individuals are more likely to embrace and utilize technologies that offer significant value.
  • Perceived ease of use: the ease or difficulty of utilizing a certain technology; higher acceptance levels are characterized by ease of use.
  • Behavioral intention refers to a user’s expressed willingness or intention to employ a certain technology. According to TAM, the perceived usefulness and simplicity impact this objective.
The present study employed TAM to examine the acceptance or rejection of VR training for simulating radiopharmaceutical production among users. Hence, to assist in designing a VR training module, we explored the factors that could influence users’ acceptance of the module by investigating the aspects that might affect the user acceptability of the VR training module to understand the numerous factors affecting students’ expectations and build a training package that supports their engagement. In order to achieve this objective, a cohort of 60 students studying RT was selected to participate in an online survey employing a Likert scale. The scale ranged from 1, indicating significant disagreement, to 5, indicating substantial agreement.

2.7.2. Content Validation: Content Validity Index (CVI)

The item content validity index (I–CVI) was applied to appraise the questionnaire’s suitability and specificity [35]. This process entails comprehensively analyzing the questions to ascertain their suitability in assessing the intended concept. Content validity pertains to the extent to which the questions in a questionnaire accurately reflect the aims and goals of the VR training module. The following steps were utilized to perform the content validation and additionally calculate the content validity index:
  • Define the construct clearly to measure.
  • Select experts and identify a panel of experts knowledgeable in VR.
  • Item generation creates questions for the measuring tool based on the construct.
  • Content review by experts panel evaluation of the questions to ensure they are clear, relevant, and indicative of the VR training module goals.
  • Gather expert advice on rephrasing, adding, or eliminating items to improve questionnaire validity.
  • Content validity index (CVI) calculation: CVI is calculated separately for each item and then averaged across all items, using the item content validity index (I-CVI).
I C V I = N u m b e r   o f   e x p e r t s   a g r e e i n g   t h a t   t h e   i t e m   i s   t h e   c o n t e n t     v a l i d T o t a l   n u m b e r   o f   e x p e r t s
In general, an I-CVI of 0.78 or greater is deemed acceptable.

2.8. Comprehensive Evaluation of the Effectiveness of VR-Assisted Teaching in Enhancing Students’ Learning Experiences

Prior to the students undertaking the video and VR training, a 5 min multi-choice pre-test was used to measure the students’ learning on completion of reading the study notes. The test questions were specifically aligned to the manufacture and quality control of 99mTc-MDP radiopharmaceutical in compliance with radiation protection rules and protocols. After the student groups had completed the video and VR training, the students were given the same 5 min test. To evaluate student knowledge retention, a 5 min test was given to the students 1 month after the training. Furthermore, the students took the same test 3 months after the training to evaluate their long-term knowledge retention. Moreover, VR has been shown to enhance memory retention by creating immersive, context-rich, interactive, emotionally engaging, and personalized learning experiences that facilitate deeper cognitive processing and recall [38,39,40,41].

2.9. Gamification Supporting Practical and Experimental Learning

Gamification entails incorporating game principles, methods, and game theory into areas that are not naturally related to gaming. Using gamification in medical education has shown considerable promise in enhancing motivation and engagement among medical students [42]. This study applied gamification to simulate quality control spotting to assess radiochemical purity. This process includes accurately transferring 10 µL of 99mTc-MDP radiopharmaceutical with a hypodermic needle onto chromatography paper coated with instant thin layer chromatography medium (iTLC). The objective is to increase spotting accuracy, increase student confidence, and reduce spillage and contamination.

2.10. Student Engagement: Physical Assessment Using Blood Pressure and Heart Rate

On the day of the study, the participants were assigned randomly to the control group (30 students) or the VR group (30 students). All participants were given verbal instructions, informing them they would participate in a control group video or VR immersive training. All participants were directed that they should not consume any food or caffeinated beverages one hour prior to the study. During the training exercise, the control and VR groups wore a blood pressure cuff (Omron, utilized for measuring blood pressure and heart rates). Similarly, the participants wore a blood pressure cuff during pre-tests and post-tests. Upon completion, all participants remained in the laboratory for 10 min before a comprehensive debriefing regarding the nature and objectives of the study. A diagram illustrating the experimental procedure is depicted in Figure 3. Existing approaches to assessing student participation in educational activities often utilize several indicators such as eye movement, facial expressions, gestures, posture, and physiological and neurological sensors integrated into wearable devices, which capture essential physiological characteristics [43,44]. According to a review by Bustos-López et al., heart rate is a significant parameter used to detect and track student engagement during educational events. Skin conductance, temperature, and conductivity are also tremendous characteristics to consider [45]. Ribeiro et al. investigated the impact of a VR game on physiological parameters, including heart rate, heart rate variability, oxygen saturation, and blood pressure, and found that it had a favorable effect on these variables [46]. The impact of immersive digital twin VR simulations on physiological parameters enables the link between heart rate and blood pressure to be studied and the development of protocols to conduct navigation experiments [47]. This aids researchers in exploring new questions and refining theories on spatial abilities, strategies, and performance.

2.11. Assessment of Satisfaction in VR-Enhanced Learning Experiences: Prioritizing Educational Significance and Practicality in VR-Assisted Teaching

Creating a student VR training satisfaction questionnaire requires careful consideration of the areas to measure, the questions to ask for constructive feedback, and the questionnaire structure. The main steps were to define the questionnaire’s purpose and objectives. Review existing student satisfaction survey themes and questions for inspiration. Ensure the question types are closed-ended Likert scale queries (strongly agree to disagree strongly). Use simple questions that are easily comprehensible. Conduct a pilot test to find any phrasing, clarity, or question flow concerns. Include positive and negative remarks for the whole picture. Maintain anonymity and confidentiality, and convey that replies are private and will not identify individual students. Use online survey technologies, ensuring the platform is user-friendly. Set realistic expectations and inform the students of the survey objectives and how their input will enhance the VR training material. Analyze and act on the results to find patterns and opportunities for improvement. These steps enabled the development of a well-structured and effective student satisfaction questionnaire to provide valuable insights to enhance the student VR training module experience.

2.12. VR Training Module Questionnaire Analysis

The Likert scale is one of the most basic and widely used psychometric tools in sociology, psychology, information systems, politics, economics, research, and other fields for measuring attitudes, opinions, and behaviors. The construction of the Likert scale is straightforward and is likely to yield a scale with excellent reliability. Furthermore, from the standpoint of participants, it is clear to comprehend and fulfill. However, it might be challenging to establish validity, and this scale needs more reproducibility. Additionally, participants may avoid selecting an extreme response, resulting in a central bias tendency. Participants may respond to the questions by agreeing or disagreeing to appease the experimenter, known as acquiescence bias. This research questionnaire has 5 points: strongly agree, agree, neutral, disagree, and strongly disagree, and a numerical score is assigned to each question to quantify the strength of response data. The response anchors strongly agree and strongly disagree at the ends of the scale, whereas a neutral item typically represents the midway in the middle of the 5-point scale.

2.12.1. Radiophobia and Safety Confidence

Radiophobia refers to an unreasonable fear or anxiety associated explicitly with being exposed to radiation. It might appear as an overabundance of fears, anxieties, or avoidance actions in reaction to the perceived danger of radiation, even when the amount of exposure is minor or insignificant.

2.12.2. Assessment of Cybersickness in VR-Assisted Teaching Materials

Cybersickness involves symptoms similar to motion sickness, with nausea, eye strain, and disorientation symptoms that are more pronounced in virtual and augmented reality settings. Therefore, a cybersickness questionnaire gathered information on cybersickness during VR-assisted teaching to understand the experiences and challenges faced by the participants. Ask participants about their VR experience and whether they have encountered cybersickness, and if they have, inquire about the factors they believe contributed to their cybersickness.

2.13. Qualitative Study of Students’ Perspectives

The objective of the qualitative study was to better understand the perspectives held by the students who participated in VR training. Thematic analysis was used to identify patterns in the data and find themes. The qualitative study and analysis adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ), a comprehensive checklist designed for interviews and focus groups [48].
The study employed purposive and snowball sampling methods, inviting eligible participants to encourage others to join the focus group (n = 10). Focus group interviews, lasting approximately 50–60 min on average, were followed by a semi-structured guide [49,50,51]. This guide explored participants’ perspectives on using VR in education and perceived benefits and drawbacks. Open-ended probing techniques were used to delve deeper into participants’ responses. Data collection continued until no new themes emerged. All interviews were conducted face-to-face and audio-recorded for analysis.

2.14. Statistical Analysis

In this study, each experiment was independently conducted with three replicates. The results from these experiments were presented as mean values accompanied by standard deviation. Furthermore, the normality of the data was assessed using residuals, while the homogeneity of variances between groups was evaluated by analyzing the variance within each group.
Statistical analyses were performed using Student’s t-test and one-way or two-way analysis of variance (ANOVA), followed by a Student Newman–Keuls post hoc test to determine statistical significance. The levels of significance were indicated as follows: ns (not significant) if p > 0.05, * p ≤ 0.05, ** p ≤ 0.01, *** p ≤ 0.001, and **** p ≤ 0.0001. All statistical analyses were conducted using GraphPad Prism 10.0 software (GraphPad Software Inc., Boston, MA, USA).

3. Results

3.1. Enhancing Virtual Reality Immersive Learning: Implementing Revised Bloom’s Taxonomy in VR Training

The Revised Bloom’s Taxonomy framework supports an immersive learning strategy by mapping taxonomy steps, fostering skills, and developing student confidence (Figure 4) [52,53]. An active learning approach enhances student involvement, enabling improved cognitive processes. VR provides instant feedback and allows for user adaptation in a safe environment. Furthermore, VR facilitates repetition at a low cost with minimal resources, helping students master protocols at each cognitive level and achieve accuracy safely. Moreover, when implemented through VR, this revised taxonomy improves technical proficiency and procedural knowledge and reduces radiophobia by familiarizing students with high-risk situations in a controlled and safe setting. As a result, VR equips students to react more calmly and knowledgeably in real-life hot lab situations.

3.2. Hot Lab: Assessment of Radiation Dose in Nuclear Medicine Controlled Areas during Preparation and Handling of 99mTc-MDP Radiopharmaceutical

The survey of the controlled areas is shown in Figure 5 to evaluate the potential risk to medical staff and trainees. The highest dose rate of 600 µSv/h was recorded immediately above the generator. To meet the occupational hourly dose rate of 25 μSv/h, radiochemists and RT students should not expose themselves to more than 0.03 h (2.4 min) in the location above the 99Mo/99mTc generator. The dose calibrator working area behind the lead shielding radiation dose is 3.00 µSv/h, with a maximum working time in this location of 8.32 h to comply with the occupational hourly dose rate of 25 μSv/h. The results of the radiation dose survey of the nuclear medicine-controlled areas are shown in Supplementary Table S1.

3.3. Personnel Radiation Exposure during Preparation and Handling of 99mTc-MDP Radiopharmaceutical

The nuclear medicine staff personnel dose measurements recorded in the hot lab over 3 days are shown in Figure 6. As expected, the Hp(0.07) dosage equivalent to the hands is significantly different from Hp(3) and Hp(10). The dose equivalents are based on a personal exposure of 30 min. Potentially, RT students’ training in the preparation and quality control of 99mTc-MDP can receive a Hp(0.07) personal dose of 13.14 μSv/h in 30 min of training, increasing to 26.27 μSv/h after 1 h of training. The current personal dose limits whole body Hp(10) ≤ 500 mSv/y, Hp(0.07) skin dose, hands, and feet ≤ 500 mSv/y and the eyes Hp(3) 20 mSv/y (average in 5 years). VR training for the preparation and quality control of 99mTc-MDP mitigates students’ receiving a personal dose.

3.4. Evaluation of Virtual Reality as a Technological Tool: Technology Acceptance Model (TAM) and Content Validation: Content Validity Index (CVI)

The results of the technology acceptance model (Table S2) reveal that an overall I-CVI score of 0.93 is considered good, confirming that the experts believe the questions are relevant. In addition, the substantial content validity index score means that the questions are acceptable and appropriate. The CVI scale ranges from 1 to 5; the score of 4.93 is close to the maximum score and suggests that the students perceive a high value in VR training. Additionally, the TAM score of 4.90 is generally considered relatively high, indicative of solid acceptance and a positive attitude towards VR training, and highly accepted as a training aid.

3.5. VR-Assisted Teaching Can Improve Students’ Learning Effectiveness (Pre-Test, Same Day Post-Test, after 1 Month, and after 3 Months)

The video training group (control group) and the VR training group were given a pre-test (PT) of 10 multiple-choice questions, each with 5 probable answers. As shown in Figure 7A, the PT’s control group and VR group test results had a similar mean score of ~34% correct answers. Immediately after the PT, both cohort groups were given lecture notes detailing the concise core knowledge of preparing and carrying out quality control of 99mTc-MDP to study for 15 min. Afterward, both groups were given the same 10 multiple-choice questions on the lecture notes. The results shown as PS in Figure 7A indicate an increased mean score of ~65%.
Then, the control group watched 3 × 5 min training videos on the preparation and quality control of 99mTc-MDP on a computer. Similarly, the VR group undertook 3 × 5 min training with immersive digital twin virtual reality simulation preparation and the quality control of 99mTc-MDP. On completion, both cohort groups were tested again with the same questions. The results indicate that the control group score (C-A) was ~78%, and the VR group test score (VR-A) was ~89%. A further test with the same 10 multiple-choice questions was conducted 1 month after the video and VR training to ascertain the knowledge retention of both groups. The control group score (C-1M) was ~68%, a reduction of ~13%, and the VR group score (VR-1M) of ~88%, in comparison, was only marginally lower after 1 month. Both cohort groups carried out a further knowledge retention test with the same questions 3 months after the video and VR training. The control group score (C-3M) of ~59% was significantly lower. Whereas the VR group score (VR-3M) of ~90% was marginally higher than previous scores.
The distribution of the correct answers to the three tests is illustrated in Figure 7, the control group’s correct answers are shown in Figure 7B–D, and the VR group’s correct answers are shown in Figure 7E–G. Immediately after the self-study, the test resulted in 25% of all the participant groups scoring > 80%. After the control group completed the video training, 57% scored > 80% (Figure 7B). In comparison, after completing the VR training, 87% of the VR group scored > 80%, 1.52-fold more significantly than the control group (Figure 7E). The control group knowledge retention test 1 month after the achieved 80% or above scores declined to 37%, representing a 2.16-fold decrease (Figure 7C). The VR group scored after 1 month, achieving scores of > 80% and declined to 80%, a 1.09-fold reduction (Figure 7F). All participants were tested again 3 months after the initial training using the same 10 multi-choice questions. The control group’s correct answers declined further, with only 13% scoring >80% (Figure 7D). The group’s decline in incorrect answers from the test after training to the 3-month test resulted in a 4.38-fold decrease. In contrast, 87% of the VR group scored > 80% in the 3-month test (Figure 7G). This signifies that a 6.69-fold more significant number of VR group participants scored >80% than the control group.

3.6. Supporting Practical and Experimental Gamification of Learning

The objective of the gamification was to represent quality control spotting to evaluate the radiochemical purity, which involves the precise transfer of a minute quantity of radiopharmaceutical substance using a hypodermic needle onto the surface of chromatography paper using instant thin layer chromatography medium (iTLC). The control group scored marginally higher than the VR group, as shown in Figure 8; this could be because the control group used a computer mouse to manipulate the cursor while the VR group used VR hand controllers. Both groups were more conversant with using a computer mouse than the VR hand controllers. Hence, the control group scored higher than the VR group.

3.7. Student Engagement: Physical Assessment of Blood Pressure and Heart Rate

Before the study, participants were told to abstain from alcohol, caffeine, nicotine, chocolate, soda, and energy drinks and engage in vigorous physical activity [54]. During the procedures, participants wore a cuff blood pressure and pulse monitor (Omicron). The systolic blood pressures and heart rates recorded pre-test (BH-PT) for the control and VR groups were very similar (Figure 9). However, during the first VR module, the VR group’s systolic, diastolic, and heart rates rose markedly compared to the control group. Furthermore, VR modules 2 and 3 recorded elevated systolic, diastolic, and heart rates. Implying the immersive simulated radiopharmaceutical preparation was life-like and stimulated the participant’s senses more than the video training. VR can stimulate the users’ vision, hearing, and, in some cases, even touch.

3.8. Satisfaction in VR-Enhanced Learning Experience: Prioritizing Educational Significance and Practicality in VR-Assisted Teaching

The results of the five-question Likert questionnaire, evaluating students’ satisfaction with both video and VR training, are illustrated in Supplementary Table S3. It is evident that there is a significant difference (*** p ≤ 0.001) in six out of the eight questions. VR training was deemed more understandable, engaging, and accessible to follow. Moreover, it enhanced understanding with better examples and exercises and achieved greater confidence compared to video training.

3.9. VR Training Module Questionnaire Analysis

3.9.1. Radiophobia

The analysis of the radiation phobia five-question Likert questionnaire is illustrated in Figure 10A. Before the training, both cohorts scored 4.21, strongly agreeing they were concerned about radiation exposure and their health. After watching the radiolabeling training videos, the control group response indicated a 1.52-fold decrease. In comparison, the VR group reduced 2.42-fold, suggesting that VR training was more successful than video training in easing students’ radiation and health worries (Figure 10B). Both student groups before training had concerns about radiation exposure during training, scoring 3.45. The control group was 1.41-fold less anxious after the video training, and the VR training group scored 2.90-fold lower. Verifying that the VR training significantly reduced radiation exposure concerns (36.23% difference). Regarding the fear of radiation exposure mistakes, both groups scored 4.14 before the training, indicating substantial agreement with the question. After the training, the control group was less likely to be influenced by their worries, with a 1.33-fold reduction. The VR training group’s concerns about working in radiological technology decreased 1.98-fold, significantly more different than the control group. Students discussing their concerns about radiation exposure with each other or instructors in both groups scored 4.12. After the video training, the control group’s score decreased by 1.06-fold, and the VR group decreased by 1.36-fold. Suggesting that both training modes reduced students’ worries, with the VR simulation dispelling the students’ fears better than the control group video training, with a 20.6% difference (Figure 10B).

3.9.2. Safety Confidence

Safety is a critical factor in the training of RT students. Figure 10C illustrates that the control group’s confidence in detecting and correcting errors in radiation safety practices during their training increased 2.63-fold after the video and hands-on training. In comparison, the VR group’s confidence increased 3.06-fold, a difference of 42.67%. As both cohorts completed training, the scores regarding the student’s ability to understand and implement safety protocols increased. The control group score increased 1.13-fold, whereas the VR group score was higher, with a difference of 42.93% between the two groups (Figure 10D). As can be seen, the VR training resulted in the RT students feeling more confident about radiation safety than the training videos. In summary, radiological technology students responded more positively to the VR immersive learning experience addressing radiation safety than the computer-based training videos (control group), as the VR-simulated hot lab was more like the real-world hot lab setting than the videos.

3.9.3. Assessment of Cybersickness in VR-Assisted Teaching Materials

The assessment of cybersickness for the control group and VR training group is illustrated in Supplementary Figure S1. As can be seen, there is little difference between the control group and the VR group, with the VR group scoring marginally higher, as expected. Overall, the assessment indicates that cybersickness was not an issue in this study.

3.10. Qualitative Study of Students’ Perspectives

Table 1 encapsulates the thematic analysis of students’ feedback, revealing their empowerment through immersive virtual reality learning in radiological technology. The analysis follows the method outlined by Braun and Clarke [55]. Participants expressed a number of positive experiences and perspectives regarding using VR in their learning environment. They highlighted how VR enhances their focus and engagement, making learning more immersive and interactive. Additionally, VR provides a safe space for trial and error, facilitating better understanding and retention of information. Participants appreciated the opportunity to familiarize themselves with the processes and protocols before practical sessions, contributing to increased confidence and reduced anxiety in real-life scenarios. The interactive nature of VR, including features like recorded voices, enhances the learning experience and makes it more enjoyable and memorable. Participants also emphasized the importance of integrating VR into the curriculum to maximize learning opportunities and practice sessions, particularly in settings like hot lab practical and clinical practice, where opportunities for hands-on experience may be limited. Overall, participants recognized VR as a valuable tool for enhancing learning outcomes and preparing them for real-world scenarios in radiological technology education.
Participants identified several challenges and limitations associated with using VR in education. Cost constraints may limit the feasibility of training multiple students simultaneously, potentially leading to unequal learning opportunities and delays in completing learning materials. Additionally, issues such as discomfort or motion sickness experienced by some users may impact the duration and effectiveness of training sessions. Despite offering engaging experiences, VR interactions may only partially replicate real-world interactions, potentially limiting the depth of learning and skill development. Furthermore, VR simulations may not always align with individual learning methods or preferences, reducing their efficacy for certain students. These considerations highlight the importance of addressing challenges and optimizing the use of VR technology to maximize its benefits in educational settings.

4. Discussion

The International Atomic Energy Authority report IAEA-TCS-39 noted that most Member States have a severe shortage of radiopharmacists, and other countries have no trained pharmacists with radiopharmacy competence. Primary issues included inadequate education and training, particularly clinical training that was poorly planned and managed, and lack of professional recognition [56]. An IAEA technical meeting in 2021 assessed the strengths, weaknesses, and possible radiation protection education and training alternatives, recognizing the need for improved global radiation protection and health safety education [57]. Similarly, a recent EU study on health professionals, radiation safety education, and training found a major resource deficit [58]. In addition, Newhauser et al. further claimed that most training programs are inadequate, declining, or nonexistent. Therefore, to meet society’s needs, appropriate resources must be allocated to recruit and retain workers [59,60], and training programs must be developed that encourage careers in radiation profession fields [61].
Moreover, according to the latest United Nations estimates in 2019, there are 83 million new births annually, and it predicts that the global population will increase to 8.6 billion by 2030, 9.8 billion by 2050, and by 2100, 11.2 billion by [62]. Healthcare services, especially nuclear medicine departments, will be strained. Equipment and skilled staff will become scarce as more patients need diagnostic and therapeutic services, resulting in equipment and competent staff constraints, requiring significant infrastructure investment and hiring and training staff to handle the extra load. Additionally, workforce development, focused training, and incentives are crucial to attracting and retaining nuclear medicine professionals. Resources are required to promote the nuclear medicine profession and maintain trained staff.
Consequently, VR has a role in training; a meta-analysis and systematic review of VR in medical education for healthcare professionals by Kim et al. found significant improvements in competence and satisfaction [63]. Similarly, this research demonstrated that VR could improve student learning. The VR learning effectiveness was demonstrated by 87% of the VR group scoring >80% in the 3-month test (Figure 7G), which was 6.69-fold higher than that of the control group. We gave radiological technology students the same exam one month and three months after training to determine the durability of learning from the video tutorials and immersive digital VR simulation. Research shows that periodic testing can accurately evaluate long-term retention and teaching strategies. Roediger and Butler found that repeated testing improves and assesses knowledge retention. They found that retrieval practice via testing improves long-term memory retention more than passive reviews [16]. Similarly, Brown et al. found that spacing learning and testing sessions, known as the “spacing effect,” improve learning retention [64]. Administering the exam one month and three months later lets us see the spacing effect in action and whether the immersive VR simulation helps recall complex procedural information better than video tutorials. Assessing knowledge retention one month and three months after training is a technique that follows educational research and seeks to determine the long-term efficacy of video tutorials against immersive VR simulations in radiological technology instruction. The results confirm our hypothesis that VR can improve long-term memory of complicated procedural knowledge.
Furthermore, VR learning has been shown to significantly impact confidence levels amongst medical students concerning their understanding of radiation safety matters, with student feedback supporting integrating VR simulation-based learning into the radiography and medical curriculum alongside lectures [65]. Miguel-Alonso et al. found that immersive VR influences learning via affective and cognitive processes. That immersion and enjoyment in VR immersive features resulted in better emotional learning results [66,67]. Moreover, this study determined that VR training enhanced the student’s learning experience and was valued more highly than video training by RT students (Figure 7). Similarly, Chávez et al. established that medical students’ perception of immersive virtual reality (IVR) gamification helped the most to improve their learning performance [68]. Additionally, Liu et al. investigated the long-term effects of IVR teaching on undergraduate healthcare students’ learning outcomes and found that undergraduates had better learning outcomes and experiences [69]. Moreover, Karpicke and Blunt demonstrated that retrieval practice (e.g., through tests) enhances learning more effectively than merely reviewing the information. They showed that students who were tested retained information longer than those who only studied the material [15]. Similarly, Bjork and Bjork discussed the concept of “desirable difficulties,” where challenging learning activities, such as spaced repetition and varied practice, lead to better long-term retention [70] as well as Agarwal et al. emphasized the benefits of frequent quizzing on students’ long-term retention of information, noting that quizzes can serve both as a learning tool and a means to assess knowledge retention [71].
Moreover, medical mistakes endanger patients and healthcare professionals, and the fear of sanctions, inadequate feedback, and organizational culture impede reporting [72]. VR offers safe practice for students and healthcare staff, reducing risks and psychological factors such as anxiety that impact clinical performance [73]. Inexperienced students entering clinical practice face challenges due to limited clinical knowledge and skill competency, which increases anxiety and decreases self-confidence. Engaging in a secure VR learning environment can enhance students’ ability to do tasks with more composure and competence, as VR increases the efficiency of the learning process [74]. Real-world objects can be represented visually in a VR environment; settings can be created that users cannot visit in real life due to various limitations with real-time interactions [75].
Notably, VR has been shown to improve learning strategies, life aims, emotional clarity, and emotional healing in research by Redondo-Rodrguez et al., which encouraged university students to study [76]. Gan et al. investigated medical students’ long-term practice performance after VR simulator teaching; after a one-year follow-up, students exhibited more defined career goals, an engaged willingness to learn, improved surgical clinical practice instructor assessments, and higher physical exam scores [77]. Furthermore, Thompson et al. verified that VR participants had a more favorable learning outcome in evaluating content and a more pronounced enhancement in their cognitive frameworks, in contrast to individuals who engaged in a two-dimensional game [78]. Through its immersive and interactive nature, VR engages many senses and delivers context-rich learning settings, improving memory retention. VR has been found to increase memory and recall. An immersive experience that facilitates cognitive processing and contextual learning helps VR users retain knowledge better than conventional approaches, according to Essoe et al. [79]. According to recent research, augmented reality-based strategies may improve knowledge acquisition by around 18% for standard augmented reality and almost 25% for enhanced augmented reality [80]. We believe that VR-enhanced multi-layered metacognitive learning modules are suited for nuclear medicine radiological hot lab simulation. This strategy utilizes VR’s immersive and interactive nature, incorporates varied learning demands, improves metacognition, and promotes sustainable education. Research validates this strategy and may significantly enhance educational results in this specialized and vital field [81,82,83]. In addition, repeated testing over extended periods is essential to evaluate the effectiveness of educational interventions in fostering long-term knowledge retention. The findings from this study could inform best practices in educational methodologies for radiological technology, potentially influencing curriculum design to favor methods that ensure sustained competence and confidence in preparing radiopharmaceuticals.
Radiation phobia can result from ignorance, a lack of understanding, or exaggerated fears, with many individuals developing radiophobia without a rational explanation, causing anxiety. Historical and sociopsychological factors have made radiation a feared process [84]. Moreover, despite the benefits of nuclear medicine, the general public’s perception of ionizing radiation remains negative [85,86]. For example, a longitudinal amyloid-PET study found that 26% of refusals to participate were because of radiation-related concerns [87]. As shown in Figure 10A, the radiophobia Likert scale verified that VR training in this study reduced student fear more than video training and helped students overcome anxiety better.
Additionally, the Kirkpatrick evaluation model was utilized to assess the training [88] and initial reactions to a training graded at the reaction level. The results ascertained that the VR training received a higher score than the video training (Figure 7). Learning is the next level in the Kirkpatrick model, which is to ascertain if the learning objectives were fulfilled and verify that the students gained new knowledge and skills and how attitudes and beliefs were changed. The next level, behavior, examined how training affects performance, with the results indicating that VR training improved knowledge and improved the implementation of quality control procedures. Moreover, qualitative approaches are employed in academic medical education to study complex phenomena [89]. This study’s qualitative research identified the students’ perspectives after the VR training, emphasizing the significance of their opinions, with “more engaged”, “easier to understand”, “better memory”, “a bridge between theoretical and practical”, and “add VR training to the curriculum” some of the key patterns.

5. Conclusions

The primary objective of healthcare education is to cultivate competent and empathetic healthcare practitioners capable of delivering safe and effective care to those seeking medical treatment. Utilizing virtual reality simulation has substantially impacted healthcare, as it has focused on maximizing its application to improve overall safety, efficacy, and productivity. Virtual reality helps students become acquainted with the hot lab environment before clinical practice to reduce radiophobia and safety concerns. This capability enabled students to pre-emptively practice aspects of radiopharmaceutical preparation that they previously saw as daunting. By giving students a way to bridge the gap between theory and practice, virtual reality allowed users to engage in simulated radiopharmaceutical preparation that made it possible for “safe fails”. This experience is critical in building confidence and developing the skills required. Moreover, this study verified that 3 months after the virtual reality training, the student knowledge retention was significantly more significant than the video training. The VR group had a 6.69-fold more significant number of virtual reality group participants, scoring >80% than the control group. Radiological technology students reacted more favorably to the VR immersive learning experience addressing radiation safety than the computer-based training (control group), as the VR-simulated hot lab was more comparable to the real-world hot lab scenario than the videos. Virtual reality as a pedagogical tool enhanced students’ information retention, active participation, performance, and satisfaction, and students requested it be added to the curriculum.
Developing a VR pedagogical tool for hot laboratory simulation shows promise for enhancing learning outcomes and reducing radiophobia in nuclear science education. By providing immersive experiences and addressing psychological barriers, VR has the potential to revolutionize the way complex subjects are taught and understood. Further research and refinement of VR educational tools are warranted to maximize their impact on learning and student engagement. Repeated testing over extended periods is essential to evaluate the effectiveness of educational interventions in fostering long-term knowledge retention. The findings from this study could inform best practices in educational methodologies for radiological technology, potentially influencing curriculum design to favor methods that ensure sustained competence and confidence in preparing radiopharmaceuticals.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/app14125041/s1. Table S1: Assessment of radiation dose in nuclear medicine controlled areas during preparation and handling of 99mTc-MDP radiopharmaceutical in hot lab. The data are presented as mean ± standard deviation (n = 3); Table S2: Evaluation of virtual reality as a technological tool: technology acceptance model (TAM) and content validation: content validity index (CVI). The data are presented as mean ± standard deviation (n = 3); Table S3: Satisfaction in VR-enhanced learning experience: prioritizing educational significance and practicality in VR-assisted teaching. The data are presented as mean ± standard deviation (n = 30). Statistical significance is indicated as follows: ** p ≤ 0.01 and *** p ≤ 0.001; Figure S1: Assessment of the cybersickness questionnaire in the training module after training simulations of the control group and VR group. The questionnaire consists of 5 points: strongly agree, agree, neutral, disagree, and strongly disagree. A numerical score is assigned to each question to quantify the strength of the response data. The response anchors “strongly agree” and “strongly disagree” are positioned at the ends of the scale. At the same time, a neutral item typically represents the midway point in the middle of the 5-point scale. Data are presented as mean ± standard deviation (n = 30).

Author Contributions

Conceptualization, S.K.M.; methodology, S.K.M. and J.C.; software, S.K.M., J.C., N.S., P.K., V.K., A.H., N.N. and W.D.; validation, S.K.M., J.C., N.S., P.K., V.K., A.H., N.N. and W.D.; formal analysis, S.K.M., N.S., P.K., V.K., A.H., N.N. and W.D.; investigation, S.K.M., J.C., N.S., P.K., V.K., A.H., N.N. and W.D.; resources, S.K.M. and J.C.; data curation, S.K.M., N.S., P.K., V.K., A.H., N.N. and W.D.; writing—original draft preparation, S.K.M.; writing—review and editing, S.K.M.; visualization, S.K.M., N.S., P.K., V.K., A.H., N.N. and W.D.; supervision, S.K.M.; project administration, S.K.M.; funding acquisition, S.K.M. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Research and Development Office (RDO) at Prince of Songkla University. The research was financially supported by the Faculty of Medicine, Prince of Songkla University (MR-PSU: 66-07-21-314) [Grant number 66-065-1, 6 October 2566].

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Office of Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University, Thailand (REC.66-373-7-2, 26 September 2023).

Informed Consent Statement

Consent to participate prior to the study: All participants provided informed consent. Our research was conducted using standard medical education procedures. All participants were informed that the study would not affect the procedures, perceptions, or efficacy of their training. Consent for publication: All participants were informed that the study data published would not be sensitive or disclose any participant’s details.

Data Availability Statement

The anonymized data set is available upon request from the corresponding author. Access to comprehensive personal information is restricted due to ethical and legal considerations. The data availability should be requested, reviewed with the corresponding author, and approved by the Office of Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University.

Acknowledgments

The authors would like to express their deepest gratitude to all participants for their collaboration and their sincere appreciation to the radiological technology staff of the Department of Radiology in the Faculty of Medicine at Prince Songkla University for their significant assistance in conducting various technical operations.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Hauptmann, M.; Byrnes, G.; Cardis, E.; Bernier, M.-O.; Blettner, M.; Dabin, J.; Engels, H.; Istad, T.S.; Johansen, C.; Kaijser, M. Brain Cancer after Radiation Exposure from CT Examinations of Children and Young Adults: Results from the EPI-CT Cohort Study. Lancet Oncol. 2023, 24, 45–53. [Google Scholar] [CrossRef] [PubMed]
  2. Lee, W.J.; Choi, Y.; Ko, S.; Cha, E.S.; Kim, J.; Kim, Y.M.; Kong, K.A.; Seo, S.; Bang, Y.J.; Ha, Y.W. Projected Lifetime Cancer Risks from Occupational Radiation Exposure among Diagnostic Medical Radiation Workers in South Korea. BMC Cancer 2018, 18, 1–10. [Google Scholar] [CrossRef] [PubMed]
  3. Güngördü, N.; Kurtul, S.; Özdil, A.; Erdoğan, M.S. Does Occupational Ionizing Radiation Exposure in Healthcare Workers Affect Their Hematological Parameters? Arch. Environ. Occup. Health 2023, 78, 80–87. [Google Scholar] [CrossRef] [PubMed]
  4. Wang, P.; Li, Y.-W.; Lu, X.; Liu, Y.; Tian, X.-L.; Gao, L.; Liu, Q.-J.; Fan, L.; Tian, M. Low-Dose Ionizing Radiation: Effects on the Proliferation and Migration of Lens Epithelial Cells via Activation of the Wnt/β-Catenin Pathway. Mutat. Res./Genet. Toxicol. Environ. Mutagen. 2023, 888, 503637. [Google Scholar] [CrossRef] [PubMed]
  5. Vassileva, J.; Holmberg, O. Radiation Protection Perspective to Recurrent Medical Imaging: What Is Known and What More Is Needed? Br. J. Radiol. 2021, 94, 20210477. [Google Scholar] [CrossRef] [PubMed]
  6. Boice, J.D., Jr.; Cohen, S.S.; Mumma, M.T.; Howard, S.C.; Yoder, R.C.; Dauer, L.T. Mortality among Medical Radiation Workers in the United States, 1965–2016. Int. J. Radiat. Biol. 2023, 99, 183–207. [Google Scholar] [CrossRef] [PubMed]
  7. Hamada, N. Noncancer Effects of Ionizing Radiation Exposure on the Eye, the Circulatory System and beyond: Developments Made since the 2011 ICRP Statement on Tissue Reactions. Radiat. Res. 2023, 200, 188–216. [Google Scholar] [CrossRef]
  8. Hayashi, S.; Takenaka, M.; Kogure, H.; Yakushijin, T.; Maruyama, H.; Hori, Y.; Yoshio, T.; Ikezawa, K.; Takagi, T.; Asai, S. A Questionnaire Survey on Radiation Protection among 282 Medical Staff from 26 Endoscopy-fluoroscopy Departments in Japan. DEN Open 2021, 1, e5. [Google Scholar] [CrossRef]
  9. Saha, G.B. Radiation Safety in Nuclear Medicine: A Practical, Concise Guide; Springer Nature: Berlin/Heidelberg, Germany, 2023; ISBN 3-031-24609-8. [Google Scholar]
  10. Frane, N.; Bitterman, A. Radiation Safety and Protection; StatPearls Publishing: Treasure Island, FL, USA, 2020. [Google Scholar]
  11. O’Connor, M.; Rainford, L. The Impact of 3D Virtual Reality Radiography Practice on Student Performance in Clinical Practice. Radiography 2023, 29, 159–164. [Google Scholar] [CrossRef]
  12. Gunn, T.; Jones, L.; Bridge, P.; Rowntree, P.; Nissen, L. The Use of Virtual Reality Simulation to Improve Technical Skill in the Undergraduate Medical Imaging Student. Interact. Learn. Environ. 2018, 26, 613–620. [Google Scholar] [CrossRef]
  13. O’Connor, M.; Stowe, J.; Potocnik, J.; Giannotti, N.; Murphy, S.; Rainford, L. 3D Virtual Reality Simulation in Radiography Education: The Students’ Experience. Radiography 2021, 27, 208–214. [Google Scholar] [CrossRef] [PubMed]
  14. Sapkaroski, D.; Mundy, M.; Dimmock, M. Virtual Reality versus Conventional Clinical Role-Play for Radiographic Positioning Training: A Students’ Perception Study. Radiography 2020, 26, 57–62. [Google Scholar] [CrossRef] [PubMed]
  15. Karpicke, J.D.; Blunt, J.R. Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping. Science 2011, 331, 772–775. [Google Scholar] [CrossRef] [PubMed]
  16. Roediger, H.L.; Butler, A.C. The Critical Role of Retrieval Practice in Long-Term Retention. Trends Cogn. Sci. 2011, 15, 20–27. [Google Scholar] [CrossRef] [PubMed]
  17. Glenn, D.I. Determining Sample Size. In A Series of the Program Evaluation and Organizational Development; University of Florida: Gainesville, FL, USA, 1992. [Google Scholar]
  18. Vajak, D.; Livada, Č. Combining Photogrammetry, 3D Modeling and Real Time Information Gathering for Highly Immersive VR Experience. In Proceedings of the 2017 Zooming Innovation in Consumer Electronics International Conference (ZINC), Novi Sad, Serbia, 31 May–1 June 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 82–85. [Google Scholar]
  19. Luhmann, T.; Robson, S.; Kyle, S.; Boehm, J. Close-Range Photogrammetry and 3D Imaging; Walter de Gruyter GmbH & Co KG: Berlin, Germany, 2023; ISBN 3-11-102967-0. [Google Scholar]
  20. Likert, R. A Technique for the Measurement of Attitudes. Arch. Psychol. 1932, 22, 55. [Google Scholar]
  21. Nemoto, T.; Beglar, D. Likert-Scale Questionnaires 2014. In Proceedings of the JALT 2013 Conference Proceedings, Kobe, Japan, 25–28 October 2013; pp. 1–8. [Google Scholar]
  22. Harpe, S.E. How to Analyze Likert and Other Rating Scale Data. Curr. Pharm. Teach. Learn. 2015, 7, 836–850. [Google Scholar] [CrossRef]
  23. Brewer-Deluce, D.; Bak, A.B.; Simms, A.J.; Sinha, S.; Mitchell, J.P.; Shin, D.; Saraco, A.N.; Wainman, B.C. Virtual Reality Bell-ringer: The Development and Testing of a Stereoscopic Application for Human Gross Anatomy. Anat. Sci. Educ. 2021, 14, 330–341. [Google Scholar] [CrossRef] [PubMed]
  24. Lateef, F. Maximizing Learning and Creativity: Understanding Psychological Safety in Simulation-Based Learning. J. Emergencies Trauma Shock 2020, 13, 5. [Google Scholar] [CrossRef] [PubMed]
  25. Vogt, J.; Oeh, U.; Maringer, F.J. Development of the Occupational Exposure during the Production and Application of Radiopharmaceuticals in Germany. J. Radiol. Prot. 2024, 44, 011508. [Google Scholar] [CrossRef]
  26. Marshall, S.K.; Prom-On, P.; Sangkue, S.; Thiangsook, W. Assessment of Radiation Exposure in a Nuclear Medicine Department during 99mTc-MDP Bone Scintigraphy. Toxics 2023, 11, 814. [Google Scholar] [CrossRef]
  27. International Atomic Energy Agency. Quality Management Audits in Nuclear Medicine Practices: Non-Serial Publications; International Atomic Energy Agency: Vienna, Austria, 2009; ISBN 1-282-87844-1. [Google Scholar]
  28. Furuta, H.; Tsujimura, N.; Nishide, A.; Kudo, S.; Saigusa, S. Conversion Factor from Dosemeter Reading to Air Kerma for Nuclear Worker Using Anthropomorphic Phantom for Further Conversion from Air Kerma to Organ-Absorbed Dose. Radiat. Prot. Dosim. 2020, 189, 371–383. [Google Scholar] [CrossRef] [PubMed]
  29. International Commission on Radiological Protection. General Principles for the Radiation Protection of Workers: Adopted by the Commission in January 1997; Pergamon Press: Oxford, UK, 1997; ISBN 0-08-042741-3. [Google Scholar]
  30. International Atomic Energy Agency. Occupational Radiation Protection: General Safety Guide; IAEA: Vienna, Austria, 2018; ISBN 92-0-102917-9. [Google Scholar]
  31. Solanki, K. Operational Guidance on Hospital Radiopharmacy: A Safe and Effective Approach; International Atomic Energy Agency: Vienna, Austria, 2008. [Google Scholar]
  32. International Commission on Radiation Units. Quantities and Units in Radiation Protection Dosimetry; International Commission on Radiation: Stockholm, Sweden, 1993; ISBN 0-913394-50-5. [Google Scholar]
  33. Protection, R. ICRP Publication 103. Ann. ICRP 2007, 37, 2. [Google Scholar]
  34. Perks, C.A.; Yahnke, C.; Million, M. Medical Dosimetry Using Optically Stimulated Luminescence Dots and microStar Readers. Available online: https://api.semanticscholar.org/CorpusID:7217500 (accessed on 14 February 2024).
  35. Al-Senan, R.M.; Hatab, M.R. Characteristics of an OSLD in the Diagnostic Energy Range. Med. Phys. 2011, 38, 4396–4405. [Google Scholar] [CrossRef] [PubMed]
  36. Giansante, L.; Santos, J.C.; Umisedo, N.K.; Terini, R.A.; Costa, P.R. Characterization of OSL Dosimeters for Use in Dose Assessment in Computed Tomography Procedures. Phys. Medica 2018, 47, 16–22. [Google Scholar] [CrossRef] [PubMed]
  37. Davis, F.D. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef]
  38. Pastor, A.; Bourdin-Kreitz, P. Comparing Episodic Memory Outcomes from Walking Augmented Reality and Stationary Virtual Reality Encoding Experiences. Sci. Rep. 2024, 14, 1–23. [Google Scholar]
  39. Cimadevilla, J.M.; Nori, R.; Piccardi, L. Application of Virtual Reality in Spatial Memory. Brain Sci. 2023, 13, 1621. [Google Scholar] [CrossRef] [PubMed]
  40. Boller, B.; Ouellet, É.; Belleville, S. Using Virtual Reality to Assess and Promote Transfer of Memory Training in Older Adults with Memory Complaints: A Randomized Controlled Trial. Front. Psychol. 2021, 12, 627242. [Google Scholar] [CrossRef] [PubMed]
  41. Yan, S.; Shen, S.; Lu, Q.; Zhong, S.; Lv, S.; Lai, S.; Luo, Y.; Ran, H.; Duan, M.; Song, K. Virtual Reality Working Memory Training Improves Cognitive Performance of Acute and Remitted Patients with Major Depressive Disorder. J. Affect. Disord. 2024, 344, 267–276. [Google Scholar] [CrossRef]
  42. Gentry, S.V.; Gauthier, A.; Ehrstrom, B.L.; Wortley, D.; Lilienthal, A.; Car, L.T.; Dauwels-Okutsu, S.; Nikolaou, C.K.; Zary, N.; Campbell, J. Serious Gaming and Gamification Education in Health Professions: Systematic Review. J. Med. Internet Res. 2019, 21, e12994. [Google Scholar] [CrossRef]
  43. Choi, D.; Seo, S.; Park, H.; Hong, T.; Koo, C. Forecasting Personal Learning Performance in Virtual Reality-Based Construction Safety Training Using Biometric Responses. Autom. Constr. 2023, 156, 105115. [Google Scholar] [CrossRef]
  44. Darnell, D.K.; Krieg, P.A. Student Engagement, Assessed Using Heart Rate, Shows No Reset Following Active Learning Sessions in Lectures. PLoS ONE 2019, 14, e0225709. [Google Scholar] [CrossRef] [PubMed]
  45. Bustos-López, M.; Cruz-Ramírez, N.; Guerra-Hernández, A.; Sánchez-Morales, L.N.; Cruz-Ramos, N.A.; Alor-Hernández, G. Wearables for Engagement Detection in Learning Environments: A Review. Biosensors 2022, 12, 509. [Google Scholar] [CrossRef] [PubMed]
  46. Ribeiro, C.M.; Gomes, R.D.A.; Monteiro, C.B.D.M.; Dias, R.M.; Simcsik, A.O.; de Araújo, L.V.; Maia, L.C.P.; de Oliveira, A.P.; de Freitas, B.L.; Dawes, H. Heart Rate Variability during Virtual Reality Activity in Individuals after Hospitalization for COVID-19: A Cross-Sectional Control Study. Electronics 2023, 12, 1925. [Google Scholar] [CrossRef]
  47. Weibel, R.P.; Grübel, J.; Zhao, H.; Thrash, T.; Meloni, D.; Hölscher, C.; Schinazi, V.R. Virtual Reality Experiments with Physiological Measures. JoVE (J. Vis. Exp.) 2018, 138, e58318. [Google Scholar]
  48. Tong, A.; Sainsbury, P.; Craig, J. Consolidated Criteria for Reporting Qualitative Research (COREQ): A 32-Item Checklist for Interviews and Focus Groups. Int. J. Qual. Health Care 2007, 19, 349–357. [Google Scholar] [CrossRef] [PubMed]
  49. Harrell, M.C.; Bradley, M. Data Collection Methods: Semi-Structured Interviews and Focus Groups; RAND Corporation: Santa Monica, CA, USA, 2009. [Google Scholar]
  50. Gill, P.; Baillie, J. Interviews and Focus Groups in Qualitative Research: An Update for the Digital Age. Br. Dent. J. 2018, 225, 668–672. [Google Scholar] [CrossRef] [PubMed]
  51. Longhurst, R. Semi-Structured Interviews and Focus Groups. Key Methods Geogr. 2003, 3, 143–156. [Google Scholar]
  52. Whitman, L.; Malzahn, D.; Madhavan, V.; Weheba, G.; Krishnan, K. Virtual Reality Case Study throughout the Curriculum to Address Competency Gaps. Int. J. Eng. Educ. 2004, 20, 690–702. [Google Scholar]
  53. Meyer, O.; Imhof, M.; Coyle, D.; Banerjee, M. Positive Learning and Pluriliteracies: Growth in Higher Education and Implications for Course Design, Assessment and Research. In Positive Learning in the Age of Information: A Blessing or a Curse? Springer: Berlin/Heidelberg, Germany, 2018; pp. 235–265. [Google Scholar]
  54. Catai, A.M.; Pastre, C.M.; de Godoy, M.F.; da Silva, E.; de Medeiros Takahashi, A.C.; Vanderlei, L.C.M. Heart Rate Variability: Are You Using It Properly? Standardisation Checklist of Procedures. Braz. J. Phys. Ther. 2020, 24, 91–102. [Google Scholar] [CrossRef]
  55. Braun, V.; Clarke, V. Using Thematic Analysis in Psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef]
  56. Training, R. TRAININGCOURSESERIES 39. Available online: https://www-pub.iaea.org/MTCD/Publications/PDF/TCS-39_web.pdf (accessed on 25 February 2024).
  57. Vassileva, J.; Applegate, K.; Paulo, G.; Vano, E.; Holmberg, O. Strengthening Radiation Protection Education and Training of Health Professionals: Conclusions from an IAEA Meeting. J. Radiol. Prot. 2022, 42, 011504. [Google Scholar] [CrossRef] [PubMed]
  58. Santos, J.; Foley, S.; Andersson, J.; Figueiredo, J.P.; Hoeschen, C.; Damilakis, J.; Frija, G.; Alves, F.; Riklund, K.; Rainford, L. Education and Training in Radiation Protection in Europe: Results from the EURAMED Rocc-n-Roll Project Survey. Insights Into. Imaging 2023, 14, 55. [Google Scholar] [CrossRef] [PubMed]
  59. Newhauser, W.D.; Williams, J.P.; Noska, M.A.; Bluth, E.I.; Townsend, L.W.; Tolmachev, S.Y.; Dewji, S.A. In Support of ICRP’s Call to Action to Strengthen Expertise in Radiological Protection Worldwide. Radiat. Environ. Biophys. 2023, 62, 287–288. [Google Scholar] [CrossRef] [PubMed]
  60. Rühm, W.; Cho, K.; Larsson, C.-M.; Wojcik, A.; Clement, C.; Applegate, K.; Bochud, F.; Bouffler, S.; Cool, D.; Hirth, G. Vancouver Call for Action to Strengthen Expertise in Radiological Protection Worldwide. Radiat. Environ. Biophys. 2023, 62, 175–180. [Google Scholar] [CrossRef] [PubMed]
  61. Giammarile, F.; Orellana, P.; Paez, D. IAEA Strategy for Enhancing the Sustainability of Nuclear Medicine in Low-and Middle-Income Countries. In Beyond Becquerel and Biology to Precision Radiomolecular Oncology: Festschrift in Honor of Richard P. Baum; Springer International Publishing: Cham, Switzerland, 2024; pp. 101–107. [Google Scholar]
  62. Wilmoth, J.; Menozzi, C.; Bassarsky, L. Global Population Growth and Sustainable Development; United Nations: New York, NY, USA, 2021. [Google Scholar]
  63. Kim, H.-Y.; Kim, E.-Y. Effects of Medical Education Program Using Virtual Reality: A Systematic Review and Meta-Analysis. Int. J. Environ. Res. Public Health 2023, 20, 3895. [Google Scholar] [CrossRef] [PubMed]
  64. Brown, P.C.; Roediger, H.L., III; McDaniel, M.A. Make It Stick: The Science of Successful Learning; Harvard University Press: Cambridge, UK, 2014; ISBN 0-674-72901-3. [Google Scholar]
  65. Rainford, L.; Tcacenco, A.; Potocnik, J.; Brophy, C.; Lunney, A.; Kearney, D.; O’Connor, M. Student Perceptions of the Use of Three-Dimensional (3-D) Virtual Reality (VR) Simulation in the Delivery of Radiation Protection Training for Radiography and Medical Students. Radiography 2023, 29, 777–785. [Google Scholar] [CrossRef] [PubMed]
  66. Miguel-Alonso, I.; Checa, D.; Guillen-Sanz, H.; Bustillo, A. Evaluation of the Novelty Effect in Immersive Virtual Reality Learning Experiences. Virtual Real. 2024, 28, 27. [Google Scholar] [CrossRef]
  67. Makransky, G.; Lilleholt, L. A Structural Equation Modeling Investigation of the Emotional Value of Immersive Virtual Reality in Education. Educ. Technol. Res. Dev. 2018, 66, 1141–1164. [Google Scholar] [CrossRef]
  68. Chávez, O.L.; Rodríguez, L.-F.; Gutierrez-Garcia, J.O. A Comparative Case Study of 2D, 3D and Immersive-Virtual-Reality Applications for Healthcare Education. Int. J. Med. Inform. 2020, 141, 104226. [Google Scholar] [CrossRef]
  69. Liu, J.Y.W.; Yin, Y.-H.; Kor, P.P.K.; Cheung, D.S.K.; Zhao, I.Y.; Wang, S.; Su, J.J.; Christensen, M.; Tyrovolas, S.; Leung, A.Y. The Effects of Immersive Virtual Reality Applications on Enhancing the Learning Outcomes of Undergraduate Health Care Students: Systematic Review With Meta-Synthesis. J. Med. Internet Res. 2023, 25, e39989. [Google Scholar] [CrossRef] [PubMed]
  70. Bjork, E.L.; Bjork, R.A. Making Things Hard on Yourself, but in a Good Way: Creating Desirable Difficulties to Enhance Learning. In Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society; Worth Publishing: New York, NY, USA, 2011; Volume 2. [Google Scholar]
  71. Agarwal, P.K.; Bain, P.M.; Chamberlain, R.W. The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations from a Teacher, a Principal, and a Scientist. Educ. Psychol. Rev. 2012, 24, 437–448. [Google Scholar] [CrossRef]
  72. Aljabari, S.; Kadhim, Z. Common Barriers to Reporting Medical Errors. Sci. World J. 2021, 2021, 6494889. [Google Scholar] [CrossRef] [PubMed]
  73. Yu, J.H.; Chang, H.J.; Kim, S.S.; Park, J.E.; Chung, W.Y.; Lee, S.K.; Kim, M.; Lee, J.H.; Jung, Y.J. Effects of High-Fidelity Simulation Education on Medical Students’ Anxiety and Confidence. PLoS ONE 2021, 16, e0251078. [Google Scholar] [CrossRef] [PubMed]
  74. Zinchenko, Y.; Khoroshikh, P.; Sergievich, A.; Smirnov, A.; Tumyalis, A.; Kovalev, A.; Gutnikov, S.; Golokhvast, K. Virtual Reality Is More Efficient in Learning Human Heart Anatomy Especially for Subjects with Low Baseline Knowledge. New Ideas Psychol. 2020, 59, 100786. [Google Scholar] [CrossRef]
  75. Chung, L.-Y. Incorporating 3D-Virtual Reality into Language Learning. Int. J. Digit. Content Technol. Its Appl. 2012, 6, 249–255. [Google Scholar]
  76. Redondo-Rodríguez, C.; Becerra-Mejías, J.A.; Gil-Fernández, G.; Rodríguez-Velasco, F.J. Influence of Gamification and Cooperative Work in Peer, Mixed and Interdisciplinary Teams on Emotional Intelligence, Learning Strategies and Life Goals That Motivate University Students to Study. Int. J. Environ. Res. Public Health 2022, 20, 547. [Google Scholar] [CrossRef]
  77. Gan, W.; Mok, T.-N.; Chen, J.; She, G.; Zha, Z.; Wang, H.; Li, H.; Li, J.; Zheng, X. Researching the Application of Virtual Reality in Medical Education: One-Year Follow-Up of a Randomized Trial. BMC Med. Educ. 2023, 23, 3. [Google Scholar] [CrossRef] [PubMed]
  78. Thompson, M.; Uz-Bilgin, C.; Tutwiler, M.S.; Anteneh, M.; Meija, J.C.; Wang, A.; Tan, P.; Eberhardt, R.; Roy, D.; Perry, J. Immersion Positively Affects Learning in Virtual Reality Games Compared to Equally Interactive 2D Games. Inf. Learn. Sci. 2021, 122, 442–463. [Google Scholar] [CrossRef]
  79. Essoe, J.K.-Y.; Reggente, N.; Ohno, A.A.; Baek, Y.H.; Dell’Italia, J.; Rissman, J. Enhancing Learning and Retention with Distinctive Virtual Reality Environments and Mental Context Reinstatement. NPJ Sci. Learn. 2022, 7, 31. [Google Scholar] [CrossRef]
  80. Ariansyah, D.; Pardamean, B.; Barbaro, E.; Erkoyuncu, J.A. Augmented Reality Training for Improved Learnability. CIRP J. Manuf. Sci. Technol. 2024, 48, 19–27. [Google Scholar] [CrossRef]
  81. Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A Systematic Review of Immersive Virtual Reality Applications for Higher Education: Design Elements, Lessons Learned, and Research Agenda. Comput. Educ. 2020, 147, 103778. [Google Scholar] [CrossRef]
  82. Moser, S.; Zumbach, J.; Deibl, I. The Effect of Metacognitive Training and Prompting on Learning Success in Simulation-based Physics Learning. Sci. Educ. 2017, 101, 944–967. [Google Scholar] [CrossRef]
  83. Pellas, N. The Influence of Computer Self-Efficacy, Metacognitive Self-Regulation and Self-Esteem on Student Engagement in Online Learning Programs: Evidence from the Virtual World of Second Life. Comput. Hum. Behav. 2014, 35, 157–170. [Google Scholar] [CrossRef]
  84. Lindberg, J.C.; Archer, D. Radiophobia: Useful Concept, or Ostracising Term? Prog. Nucl. Energy 2022, 149, 104280. [Google Scholar] [CrossRef]
  85. Lindberg, J.C. ‘J’Accuse.!’: The Continuous Failure to Address Radiophobia and Placing Radiation in Perspective. J. Radiol. Prot. 2021, 41, 459. [Google Scholar] [CrossRef] [PubMed]
  86. Dauer, L.T.; Thornton, R.H.; Hay, J.L.; Balter, R.; Williamson, M.J.; St. Germain, J. Fears, Feelings, and Facts: Interactively Communicating Benefits and Risks of Medical Radiation with Patients. Am. J. Roentgenol. 2011, 196, 756–761. [Google Scholar] [CrossRef] [PubMed]
  87. Bader, I.; Bader, I.; Lopes Alves, I.; Vállez García, D.; Vellas, B.; Dubois, B.; Boada, M.; Marquié, M.; Altomare, D.; Scheltens, P. Recruitment of Pre-Dementia Participants: Main Enrollment Barriers in a Longitudinal Amyloid-PET Study. Alzheimer’s Res. Ther. 2023, 15, 189. [Google Scholar] [CrossRef]
  88. Kirkpatrick, J.D.; Kirkpatrick, W.K. Kirkpatrick’s Four Levels of Training Evaluation; Association for Talent Development: Alexandria, VA, USA, 2016; ISBN 1-60728-102-3. [Google Scholar]
  89. Anderson, C. Presenting and Evaluating Qualitative Research. Am. J. Pharm. Educ. 2010, 74, 141. [Google Scholar] [CrossRef]
Figure 1. Flow chart training modules for the control group and virtual reality training group. (BP = blood pressure, HR = heart rate).
Figure 1. Flow chart training modules for the control group and virtual reality training group. (BP = blood pressure, HR = heart rate).
Applsci 14 05041 g001
Figure 2. Personnel dose assessment to evaluate radiation exposure during the preparation and handling of 99mTc-MDP radiopharmaceutical at various body parts, including Hp(0.07) measurements at (A) the left thumb, (B) the left index finger, (C) the left ring finger, (D) the left palm, (E) the right thumb, (F) the right index finger, (G) the right ring finger, and (H) the right palm; Hp(3) measurements at (I) the left eye lens and (J) the right eye lens; and Hp(10) measurements for (K) the whole body.
Figure 2. Personnel dose assessment to evaluate radiation exposure during the preparation and handling of 99mTc-MDP radiopharmaceutical at various body parts, including Hp(0.07) measurements at (A) the left thumb, (B) the left index finger, (C) the left ring finger, (D) the left palm, (E) the right thumb, (F) the right index finger, (G) the right ring finger, and (H) the right palm; Hp(3) measurements at (I) the left eye lens and (J) the right eye lens; and Hp(10) measurements for (K) the whole body.
Applsci 14 05041 g002
Figure 3. Study setting and student engagement in control and virtual reality simulation groups. (A,B) Interface of radiopharmaceutical quality control gamification. (C) Physical assessment of the participant’s blood pressure and heart rate throughout the training. (D) Self-study using paper-based learning materials. (E) YouTube videos using a desktop computer (control group). (F) Virtual reality simulation (VR group). (G) Gamification of the control group. (H) Gamification of the VR group.
Figure 3. Study setting and student engagement in control and virtual reality simulation groups. (A,B) Interface of radiopharmaceutical quality control gamification. (C) Physical assessment of the participant’s blood pressure and heart rate throughout the training. (D) Self-study using paper-based learning materials. (E) YouTube videos using a desktop computer (control group). (F) Virtual reality simulation (VR group). (G) Gamification of the control group. (H) Gamification of the VR group.
Applsci 14 05041 g003
Figure 4. Revised Bloom’s Taxonomy framework supporting virtual reality immersive learning strategy as a pedagogical tool in a radiological hot laboratory.
Figure 4. Revised Bloom’s Taxonomy framework supporting virtual reality immersive learning strategy as a pedagogical tool in a radiological hot laboratory.
Applsci 14 05041 g004
Figure 5. The hot lab floor plan illustrates the layout and arrangement of equipment and workstations for the preparation and handling of 99mTc-MDP radiopharmaceutical, with designated controlled areas for assessing radiation dose in nuclear medicine procedures. The locations of workstations in hot lab, including (1) hot lab entrance 1, (2) storage 1, (3) sink, (4) 99Mo/99mTc generator, (5) 99Mo/99mTc generator work area behind shielding, (6) dose calibrator 1, (7) dose calibrator work area behind shielding, (8) storage 2, (9) waste storage room entrance, (10) waste storage freezer, (11) dry waste storage, (12) waste storage room, (13) area at the side of 99Mo/99mTc generator, (14) laminar flow cabinet, (15) laminar flow cabinet waste disposal area, (16) laminar flow cabinet dose calibrator, (17) center of hot lab, and (18) hot lab entrance 2. The arrow indicates the direction for surveying the controlled areas during the preparation and handling of the 99mTc-MDP radiopharmaceutical.
Figure 5. The hot lab floor plan illustrates the layout and arrangement of equipment and workstations for the preparation and handling of 99mTc-MDP radiopharmaceutical, with designated controlled areas for assessing radiation dose in nuclear medicine procedures. The locations of workstations in hot lab, including (1) hot lab entrance 1, (2) storage 1, (3) sink, (4) 99Mo/99mTc generator, (5) 99Mo/99mTc generator work area behind shielding, (6) dose calibrator 1, (7) dose calibrator work area behind shielding, (8) storage 2, (9) waste storage room entrance, (10) waste storage freezer, (11) dry waste storage, (12) waste storage room, (13) area at the side of 99Mo/99mTc generator, (14) laminar flow cabinet, (15) laminar flow cabinet waste disposal area, (16) laminar flow cabinet dose calibrator, (17) center of hot lab, and (18) hot lab entrance 2. The arrow indicates the direction for surveying the controlled areas during the preparation and handling of the 99mTc-MDP radiopharmaceutical.
Applsci 14 05041 g005
Figure 6. Personnel radiation exposure during the preparation and handling of 99mTc-MDP radiopharmaceutical includes exposure measurements at various body parts, including Hp(0.07) at the right thumb, right index finger, right ring finger, right palm, left thumb, left index finger, left ring finger, and left palm; Hp(3) at the right eye lens and left eye lens; and Hp(10) for the whole body. The data are presented as the mean ± standard deviation (n = 3).
Figure 6. Personnel radiation exposure during the preparation and handling of 99mTc-MDP radiopharmaceutical includes exposure measurements at various body parts, including Hp(0.07) at the right thumb, right index finger, right ring finger, right palm, left thumb, left index finger, left ring finger, and left palm; Hp(3) at the right eye lens and left eye lens; and Hp(10) for the whole body. The data are presented as the mean ± standard deviation (n = 3).
Applsci 14 05041 g006
Figure 7. Evaluation of the effectiveness of VR-assisted teaching in enhancing students’ learning experience at various time points: pre-test, same-day post-test, after 1 month, and after 3 months. (A) Test scores (%) are presented for the following conditions: pre-test (PT), same-day post-test of self-study (PS), same-day post-test of watching videos using a desktop computer (control group) (C-A), same-day post-test of virtual reality simulation (VR group) (VR-A), post-test of the control group after 1-month training (C-1M), post-test of VR after 1-month training (VR-1M), post-test of the control group after 3 months training (C-3M), and post-test of VR after 3 months training (VR-3M). The distribution of the number of correct responses for the control group is shown in (B) same-day post-test, (C) post-test after 1 month of training, and (D) post-test after 3 months of training. Similarly, the distribution of the number of correct responses for the VR training group is depicted in (E) same-day post-test, (F) post-test after 1 month of training, and (G) post-test after 3 months of training. The data are presented as mean ± standard deviation (n = 30). Statistical significance is indicated as follows: ns (not significant) when p > 0.05, * p ≤ 0.05, and ** p ≤ 0.01.
Figure 7. Evaluation of the effectiveness of VR-assisted teaching in enhancing students’ learning experience at various time points: pre-test, same-day post-test, after 1 month, and after 3 months. (A) Test scores (%) are presented for the following conditions: pre-test (PT), same-day post-test of self-study (PS), same-day post-test of watching videos using a desktop computer (control group) (C-A), same-day post-test of virtual reality simulation (VR group) (VR-A), post-test of the control group after 1-month training (C-1M), post-test of VR after 1-month training (VR-1M), post-test of the control group after 3 months training (C-3M), and post-test of VR after 3 months training (VR-3M). The distribution of the number of correct responses for the control group is shown in (B) same-day post-test, (C) post-test after 1 month of training, and (D) post-test after 3 months of training. Similarly, the distribution of the number of correct responses for the VR training group is depicted in (E) same-day post-test, (F) post-test after 1 month of training, and (G) post-test after 3 months of training. The data are presented as mean ± standard deviation (n = 30). Statistical significance is indicated as follows: ns (not significant) when p > 0.05, * p ≤ 0.05, and ** p ≤ 0.01.
Applsci 14 05041 g007
Figure 8. Practical and experimental gamification of learning. (A) Number of correct responses (out of 3 targets) for the control group, virtual reality simulation, and hands-on labeling quality control training. (B) Targeting accuracy (%) for the training simulation and hands-on labeling of the control and virtual reality simulation training groups. (C) Targeting precision (%) for the training simulation and hands-on labeling of the control and virtual reality simulation training groups. The data are presented as the mean ± standard deviation (n = 30). Statistical significance is indicated as ns (insignificant) when p > 0.05.
Figure 8. Practical and experimental gamification of learning. (A) Number of correct responses (out of 3 targets) for the control group, virtual reality simulation, and hands-on labeling quality control training. (B) Targeting accuracy (%) for the training simulation and hands-on labeling of the control and virtual reality simulation training groups. (C) Targeting precision (%) for the training simulation and hands-on labeling of the control and virtual reality simulation training groups. The data are presented as the mean ± standard deviation (n = 30). Statistical significance is indicated as ns (insignificant) when p > 0.05.
Applsci 14 05041 g008
Figure 9. Physical assessment of blood pressure and heart rate for student engagement responses: (A) systolic blood pressure; (B) diastolic blood pressure; (C) heart rates are presented for the following conditions: pre-test (BH-PT), same-day post-test of self-study (BH-S), before watching video 1 of the control group (BH-C1), after watching video 1 of the control group (BH-C2), after watching video 2 of the control group (BH-C3), after watching video 3 of the control group (BH-C4), before gamification of the control group (BH-CG1), after gamification of the control group (BH-CG2), before VR 1 of the VR group (BH-V1), after VR 1 of the VR group (BH-V2), after VR 2 of the VR group (BH-V3), after VR 3 of the VR group (BH-V4), before gamification of the VR group (BH-VG1), after gamification of the VR group (BH-VG2) (n = 30).
Figure 9. Physical assessment of blood pressure and heart rate for student engagement responses: (A) systolic blood pressure; (B) diastolic blood pressure; (C) heart rates are presented for the following conditions: pre-test (BH-PT), same-day post-test of self-study (BH-S), before watching video 1 of the control group (BH-C1), after watching video 1 of the control group (BH-C2), after watching video 2 of the control group (BH-C3), after watching video 3 of the control group (BH-C4), before gamification of the control group (BH-CG1), after gamification of the control group (BH-CG2), before VR 1 of the VR group (BH-V1), after VR 1 of the VR group (BH-V2), after VR 2 of the VR group (BH-V3), after VR 3 of the VR group (BH-V4), before gamification of the VR group (BH-VG1), after gamification of the VR group (BH-VG2) (n = 30).
Applsci 14 05041 g009
Figure 10. Training module questionnaire analysis. (A) The percentage reduction and (B) percentage difference between the control group and virtual reality simulation (VR group) and radiophobia (%) after training, including (i) I am anxious about the potential health effects of radiation exposure during my training. (ii) I often worry about being exposed to radiation during my training. (iii) The fear of making a mistake and causing radiation exposure affects my training performance. (iv) My concerns about radiation exposure affect my motivation to pursue a career in radiological technology. (v) I often discuss my concerns about radiation exposure with fellow trainees or instructors. (C) The percentage increase and (D) percentage difference between the control group and virtual reality simulation (VR group) and in safety confidence (%) after the training, including (a) I am confident in my ability to detect and correct errors in radiation safety practices during my training. (b) I am confident about implementing radiation safety measures during procedures or experiments. (c) I often seek instructor guidance or clarification regarding radiation safety procedures. (d) The training program prepares me well to manage my safety concerns related to radiation exposure. (e) I feel adequately informed about the training program’s radiation safety protocols and practices. (f) I have experienced avoidance behaviors (e.g., avoiding specific tasks or procedures) due to my concerns about radiation exposure during the training. The questionnaire consists of 5 points: strongly agree, agree, neutral, disagree, and strongly disagree. A numerical score is assigned to each question to quantify the strength of the response data. The response anchors “strongly agree” and “strongly disagree” are positioned at the ends of the scale. At the same time, a neutral item typically represents the midway point in the middle of the 5-point scale. Data are presented as mean ± standard deviation (n = 30).
Figure 10. Training module questionnaire analysis. (A) The percentage reduction and (B) percentage difference between the control group and virtual reality simulation (VR group) and radiophobia (%) after training, including (i) I am anxious about the potential health effects of radiation exposure during my training. (ii) I often worry about being exposed to radiation during my training. (iii) The fear of making a mistake and causing radiation exposure affects my training performance. (iv) My concerns about radiation exposure affect my motivation to pursue a career in radiological technology. (v) I often discuss my concerns about radiation exposure with fellow trainees or instructors. (C) The percentage increase and (D) percentage difference between the control group and virtual reality simulation (VR group) and in safety confidence (%) after the training, including (a) I am confident in my ability to detect and correct errors in radiation safety practices during my training. (b) I am confident about implementing radiation safety measures during procedures or experiments. (c) I often seek instructor guidance or clarification regarding radiation safety procedures. (d) The training program prepares me well to manage my safety concerns related to radiation exposure. (e) I feel adequately informed about the training program’s radiation safety protocols and practices. (f) I have experienced avoidance behaviors (e.g., avoiding specific tasks or procedures) due to my concerns about radiation exposure during the training. The questionnaire consists of 5 points: strongly agree, agree, neutral, disagree, and strongly disagree. A numerical score is assigned to each question to quantify the strength of the response data. The response anchors “strongly agree” and “strongly disagree” are positioned at the ends of the scale. At the same time, a neutral item typically represents the midway point in the middle of the 5-point scale. Data are presented as mean ± standard deviation (n = 30).
Applsci 14 05041 g010
Table 1. Summary of thematic analysis regarding students’ perspectives in virtual reality immersive learning in radiological technology students (n = 10).
Table 1. Summary of thematic analysis regarding students’ perspectives in virtual reality immersive learning in radiological technology students (n = 10).
ThemeCodeExamplesFrequency
n%
Useful learning toolEngage“VR improves my focus and engagement, ensuring that I remain completely focused without breaks in concentration.”660
Easy to
understand
“Easier to learn and understand the radiopharmaceutical preparation process and quality control steps.”
“Opportunity to see all the process and steps before lab practice and easy-to-follow protocols.”
990
Memory“It (VR) helps me remember information by doing and interacting rather than reading/hearing.”880
Safety
confidence
“You may fail, make mistakes, and learn from them.”
“VR…It’s building my confidence through providing a secure environment conducive to experimentation and trial and error.”
“I am not so scared of working in the hot lab.”
770
Learning
experiences
“I really liked the recorded voice. That makes the experience even better for me, because I actually interacted with that voice and interactions, and, sure, it is pleasant, I feel to myself [smiles].”
“It’s memorable because it’s more interactive, enjoyable, and personal.”
770
Implement in curriculum“VR might be used in every lesson; could you incorporate it into the curriculum.”880
Getting ready for the clinical practice“There are only a few opportunities for me to practice the skills during hot lab practical, and while I am in clinical practice, I do not have the chance to practice them as frequently as I would want. VR training would enable me to practice as much as possible before clinical practice. It would be fantastic if there were booster classes that utilized virtual reality training before clinical practice.”
“VR before hot lab practice and clinical practice would be beneficial.”
“If I’m in clinical practice, I think I’ll feel nervous, especially when attempting something for the first time. This is where VR could be incredibly useful. It would allow me to practice hands-on procedures in a simulated environment and gain experience and confidence without the pressure of being in an actual clinical setting.”
10100
Challenges and threatsResources
incurred by the
technology
“Due to costs, it is not feasible to train many students at the same time.”
“It could be that insufficient VR devices could lead to students lagging behind, as some might not be able to complete the learning materials until a week after others have finished. As a result, the initial group would have a seven-day learning advantage, while the latter group would be left waiting without actively participating.”
770
Simulation
sickness
“I have experienced discomfort or motion sickness when using VR systems, which can limit the duration and effectiveness of training sessions.”220
Interactivity limitations“While VR can provide engaging experiences, the amount of involvement may be restricted compared to real-world interactions, reducing the depth of learning and skill development.”330
Lack of
personalization
“I think VR simulations may not always match individual learning methods or preferences, limiting their efficacy for all interested students.”220
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Marshall, S.K.; Sirieak, N.; Karnkorn, P.; Keawtong, V.; Hayeeabdunromae, A.; Noomad, N.; Durawee, W.; Cheewakul, J. Nuclear Medicine Radiological Hot Laboratory Simulation: A Mixed-Method Intervention Study on Immersive Virtual Reality for Sustainable Education. Appl. Sci. 2024, 14, 5041. https://doi.org/10.3390/app14125041

AMA Style

Marshall SK, Sirieak N, Karnkorn P, Keawtong V, Hayeeabdunromae A, Noomad N, Durawee W, Cheewakul J. Nuclear Medicine Radiological Hot Laboratory Simulation: A Mixed-Method Intervention Study on Immersive Virtual Reality for Sustainable Education. Applied Sciences. 2024; 14(12):5041. https://doi.org/10.3390/app14125041

Chicago/Turabian Style

Marshall, Suphalak Khamruang, Nantakorn Sirieak, Pornchanok Karnkorn, Virunyupa Keawtong, Awatif Hayeeabdunromae, Nadia Noomad, Wanita Durawee, and Jongwat Cheewakul. 2024. "Nuclear Medicine Radiological Hot Laboratory Simulation: A Mixed-Method Intervention Study on Immersive Virtual Reality for Sustainable Education" Applied Sciences 14, no. 12: 5041. https://doi.org/10.3390/app14125041

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop