Enhanced Preoperative Pancreatoduodenectomy Patient Education Using Mixed Reality Technology: A Randomized Controlled Pilot Study

Abstract

(1) Background: Mixed Reality (MR) technology, such as the HoloLens, offers a novel approach to preoperative education. This study evaluates its feasibility and effectiveness in improving patient comprehension and comfort during informed consent for pancreatoduodenectomy. (2) Methods: A single-center, randomized, controlled pilot study was conducted between February and May 2023. Patients recommended for pancreatoduodenectomy were randomized into a control group receiving standard education or an intervention group using the HoloLens. Pre- and post-intervention surveys assessed patient understanding and comfort. (3) Results: Nineteen patients participated (8 HoloLens, 11 control). Both groups showed improved comprehension post-intervention, but only the HoloLens group demonstrated a statistically significant increase (Z = −2.524, p = 0.012). MR users had a greater understanding of surgical steps compared to controls, and 75% of participants in both groups reported high comfort levels with the surgery. MR integration was feasible and did not disrupt clinical workflow. (4) Conclusions: These findings suggest that MR can enhance preoperative education for complex procedures. However, limitations include the small sample size and single-center design, necessitating larger studies to confirm its broader applicability. MR-based education represents a promising tool to improve patient engagement and comprehension in surgical decision making.

1. Introduction

Over 8000 pancreatoduodenectomies are performed annually in the United States, with a high morbidity rate exceeding 50% [1]. This complex and high-risk procedure requires detailed preoperative education to ensure that patients understand their diagnosis, the planned surgical procedure, potential risks, and postoperative care. Effective communication is crucial, as inadequate understanding can lead to confusion, anxiety, and potentially poor decision making. Traditional educational methods, such as verbal explanations and static diagrams, often fall short in conveying the complex, multi-dimensional nature of medical procedures like pancreatoduodenectomy. These methods tend to be passive, relying heavily on text and two-dimensional images, which may not be sufficient for patients with limited medical knowledge or those facing the stress of an impending major surgery [2].

Mixed Reality (MR) technologies, particularly devices such as the HoloLens, have emerged as a promising tool in medical education, enabling interactive and immersive experiences that bridge the gap between abstract concepts and real-world understanding [3]. MR has shown considerable promise in medical training, with studies demonstrating its effectiveness in enhancing the comprehension of complex anatomical structures and surgical procedures [4,5]. For example, several studies have demonstrated how MR can facilitate a more intuitive and detailed visualization of anatomy, allowing for a deeper understanding of patient-specific conditions and surgical techniques [6]. While much of the research has focused on MR’s applications in medical training, there has been limited exploration into its potential to improve patient education, particularly in the informed consent process.

The use of MR in patient education holds significant promise, especially in complex surgeries like pancreatoduodenectomy, which involves intricate anatomical structures and a high degree of technical skill. Studies have highlighted how MR’s immersive, three-dimensional visualizations can simplify complex medical information, making it more accessible and understandable for patients [7,8,9]. By offering a dynamic, interactive platform, MR could significantly enhance patients’ comprehension of their diagnosis, surgical procedure, and associated risks, potentially leading to increased comfort and confidence during the consent process. Thus, our study design was inspired by prior studies that demonstrated an increased usage of MR (HoloLens) in allaying patient anxiety and improving outcomes.

In addition to MR, various other digital tools have been explored for patient education, including videos, animations, and virtual reality. However, MR’s ability to provide a highly personalized and interactive experience, where patients can engage with three-dimensional (3D) models of their own anatomy and see the planned surgical procedure from different angles, distinguishes it from other methods [10,11]. Virtual reality (VR) is a digital simulated experience based on reality-derived images. Haptic devices, such as controllers with position trackers and motion capture data gloves, are used to navigate 3D models and at the same time interact with the simulated environment in real time using natural senses and motion. MR is a computer system that synchronizes reality and 3D model constructions, allowing surgeons to link the real surgical field view to 3D model constructions of the surgical case, enabling interaction with surgical targets in both the physical and virtual world. This distinction highlights MR’s enhanced ability to allow interaction with the surgical targets in both physical and virtual worlds, compared to VR or traditional computer programs. Despite these promising developments, the application of MR in patient education, particularly in the context of complex surgeries such as pancreatoduodenectomy, remains largely underexplored.

While our study explores the use of MR, it is important to acknowledge the variety of existing approaches to patient education during the informed consent process. Traditional methods include verbal explanations, written materials, and 2D pictorial diagrams, as employed in our control group design. These methods are ubiquitous in clinical practice due to their simplicity and low cost. However, they often suffer from significant limitations. Patients may have difficulty understanding complex medical information delivered solely through verbal explanations, and written materials may be challenging for patients with low literacy or visual impairments. Two-dimensional diagrams, while helpful for illustrating basic concepts, lack the depth and interactivity needed to fully convey the spatial relationships involved in surgical procedures.

More recently, digital tools have become increasingly prevalent, encompassing video presentations, interactive computer programs, and virtual reality (VR) simulations [12]. Videos can offer dynamic visual and auditory explanations, enhancing engagement compared to static text or diagrams. Interactive computer programs can provide a self-paced learning experience, allowing patients to explore information at their own speed. VR simulations can offer immersive experiences, allowing patients to visualize surgical procedures in a 3D environment. However, video presentations can be passive, failing to address specific patient questions, and may not be tailored to individual patient anatomy. Interactive computer programs may still lack the tactile and spatial understanding afforded by physical interaction. VR, while immersive, can be disorienting for some patients and may not fully integrate the physician into the educational process.

Unlike VR, MR allows patients to remain grounded, connecting with their surroundings and the educating physician. This is a significant advantage, as it allows for dynamic, real-time interaction and the clarification of doubts, which is crucial in the informed consent process. The interactive 3D visualizations in MR allow for a more intuitive understanding of complex anatomy and surgical procedures than traditional 2D methods. Furthermore, the head tracking and spatial mapping capabilities of devices like the HoloLens enable a personalized and interactive experience where patients can explore anatomical models from different angles and distances, with the physician present to guide them. This allows for a dynamic question-and-answer session and enables the physician to tailor the education to the specific needs of the patient, something that other methods struggle to replicate. The ability to integrate the physician into the educational process, facilitating a more personalized and interactive experience, sets MR apart from other modalities.

2. Materials and Methods

This single-center, prospective, randomized, controlled pilot study was conducted at the Methodist Richardson Medical Center between February and May 2023. The study aimed to evaluate the effectiveness of Mixed Reality (MR) technology in enhancing patient education for individuals scheduled to undergo pancreatoduodenectomy for a pancreatic mass. MR offers a unique advantage by providing immersive, three-dimensional visualizations that enhance patient understanding of complex surgical procedures. This is highlighted in our previous section, which cites studies demonstrating MR’s effectiveness in simplifying complex medical information and improving patient comprehension. Institutional Review Board (IRB) approval was obtained before study initiation.

Patients were eligible for inclusion if they were 18 years or older, fluent in English, and had no severe visual or hearing impairments that could interfere with engagement in the educational intervention. Exclusion criteria included individuals younger than 18 years, non-English speakers, and those with significant sensory impairments. The majority of candidates who met our inclusion criteria and were asked to participate agreed, with less than 5 declining. Our goal was to enroll 1–2 patients per week during the 3-month study period, which resulted in a total number of 19 patients. Many clinical patients were ineligible because the clinic addresses conditions beyond pancreatic masses.

Participants were randomized into two groups: a control group and an intervention group. Both groups received a standardized discussion led by the surgeon, which included information about the pancreatoduodenectomy procedure. The control group was provided with physician-drawn diagrams on a whiteboard to supplement the discussion, while the intervention group used the Microsoft HoloLens, an MR device, for an interactive and immersive educational experience.

The control group received in-office education from the surgeon, including explanations of surgical indications, relevant anatomy, and procedural steps, supplemented by two-dimensional diagrams drawn on a whiteboard. The intervention group also received in-office physician education on surgical indications and procedural steps. However, instead of the whiteboard, the relevant anatomy was explained using the MR device. The surgeon followed up with each patient in the intervention group to provide clarification as needed. As indicated in our results, most patients in the intervention group reported confidence in their understanding of the procedure and relevant anatomy following the MR device demonstration.

The MR program, developed by the Burnett School of Medicine Simulation Team and the lead author, consisted of a 10-slide interactive presentation, each lasting approximately 3–4 minutes. The program included 3D visualizations of patient anatomy and a step-by-step walkthrough of the pancreatoduodenectomy procedure [Figure 1]. Patients in the intervention group wore the HoloLens while either seated or standing, depending on their preference. The device is stand-alone and portable, requiring no external computer connection. The room was arranged to allow patients to move their heads freely and interact with the models from various angles.

All participants completed a pre-intervention survey assessing their date of tumor diagnosis, age, sex, education level, and the number of support members present during the visit. Additionally, a 5-point Likert scale was used to measure patients’ perceived understanding of their diagnosis, surgical procedure, and aspects of postoperative care. Following the educational session, a post-intervention survey was administered. This survey included the same 5-point Likert scale questions in a re-ordered sequence, along with an additional question assessing the patient’s comfort level with the upcoming surgery. The survey was designed by the research team, including surgical residents and medical students, based on validated instruments from previous studies on patient education and surgical consent [13].

Data were analyzed using Statistical Package for the Social Sciences (SPSS) for Macintosh, Version 28.0 (IBM Corp., Armonk, NY, USA). As the data were non-parametric, comparisons between groups were performed using the Mann–Whitney U test, while changes within each group from pre- to post-intervention were assessed using related-samples Wilcoxon signed-rank tests. A significance level of p < 0.05 was set for all statistical analyses.

3. Results

A total of 19 patients were enrolled in the study, with 8 patients assigned to the HoloLens intervention group and 11 patients assigned to the control group. The higher number of patients in the control group was due to scheduling availability for patients undergoing the Whipple procedure and the occasional unavailability of the HoloLens headsets, which were borrowed from the Burnett School of Medicine at Texas Christian University. The median age of participants was 67 years (interquartile range [IQR] = 12 years). The demographic data are summarized in

Table 1. Highest level of education completed by patient.

	Counts
Education Level	Control	HoloLens
High school	3	2
Trade school	1	1
Bachelor’s degree	4	4
Master’s degree	3	0
Doctorate	0	1

, which shows the highest level of education completed by patients in both groups. The two groups had a similar distribution of education levels, with the majority having completed at least a high school education, though there were slight differences in the number of patients with higher degrees between the groups.

Both the control and HoloLens groups showed improvements in understanding post-intervention, but the HoloLens group had a statistically significant increase in comprehension of the surgical procedure. The pre- and post-intervention survey results, assessed using a 5-point Likert scale, are summarized in

Table 2. The 5-point Likert scale survey results assessing patient perceived understanding.

Survey Component	Control (N = 11)	HoloLens (N = 8)
Condition/disease
Pre-intervention, median (range)	3.5 (1–5)	4.0 (3–5)
Post-intervention, median (range)	4.0 (3–5)	5.0 (4–5)
Z-score (p-value)	−1.28 (0.200)	−1.90 (0.058)
Expected surgical benefit
Pre-intervention, median (range)	3.5 (1–5)	4.0 (4–5)
Post-intervention, median (range)	4.0 (2–5)	5.0 (4–5)
Z-score (p-value)	−1.73 (0.083)	−1.73 (0.083)
Risks/side effects/complications
Pre-intervention, median (range)	4.0 (1–5) a	3.5 (2–4)
Post-intervention, median (range)	4.5 (3–5)	5.0 (4–5)
Z-score (p-value)	−2.04 (0.041) a	−2.60 (0.009)
Other available treatment options
Pre-intervention, median (range)	3.0 (1–5) a	4.0 (2–5)
Post-intervention, median (range)	5.0 (3–5)	4.0 (4–5)
Z-score (p-value)	−2.06 (0.040)	−1.63 (0.102)
Disease progression without surgery
Pre-intervention, median (range)	4.0 (1–5) a	4.0 (2–5)
Post-intervention, median (range)	4.0 (3–5)	4.5 (3–5)
Z-score (p-value)	−1.13 (0.257)	−0.82 (0.414))
Preoperative sequence of events
Pre-intervention, median (range)	4.0 (1–5) a	4.0 (1–4)
Post-intervention, median (range)	4.0 (3–5)	4.0 (2–5)
Z-score (p-value)	−2.06 (0.039) a	−2.06 (0.039)
Operative steps
Pre-intervention, median (range)	4.0 (1–5)	4.0 (1–5)
Post-intervention, median (range)	4.0 (2–5)	5.0 (4–5)
Z-score (p-value)	−1.20 (0.230)	−2.46 (0.014)
Recovery
Pre-intervention, median (range)	3.0 (1–5)	3.5 (1–4)
Post-intervention, median (range)	4.0 (2–5)	5.0 (4–5)
Z-score (p-value)	−1.59 (0.112)	−2.59 (0.010)
Operative results
Pre-intervention, median (range)	3.5 (1–5) a	4.0 (2–5)
Post-intervention, median (range)	4.0 (3–5)	5.0 (2–5)
Z-score (p-value)	−2.26 (0.024) a	−1.89 (0.059)
Planned follow-up
Pre-intervention, median (range)	3.5 (1–5) a	3.5 (2–5)
Post-intervention, median (range)	4.0 (3–5)	4.5 (4–5)
Z-score (p-value)	−2.05 (0.040) a	−2.12 (0.034)
Potential side effect management
Pre-intervention, median (range)	2.5 (1–5) a	3.5 (2–5)
Post-intervention, median (range)	4.0 (3–5)	4.5 (2–5)
Z-score (p-value)	−2.33 (0.020) a	−1.89 (0.059)
Informational point of contact
Pre-intervention, median (range)	3.0 (1–5) a	4.0 (2–5)
Post-intervention, median (range)	4.0 (2–5)	4.5 (2–5)
Z-score (p-value)	−2.04 (0.041) a	−1.73 (0.083)
Overall comfort level	4.0 (3–4)	4.0 (3–5)

Scale responses are indicated by 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree. ^a n = 10; two participants with incomplete data.

. The HoloLens group demonstrated a significant improvement in their understanding of the surgical steps (Z = −2.46, p = 0.014), while no significant differences were found in the control group (Z = −1.20, p = 0.230).

Overall, both groups reported high comfort levels regarding the surgery, with 75% of participants in each group feeling ‘comfortable’ or ‘very comfortable’ proceeding with the procedure. Despite these high comfort levels, the HoloLens group showed statistically significant improvements in several areas of understanding, including the operative steps, recovery, and risks/side effects/complications, as shown in Table 2.

For the HoloLens group, significant improvements were noted in their comprehension of the risks/side effects/complications (Z = −2.60, p = 0.009) and recovery (Z = −2.59, p = 0.010), with median post-intervention scores of 5.0 (range 4–5) compared to 3.5 (range 1–4) pre-intervention. In contrast, the control group showed improvements, but they were not statistically significant, particularly for recovery (Z = −1.59, p = 0.112) and risks/side effects/complications (Z = −2.04, p = 0.041). The intervention group also showed significant improvements in understanding the overall surgical procedure, with a median post-intervention score of 5.0 (range 4–5), while the control group only showed modest improvement in these areas.

The HoloLens group also showed improvements in understanding the overall condition/disease, expected surgical benefits, and the preoperative sequence of events [Figure 2]. For example, in the category of expected surgical benefits, the HoloLens group achieved a post-intervention median score of 5.0 (range 4–5), which was statistically significant (Z = −1.73, p = 0.083). While both groups showed improvements, the differences between the groups were more pronounced in the intervention group (see Table 2 for full results).

Both groups reported high comfort levels regarding the surgery, with the overall median comfort level being 4.0 for both groups. This suggests that the method of education did not affect patients’ comfort in undergoing surgery. However, the HoloLens group demonstrated a significantly higher understanding of surgical steps and other relevant aspects, such as recovery and potential side effects, compared to the control group. Importantly, these findings suggest that MR technology, particularly the HoloLens, may enhance patient comprehension of complex surgical procedures.

4. Discussion

This pilot study is one of the first to investigate the use of Mixed Reality (MR) technology in preoperative education for patients undergoing pancreatoduodenectomy. Our findings suggest that MR, particularly with the HoloLens, significantly enhances patient understanding and comfort regarding complex surgical procedures. The interactive 3D visualizations provided by the HoloLens allowed patients to better grasp the surgical steps, risks, and recovery process, leading to higher levels of understanding compared to traditional educational methods. These results are consistent with findings from previous studies exploring the use of MR in medical education, though few have specifically focused on patient education for complex surgeries like pancreatoduodenectomy.

Several studies have demonstrated the efficacy of MR for enhancing medical training, especially in the context of surgical procedures [6,9,14]. Other research has explored the potential of MR systems, such as Microsoft’s HoloLens and the Magic Leap, for enhancing surgical education and patient engagement [3,4]. These systems have shown promise in facilitating a better understanding of human anatomy and procedural steps, offering advantages over traditional 2D methods by providing interactive, 3D visualizations that can be manipulated in real-time [15,16,17,18]. Our findings are in line with these studies, as they too found that MR technologies like the HoloLens significantly improved understanding, particularly in complex medical concepts, when compared to static visual aids or verbal explanations.

Our study shares the advantage of enhanced visualization and patient engagement with the above-mentioned studies. However, our study is unique in its specific focus on patient education for pancreatoduodenectomy, a complex surgical procedure. While other studies have focused on medical training, our study sought to improve patient education as part of the informed consent process. Additionally, our study design incorporated a control group, which allowed us to compare the effectiveness of MR to traditional education methods. This rigorous approach provides a stronger basis for concluding that MR offers a significant improvement in patient education.

However, while MR technology offers substantial benefits, it is not without challenges. One limitation of our study is the relatively small sample size, which limits the generalizability of our findings. Future studies with larger and more diverse patient populations are needed to confirm these results. Additionally, while the HoloLens was well-received by most participants, not all patients may feel comfortable using such advanced technology. Some patients, particularly those with minimal familiarity with technology or those experiencing sensory impairments, may face difficulties engaging with MR systems. Although we excluded patients with significant visual or hearing impairments, further research is needed to assess the usability of MR in diverse patient populations, including those with varying levels of technological literacy.

Another limitation of this study is its focus on a single institution, which may not fully capture the potential challenges of implementing MR technology across different healthcare settings. Although the HoloLens did not disrupt clinical workflows in our setting, the feasibility of integrating MR into routine practice may vary depending on the resources available at other hospitals or clinics. The cost of MR devices, as well as the need for specialized personnel to operate and maintain them, may limit widespread adoption, especially in underfunded or resource-poor healthcare environments. However, as the technology becomes more accessible and affordable, it is plausible that MR devices could be integrated into clinical settings on a larger scale.

Additionally, while the HoloLens proved effective in educating patients about the surgical procedure, it is important to note that not all patients may benefit equally from this technology. Some patients may prefer more traditional forms of education or may feel overwhelmed by the technological complexity. Patient education preferences and the potential for MR to enhance comprehension could vary significantly based on individual patient characteristics, including age, education level, and prior experience with technology. In our study, the patient cohort had a high level of education, which may have influenced their comfort with using MR. Further studies should explore how MR impacts patient education in more diverse populations, including those with lower education levels or less familiarity with technology.

Regarding scalability, while the HoloLens is a promising tool, it is not yet universally available or affordable for every patient. The cost of devices and the training required to use them effectively remain significant barriers. If the cost of MR systems decreases and healthcare systems integrate these technologies into their standard practices, the potential to improve patient education and satisfaction could be substantial. However, for MR to be available to every patient, it would require substantial investment in infrastructure, training, and ongoing support, which may not be feasible in all healthcare settings. Additionally, the study, initiated in the first half of 2023, was not extended due to the graduation of several authors from medical school or residency programs. The last author, who maintains a research lab, may consider a follow-up long-term study, contingent upon availability and funding.

5. Conclusions

This pilot study demonstrates that MR technology, such as the HoloLens, can significantly enhance preoperative education for patients undergoing complex surgeries like pancreatoduodenectomy. By improving patient understanding and comfort, MR has the potential to play a crucial role in the informed consent process and patient-centered care. Additionally, the ease of use and integration of MR into clinical workflows highlights its feasibility for broader application in medical education. The process of generating the learning material was straightforward, and the HoloLens device proved to be user-friendly for both patients and clinicians, suggesting that MR could be a valuable tool for widespread use in various medical settings.

Given its potential to improve patient education, MR could streamline educational content, allowing healthcare providers to efficiently create tailored patient learning experiences, regardless of procedure complexity. This study sets the stage for further exploration into the broader use of MR in healthcare and its potential to improve outcomes and engagement across diverse patient populations.