Mobile Mixed Reality for Experiential Learning and Simulation in Medical and Health Sciences Education

Abstract

New accessible learning methods delivered through mobile mixed reality are becoming possible in education, shifting pedagogy from the use of two dimensional images and videos to facilitating learning via interactive mobile environments. This is especially important in medical and health education, where the required knowledge acquisition is typically much more experiential, self-directed, and hands-on than in many other disciplines. Presented are insights obtained from the implementation and testing of two mobile mixed reality interventions across two Australian higher education classrooms in medicine and health sciences, concentrating on student perceptions of mobile mixed reality for learning physiology and anatomy in a face-to-face medical and health science classroom and skills acquisition in airways management focusing on direct laryngoscopy with foreign body removal in a distance paramedic science classroom. This is unique because most studies focus on a single discipline, focusing on either skills or the learner experience and a single delivery modality rather than linking cross-discipline knowledge acquisition and the development of a student’s tangible skills across multimodal classrooms. Outcomes are presented from post-intervention student interviews and discipline academic observation, which highlight improvements in learner motivation and skills, but also demonstrated pedagogical challenges to overcome with mobile mixed reality learning.

1. Introduction

Recently, there has been a shift in higher education and medical health education away from the traditional education practice of didactic lectures and tutorials [1,2] to self-directed [3] and online education [4]. Along with this multimodal (face-to-face and distance) shift in pedagogy, there is growing evidence that simulation improves learners’ competence and skills, especially when compared to traditional didactic methods or no simulation training [5,6], but cost can be a prohibitive factor [7]. This has led to equality issues as discipline experts and educators are expected to be at the forefront of technology and its use in the classroom [8], which is compounded when students are studying in multimodal learning environments [9].

To assist with this, innovative technologies, such as 3D printing, augmented reality (AR), virtual reality (VR), and mobile bring your own (BYOD) smartphones, are becoming ubiquitous and available for use in the classroom. Each technology has features that could make it useful in the classroom. Augmented Reality (AR) looks to augment the digital world with physical objects, allowing a real world user to seamlessly interact with digital components. Virtual Reality (VR) provides an immersive experience, removing the physical and putting the user into a totally virtual environment. BYOD provides a mechanism for these technologies to be delivered seamlessly using commercially available devices to a majority of users. Three-dimensional (3D) printing provides the technology for anything that can be digitally designed to be replicated in the real world, giving digital reality a real physical form. Mixed reality (MR), a continuum of these innovative technologies, provides a framework to position real and virtual worlds [10], resulting in the development of new paradigms, tools, techniques, and instrumentation that allow for visualizations at different and multiple scales and the design and implementation of comparative mixed reality pedagogy across multiple disciplines [11].

The New Media Consortium (NMC) Higher Education Horizon Reports specifically highlight AR/VR [12], 3D printing [13,14], and BYOD [12,15] as key educational technologies and drivers for learner engagement, but the uptake in education has been hindered by cost, expertise, and capability. This is now changing with the recent wave of low-cost immersive 3D VR technology by vendors such as Oculus Rift (oculusvr.com) and BYOD Mobile VR by Google through cardboard (vr.google.com/cardboard) and Samsung through Gear VR (samsung.com/global/galaxy/gear-vr/); powerful free interactive 3D visualisation game software platforms, such as Unity 3D (unity3D.com); and integrated AR plugins such as Vuforia by PTC (vuforia.com). This presents an opportunity to explore methods to provide accessible smartphone-driven BYOD training to learners across multimodal classrooms using cutting-edge mobile MR [16,17,18,19] using AR, VR, 3D printing, and free game engine technology [20].

This paper analyses two mobile mixed reality learning interventions across two Australian higher education health classrooms. The first intervention is focused on knowledge and learner perceptions with the second focused on learned skills. Although there is a paucity of research investigating both skills-development and knowledge acquisition through the use of virtual learning environments, most studies focus on a single discipline, focusing on either skills or the learner experience and a single delivery modality. This paper is unique by linking cross-discipline knowledge acquisition and the development of a student’s tangible skills across multimodal classrooms. For both mixed reality learning interventions, the decision was made to use the off-the-shelf, free game engine development platform, Unity 3D, the Google Cardboard Application Programming Interface (API), the Oculus Rift Software Development Kit (SDK) through Samsung Gear VR (to provide a mobile stereoscopic view), and the Vuforia AR plugin for Unity 3D to process the augmented reality markers. While each study has been presented independently [9,17,18,19,21], a comparison of the two provides validation for universities or lecturers seeking to adopt mobile mixed-reality within their health sciences and medicine courses, answering the research question, “What general affordances can the use of mixed reality for knowledge acquisition and skills development bring to the discipline of health education?”

2. Literature Review

The internet has assisted in generating vast reservoirs of knowledge regarding health and medicine, to the extent that students require a number of strategies when tacking the sheer amount of content within a modern curriculum. For example, in 1950 the doubling time for medical knowledge was 50 years, in 1980, 7 years, and in 2010, 3.5 years [22]. To keep up with the current medical literature in 2004, students studying epidemiology (patterns, causes, and effects of health and disease conditions) alone would require an estimated 627.5 h per month to evaluate the number of articles produced [23]. With increasing numbers of publications each year [24], it is expected that by 2020, medical knowledge will double every 73 days. As the required content increases, there are other factors contributing to the potential difficulty of studying health sciences and medicine. For example, many students studying medicine are now direct-entry undergraduates, compared to when medical schools were predominantly postgraduate, meaning that additional support and scaffolding is required whilst they develop learning strategies [1]. These and other changes in modern curricula are requiring universities to provide content in an efficient and effective manner while not allowing the requirements on students to become considerably more challenging.

However, modern technology can allow this, and provide health students with learning modules that are far more experiential that other modes [2,17,18] and learning that can be beneficially self-directed [3]. New multimedia knowledge can be updated instantly within an application, and important concepts presented to the student that can assist their understanding of overall concepts [4]. Through moving away from a textbook and into the virtual space, students can instantly engage with experiential learning, which provides a better method to understand, decode, and learn course material [21]. The increased engagement within the virtual world can also help to keep up with the changing technology-driven landscape of education [25], and its application in medicine and health science could be considered a welcome addition to the modern educator’s toolkit. In health education, there is growing evidence that simulation improves learners’ competence and skills [6], especially when compared to traditional didactic methods or no simulation training [5]. Ainsworth [26] highlights this by considering the differences between the representation technology and taking advantage of the differences between the visualization methods. Dalgarno and Lee [27] and Carbonell-Carrera and Saorin [28] explore this through the learning affordances of 3D virtual environments. In particular, Carbonell-Carrera and Saorin [28] highlight the immersive orientation of VR and AR, highlighting heightened skills development in spatial thinking and navigation with no negative effects across gender. Previous research investigating VR and AR (Figure 1A [21]) use within a health science and medical curricula found these modes to be as effective for student learning as the commonly used tablet-based applications. In addition, the study demonstrated that there is no impact on learning when anatomical lessons are presented in a more accessible mobile-VR platform, such as the Samsung Gear VR, in place of the more expensive VR platforms, such as the Oculus Rift (Figure 1B, [17]).

This enhanced competence afforded by simulation is especially important in skills development, an integral proficiency for students studying in health education. In particular, airways management education and hands-on training is essential [29] and changes attitudes and behavior for all health professionals [30]. This is especially true for trainee surgeons or paramedics studying high-priority invasive skills, such as direct laryngoscopy with foreign body removal in pre-emergency care [31], where students require confidence and experience to execute skills correctly. Physical task simulation trainers (manikins) are an ideal setting for novice learners to train these important hands-on airways skills, as they isolate specific tasks to enhance procedural or surgical techniques using three-dimensional (3D) parts of the body or limb to represent a part of the whole [32]. This, in turn, focuses training on the specific task, rapidly developing automatic skills [33] by reducing cognitive load [34], which leads to enhanced learning outcomes [35].

However, universities in general are lagging behind in innovative pedagogy, especially when students are studying from a distance [9], with most prior work formed around two-dimensional (2D) words and pictures [36], with less attention given to complex skills learning environments using interactive visualizations, games, and simulations [20]. Brydges et al. [37] explore these issues from a self-regulated learning perspective and recommend that learning design should support and help prepare individuals for future learning by assisting learners with self-regulated learning through simulation, including observation (e.g., watching a video), emulation (e.g., imitating the instructional video), measuring self-control (e.g., goal setting), and an observational measure of learning transfer.

A common discussion point across the health and medical education literature, and task simulation training, is the concept of simulation fidelity or “the degree to which a simulation looks, feels, and acts like a human patient” [38] (p. 389). This thinking has led to a perceived need for high levels of fidelity (realism) in health education simulations, requiring very expensive on-site equipment and intensive clinical residential training [7]. However, fidelity is imprecise and focus should be on the underlying principles [39] for effective learning, including physical resemblance (tactile, visual, auditory, and olfactory features of the simulator), as this can be reduced with minimal or no loss of educational effectiveness provided there is appropriate correspondence between the functional aspects of the simulator and the applied context, for example, contextual cues and spatial arrangement of components. Hamstra et al. [38] refer to this as “functional task alignment” (p. 387) and suggest that the choice of physical visualization for maximum training effectiveness depends more on the human functional factors, including context, task, stage of learning, learner ability, capabilities, task difficulty, and instructional features, and less on the simulator itself.

One approach to evaluate the value of making an effective simulation is to assign a cost to the simulated environment. Zendejas et al. [7] examine simulation effectiveness by placing a cost on the methods to enhance instructional design features, transfer of learning, learner engagement, and immersion, and relates this to simulator affordability, availability, mobility, and effectiveness. Although there are a range of intrinsic benefits in learning anatomy and physiology in health sciences and medicine through the use of VR [21], this method of VR is costly in these Zendejas et al. [7] terms, and not accessible to most students in these courses. As such, the growing interest in mobile-based VR, even to the extent that students are able to utilize their own smartphones or devices, is showing great promise in enhancing health and medical learning [17,18].

Given the increased impact in running face-to-face residential schools, the cost of high-fidelity simulations, and the general pedagogy shift towards multimodal education, new pedagogical approaches and methods of education delivery are required to assist students and improve equality especially in regard to self-directed simulation training and visualization. To assist with these issues, technologies such as 3D printing, augmented reality, virtual reality, and mobile bring your own devices—collectively referred to in this paper as mixed reality—are becoming available for use both commercially and freely through open source software development kits and free game engines and thus are able to be studied more readily in the multimodal classroom. Additionally, many educators and students already have skills in using applications such as Unity 3D and 3D design. It is a common teaching program now in high schools and design schools, and more and more people are becoming proficient in coding with excellent free resources available through Unity 3D and online communities. This assists in making the development of the content production lower, although one must still recognize that there is an associated cost in producing the simulations and that considerations need to be made for the visualization and simulation requirements in relation to budget.

3. Experimental Design

This paper analyses two mobile mixed reality learning interventions across two Australian higher education health classrooms. The first intervention is focused on knowledge and learner perceptions with the second focused on learned skills. For both mixed reality learning interventions, the decision was made to use the off-the-shelf, free game engine development platform, Unity 3D, the Google Cardboard Application Programming Interface (API), the Oculus Rift SDK through Samsung Gear VR (to provide a mobile stereoscopic view), and the Vuforia AR plugin for Unity 3D to process the augmented reality markers.

3.1. Knowledge Acquisition

In the knowledge acquisition and student perception intervention, (n = 46) 1st year volunteer physiology and anatomy students took part in lessons on the brain, brainstem, and spinal cord and provided written feedback on their learning experiences after completing a lesson in VR and AR. Participants were free to explore the devices and utilize whichever modes they preferred to learn in, with some participants using the Oculus Rift VR and mobile AR through tablet and other participants using the mobile Samsung Gear VR. Each of the participants enrolled in the course had already utilized the Oculus Rift VR learning through high-powered laptops as a regular component of their study. As such, this study focused on the introduction of mobile mixed reality, in the form of tablet-based AR and VR through the Samsung Gear VR (Figure 2), and investigated the perceptions of learning from these mobile devices.

In this intervention study, narrated anatomy lessons using 3D models for the brain, spinal cord, and brainstem were used. Models of the brain, spinal cord, brainstem, and complete human body were purchased from TurboSquid (turbosquid.com, New Orleans, LA, USA) and edited, colourised, and assessed for accuracy and visibility in the virtual environment (Figure 3).

For the anatomy of the spine, a formal narration by a specialist anatomist was used. This provided an overview of both anatomy and clinical significance of the spine. The delivery was formal and slow-paced to allow the learner to find the structures discussed. For the anatomy of the brain, a formal narration by a physiologist was used. This provided an overview of both anatomy and physiology of the brain. The delivery was formal and the pace was fast, giving less time for the learner to locate the structure in the model. For the anatomy of the brainstem, a more conversational-style narration by a physiologist and the researcher was used and discussed the different structures of the brainstem and their clinical relevance. The delivery style was casual and followed a conversation about the anatomical structure between the teacher and a student.

3.2. Skills Development

This project stemmed from a need, identified in Cowling, Moore, and Birt [9] through course evaluations, for more opportunity for distance students to practice skills (currently, they can only be practiced in a five-day hands-on residential school). It was found that online paramedic students are at a disadvantage in that they cannot train on physical manikins and use physical surgical instruments. Discussions between the lead author and the paramedics discipline academic were focused on what skill(s) could be re-created through a mixed reality approach that would provide the most benefit to the students. In the end, laryngoscopy (looking down the throat) for foreign body removal was chosen as it is classed as a priority invasive skill and supported by the requirement for hands-on training [29].

Specifically, the design of the simulation was linked to the key elements of the Australian, Queensland State Ambulance Service Airways Management Clinical Practice Procedures [40] and explored the following key skills: (i) Direct laryngoscopy (Large adult, using Macintosh Blade Size 4): “the technique used to achieve optimal visualization of the glottis for the purpose of oral endotracheal tube insertion or removal of a foreign body” (p. 342); (ii) Magill forceps: “removal of pharyngeal foreign bodies causing airways obstruction in an obtunded patient” (p. 354). The aim was to provide distance paramedic students a haptic, visual, and auditory feedback mechanic to assist in learning and practicing the airway skills in line with airways and simulation education research [5,6,29,37].

Student volunteers were then shipped a kit consisting of 1:1 scale 3D printed instruments of the actual physical tools (Laryngoscope, Mac Blade 4, and Magill Forceps) and provided with a smartphone augmented and virtual reality simulation application linked to the key skills and steps in [40]. In addition, the students also received instructions via a tutorial video on how to use the tools and were encouraged to practice for 1 week prior to the residential school scheduled for late in the semester.

The development of the tools, the feasibility of the design methodology, and specific learning results are available in two publications from the two phases of the study [18,19]. This is supplemented by access to the simulation, video demos, and downloadable tools from www.mixedrealityresearch.com/#paramedics. For reader support, a diagram outlining the tools and screenshots from the simulation across the two studies has been provided in Figure 4.

The focus of the simulation to be task-appropriate means that only pertinent information relating to the key learning outcomes was to be included and aligned with Hamstra et al. [38], “functional task alignment” (p. 387), and the fidelity research in [7]. To this end, a tutorial was provided explaining to the student how to hold the instruments and use the simulation through both text and audio cues. First, all markers need to be identified by the camera through a process of image recognition.

Then, the virtual objects are recognized within the smartphone/camera view of the actual physical tools. That indicates to the user that the simulation seen on their phone has ‘recognized’ the markers and makes them more confident to use the tool to practice their skills. An airways manikin dummy is displayed (in phase one this is monoscopic and in phase two this is stereoscopic), and a series of steps with audio and visual cues is presented to the user. The aim of the simulation is to follow the steps required to insert the Laryngoscope correctly, then the forceps remove a foreign body lodged in the patient’s throat. Cues are provided during the mixed reality simulation to indicate whether the procedure has been successful.

4. Methodology

For this research work, a design-based research (DBR) methodology [41] was used in implementing and conducting the research in the classroom (see Figure 5). Specifically, the four steps of Reeves’ [42] (p. 59) methodology were followed through (i) the analysis of the problem and literature review; (ii) design of the simulation solution; (iii) iterative implementation of that solution into the classroom by relevant discipline academics; and (iv) a loop back to design refinement and further iterative testing to assess the relevance of how the case studies presented led to the development of an effective mobile mixed reality simulation.

In the knowledge acquisition and student perception intervention, (n = 46) 1st year physiology and anatomy students provided written feedback on their learning experiences after experiencing a lesson in VR and AR. Participants were free to explore the devices and utilize whichever modes they preferred to learn in. In total, (n = 42) participants used the Oculus Rift VR and mobile AR through a tablet with (n = 37) participants using the mobile Samsung Gear VR. Ethics was approved by the Bond University Human Research Ethics Committee.

For the skills development intervention, all 2nd year distance paramedic students at an Australian University were invited to participate across the two rounds of the intervention (n = 159). The list was checked and edited to ensure that only students with compatible phones (able to install the application) were included in the study, and then a stratified sample of approximately 30 candidates from each round was selected from this list in a blind fashion without the participation of the discipline expert. When students arrived at the residential school, a pre-test was conducted with all students to assess skill competency prior to the residential school and to assess the difference between selected students and other students. There was a statistical significance in students passing the pre-test when they were given the tools compared with those that were just given the traditional learning artefacts. Ethics was approved by the CQ University Human Research Ethics Committee.

In both cases, included within both the skills and knowledge interventions were student qualitative comments and staff observations combined with additional open-ended one-on-one interview data collected from interviews conducted with the discipline academic for each of the interventions presented. These surveys and interviews were covered by the relevant ethics application submitted for each individual case. In these interviews, participants were asked to comment on how the mobile mixed reality simulation better supported the key learning attributes of problem solving, critical thinking, creativity, and innovation. They were further asked to reflect on how the mobile mixed reality technology affected these attributes. Data from the interviews was collated and transcribed, and themes were identified. These were then combined with existing data from each case study and are presented below.

5. Results

Results below are presented for each of the intervention cases, and then integrated in the discussion section.

5.1. Knowledge Acquisition and Student Perceptions

Participants provided written feedback after using each mode of learning for a different anatomical structure (Oculus Rift: Skull, Gear VR: Brainstem, AR: Brain). Participants rated the three devices in order regarding their perceptions of the most beneficial for their learning of anatomy (rating of #1 being the highest, #3 being the lowest perceived benefit). The highest mean rating was assigned to virtual reality on the Oculus Rift (n = 22), followed by augmented reality on the tablet (n = 19), and finally virtual reality on the Samsung Gear VR (n = 14). Some of the common themes as to why the Oculus Rift was rated as the best tool for learning over the other modes was that it provided “interactivity … engagement and minimal distraction”. Participants also found it easier to manipulate 3D objects within this device compared to the Gear VR or augmented reality application.

5.1.1. Mobile-Based VR

It was regarded as easy to absorb information because of the interactive podcast guiding the listener through each structure step-by-step. Participants noted in written feedback that the input to the Oculus Rift (mouse and keyboard) was more effective to navigate models than the bluetooth controller used for the Gear VR. Some examples of participant comments included, “Being able to move around and be inside the skull in conjunction with speech helps to learn”. “The Oculus Rift was similar to the Gear VR, but the mouse is easier to use”. Some of the problems experienced when using the Gear VR was feeling dizzy, finding it difficult to navigate, and not being able to use it properly with spectacles (glasses). In both the mobile and Oculus Rift forms of VR, participants highly regarded the visual aspects of the models and how interactive their learning experience was. They felt more involved with the learning module and found it to be a great approach to learning and alternative to the current methods. It helped participants view the structure from different viewpoints and gain a better understanding of its structure in 3D space.

5.1.2. Mobile Augmented Reality

Those that preferred augmented reality as a learning mode noted in written feedback that they highly regarded the easy access to this mode of learning and the ability to rapidly manipulate the model. “[Learning can be] easily done at home with no extra equipment needed”. However, there were multiple reports on having difficulties learning using this mode due to the flickering of the model, which was distracting. “The augmented reality was just too much of a struggle”. Most participants (n = 41) using the AR stated that they preferred learning using different information delivery approaches, and thought that AR accompanied the VR devices well, for surface anatomy. All participants reported this type of multimodal learning environment learning as useful for their studies. Offering AR as a mode of learning was greatly appreciated, as it targeted different ways in which people learn new content. It also provided more freedom in choosing their own pace of learning and technology they prefer to use in comparison to the traditional more rigidly structured learning, such as attending lectures.

In both the mobile AR and VR exercises, an audio podcast was played in the background, describing the features of the anatomical model being rendered, and participants noted that different structures were being highlighted as they were mentioned in the podcast, which “helped with overall understanding of the content”. In particular, subjects noted that the technology allowed them to dissect parts of the brainstem, which is not often possible with cadavers, and enjoyed having the ability to navigate in and out of the structure. Whilst many participants noted that these teaching methods should not replace the traditional lecture-style learning, the introduction of the Gear VR and augmented reality on tablets was thought to be a great supplementary material. The novelty of the technology also made it more enticing for the participant and they were more likely to use the learning mode. “I think it would be effective to include these technologies with the lectures to provide alternatives for students to learn … Having the podcast there while going through specific parts [of the model] at your own pace helps a lot … I am quite visual and learn better with interaction … it is great to see [anatomical structures] in 3D without having to use cadavers”. Two participants reported having troubles using the different learning technology, specifically with virtual reality. One participant was not sure how to navigate the environments, and the other participant found it troublesome to use glasses.

5.1.3. Mobile Learning Environment

The podcast running in the background of both the AR and Gear VR went for 5–7 min, which participants (n = 32) found to be appropriate. However, 12 participants suggested in written feedback a longer time would be more suitable for learning, whilst 2 participants desired a shorter time. Regarding the speed of the voice instruction within the podcast, participants equally enjoyed the rapid-speaking (n = 22) and slow-speaking (n = 24) approaches. More students (n = 29) preferred the podcast to be casual and in a conversational tone over a more formal monotone form of instruction. Participants regarded the casual as a better podcast style, because they found a conversation to be easier to follow than an in-depth explanation of the structure. Whilst the podcast audio did contain a lot of anatomical terms, the more casual explanation of them made it easier for the student to understand it. The participants were also able to relate to a conversation more than a formal explanation, making it easier to learn and increasing their comprehension of the subject. It made it clearer which part of the model was being discussed, as the student might not be familiar with some of the anatomical terms. The negatives of having a casual podcast is that it can be distracting and therefore guide the student away from the content. “I remember conversations better than facts … the podcast had the formal terms and the more casual explanation of them”.

Participants who favored the formal podcast style (n = 19) over the casual noted that all of the information in the podcast is relevant whereas a conversation-type of podcast introduces a lot of information that the student does not need for study. Because it is more concentrated on content, some participants found it was less distracting. However, others found this approach to be boring, therefore leading the student to pay less attention to the learning activity. “All information from the formal approach is relevant”. “I find it easier to remember formal style, although it is better with story examples”. There were also some students (n = 3) that regarded both types of podcasts as beneficial for their learning. “Both are helpful, but perhaps for immediate understanding casual style is better”.

5.2. Skills Development

Results from the skills development intervention with paramedic learners show a statistically significant improvement for students who were provided with the mobile simulation tools ahead of the residential school (for specific results from these two loops please see [18,19]). The discipline academic indicated that the learners were “very excited to get the chance to try this new mixed reality driven pedagogy”. They found it “great to be able to receive their simulation and tools and practice the skills at home”. It helped the learners to “feel more involved in the course and less isolated and alone”. When the 3D printed tools and augmented reality simulation were used by the learners, the discipline academic reported that “students had some struggles with the setup of the equipment and progression through the required steps”. This was especially prevalent in the later steps when introducing the Magill forceps and removing the foreign body. The learners commented that “there needs to be more depth in the simulation my hands seem to pass by the simulated airways manikin”. The learners commented that they “spent too much time focusing on the markers and not on the simulated airways manikin”, resulting in frustration when the simulation would restart due to them not progressing through the steps of the skill. Steps have been taken over the iterations to improve on these issues, with the current version reducing these problems with equipment setup, progression, and simulation depth by replacing the AR view with a VR view and increasing the size of the markers tracked.

Finally, some of the learners commented that they “did not get around to using the simulation as much as they would have liked”. This time struggle was attributed to other studies, work, and family commitments. On reflection, the discipline expert noted that she “should have encouraged more frequent use of the simulation with a reminder and linkage to the learning tasks”. Integration of the simulation into the course was reported to have been “relatively seamless”. Students exhibited “excitement about experiencing a new mode of learning”; however, some learners seemed to think that it may be “extra work” for them. These results suggest that whilst students found that these technologies helped them to be more involved, more work needs to be done to create accurate simulations that minimize technology limitations so that students can focus on skill development.

6. Discussion

Given these results, attention is now given to the research question, “What general affordances can the use of mixed reality for knowledge acquisition and skills development bring to the discipline of health education?” Note that whilst individual results have previously been reported, a correlation between these results and what they mean for the use of mixed reality in the general field of health education has not previously been presented.

Firstly, it is clear from the results that, as presented in the literature review, additional support and scaffolding is required in modern health and medical education courses to improve and develop new learning strategies [1] in light of ever increasing knowledge requirements and focus on skills. In particular, we are required as educators to develop these strategies by delivering content in smarter and more challenging ways while maintaining consistent and well-supported pedagogy within the classroom. Given this shift in pedagogy and the increased cost of education, in particular simulation and hands-on residential training [7], new affordable and mobile methods must be examined.

In both cases presented, these new methods were trialed and it was determined that new mixed reality multimedia applications delivered through AR/VR on BYOD is becoming possible through free game engines such as Unity 3D and free software APIs delivered by Google and Vuforia [20]. These modern technologies offer improved learning affordances [27], are supported through mobile mixed reality delivery, and allow health and medical students improved experiential learning [2,17,18]. Although not of the highest visual fidelity, these mobile platforms still enable the core tenants of enhanced learning outcomes [35], reduction of cognitive load [34], and “functional task alignment” [38] (p. 387) by focusing on the pedagogy and human functional factors of content, task, learning stage, ability, capabilities, task difficulty, and instructional features to transfer learning [39].

Secondly, in analyzing the results of the two mobile mixed reality interventions, there was a clear theme towards more interaction that emerged. In line with our previous work and illustrated in the student comments “I remember conversations better than facts”, “the podcast had the formal terms and the more casual explanation of them”, and “I am quite visual and learn better with interaction”, there is indeed a shift in the health sciences and medical education learners away from traditional didactic lectures and tutorials towards more conversational, self-paced, and visual interaction methods of learning. This reflects Brydges [37] in that simulation learning design must support and prepare the individuals by assisting them through self-regulated pacing. In particular, the mobile mixed reality pedagogy, as highlighted by learners making statements such as “having the podcast there while going through specific parts [of the model] at your own pace helps a lot”, and “great to be able to receive their simulation and tools and practice the skills at home”, promotes the self-directed and mobile online learning research of Murad et al. [3] and Clark and Mayer [4].

This shift in multimodal pedagogy is highlighted in simulation research [5,6], but has not yet been common in the mobile mixed reality learning literature given its infancy. In particular, within multimodal (online and face-to-face classrooms) there has been a disadvantage gap with students studying at distance [9] not only in the isolation of studying online but also when technology is introduced which is illustrated in the learner quote, “[I] had some struggles with the setup of the equipment and progression through the required steps”. However, this is improving with awareness of the underpinning technology and design methods [26]. In fact, mobile mixed reality technology allows for more than just learning skills at distance, as illustrated in the following quote: “[I] feel more involved in the course and less isolated and alone”.

Finally, it was clear from the two intervention studies that the simulation provided by mobile devices, although not as powerful as the desktop-computer-powered Oculus Rift mixed reality or physical simulators, has no negative impacts on the learners. In fact, the simulation improves learners’ competence and skills with the skills intervention showing statistically significant improvements in the learners that received the mobile mixed reality simulation tools prior to residential school. This supports the general simulation pedagogy consensus [5,6] and the argument of Zendejas [7] that fidelity is not the most important aspect of simulation pedagogy but rather the design and human affordances.

7. Conclusions

Mixed reality places the control of the learning experience into the hands of the participant. The self-directed approach allowed by this technology can enhance experiential learning, engagement, and experience whilst tackling challenging content in the medicine and health sciences. The use of mobile devices has brought the potential of this technology into each and every learning session. Previously, augmented and virtual reality was expensive, cumbersome, and exclusive to few educational institutions, and simulation pedagogy through physical simulation trainers is very expensive. However, the introduction of freely available game engines, such as Unity 3D, free AR/VR SDKs and mobile-based devices, such as tablets and smartphones with GPU-enabled processors and high-quality screens, can now reproduce adequate fidelity and education-worthy environments.

Nonetheless, although desktop-powered virtual- and augmented-reality devices have shown promise in recent years, educators have been wary of moving into the mobile space due to the limitations of these devices. This study validates the use of mobile devices in university undergraduate health sciences curricula, and shows that not only are these modes useful for enhancing the development of physical skills in students, but they are also received favorably. The student learning experience using mobile mixed reality is equivalent to the more expensive, cumbersome, and less accessible desktop-based devices across both the face-to-face and distance education modality cohorts.