Design and Usability Testing of an Augmented Reality (AR) Environment in Pharmacy Education—Presenting a Pilot Study on Comparison between AR Smart Glasses and a Mobile Device in a Laboratory Course

Abstract

An essential feature of pharmacy education is the teaching of theoretical knowledge with the support of practical work in the laboratory. When properly utilized, laboratory activities have the potential to enhance students’ achievement, conceptual understanding, and positive attitudes towards learning. In this pilot study, an augmented reality (AR) environment was designed and introduced for teaching laboratory skills in pharmacy education at the university level. The AR environment was used by pharmacy students (n = 36), featuring gate questions, information screens, Quick Response codes, think-aloud questions, and instant feedback. The environment was utilized with smart glasses and mobile devices with the aim of comparing the support to students’ performance. User experience was evaluated through self-efficacy beliefs and anxiety towards the technology. As a result, students found the environment a useful supplement to traditional laboratory teaching. Smart glasses and mobile devices were both accepted with great positivity but neither being clearly preferred over the other. Smart glasses were noted to provide sufficient feedback in the right stages of work. In contrast, mobile devices promoted the learning process more than the smart glasses. The self-efficacy results for mobile device use were higher, especially related to device handling and operating the AR environment. The pilot study gives educators valuable insights on the usability of AR technology in guiding laboratory tasks, although future work should involve larger and more diverse samples, as well as different learning tasks.

1. Introduction

Pharmacy education is based on theoretical and practical teaching with the objective of educating experts for work positions from drug development to official duties in the pharmaceutical sector, and for patient-centered duties in community and hospital pharmacies [1,2]. Commonly, pharmacy curriculums consist of lectures and practical studies, such as laboratory work and traineeships. In lectures, students learn the fundamentals of pharmaceutical sciences, whereas in the laboratory and during traineeships, students are introduced to chemicals, practical skills, and various instruments [3].

Laboratory courses have been argued to be an essential part of science education [4]. However, pharmacy education is undergoing changes, reflecting the evolution of pharmacy practice and the increasing adoption of new technologies for teaching and learning [5,6]. The challenge in healthcare education, including pharmacy, is that the amount of scientific knowledge is increasing faster than ever before, and new content needs to be incorporated into an already crowded curriculum [2]. Study programs now place emphasis on clinically-oriented practice and less on compounding-based practice [7,8]. However, the laboratory training and knowledge of compounding is elemental in securing patient safety. The content knowledge in compounding enables high quality patient counselling in practice. Furthermore, the graduates will be occupied in various pharmaceutical tasks such as drug industry, administrative duties and research-based teaching [9]. One of the challenges in physical laboratories is that, when student numbers are high and groups are formed for performing experiments, the possibilities to meet the expected learning outcomes can be compromised due to a lack of individualized learning. These challenges have been further exacerbated in times such as during COVID-19, or in times of resource shortages [10,11,12]. At the same time, in order to provide valuable learning experiences, courses must be constructed effectively, considering the time, labor, and materials associated with laboratory work [3]. However, students’ reactions to practical work can be negative, and this may reflect onto students’ views as a lack of a clear purpose behind laboratory work [13]. It has been observed that many students fear chemistry-related laboratory activities, and this is characterized by the disappointment students feel towards the laboratory work [14]. Thus, less interest in incorporating advanced laboratory courses into individual study plans is seen among students. Simultaneously, pharmacy education needs to embrace a variety of teaching and learning approaches to ensure that graduates meet the demands of a future workforce. Information and communications technology (ICT) competency is essential for pharmacy graduates as technology has been integrated into work environments [5]. The use of immersive technology in education has been shown to increase the confidence in ICT skills and digital competencies [15] that are increasingly important nowadays. The modernization and digitalization of health services has set increasing requirements for digital competencies in pharmacy practice [16].

In the education sector, the growing use of immersive technologies, such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), has generated new possibilities for enhancing students’ skills and ways of learning [17,18]. Augmented reality is an emerging technology in which virtual information is added on top of the real world, with continuous and implicit user control over the point of view and interactivity [19]. This technology comes in various forms, from wearables and smart glasses that use retinal projection to the more commonly used handheld displays, such as smartphones [19,20]. The layers of AR that are added can be sensory (sound, video, graphics, or haptics) or simply data-based.

The number of studies focusing on AR technology in education and other fields [21] has increased in 2013–2017 [22], and therefore AR technology implementation is increasing [23,24]. Educational associations have recognized AR as one of the promising technologies [25], and together with VR, they may have the potential to be a tool in education [22,26]. Educational usability of AR has been studied in all education sectors, from kindergarten to higher education [22]. Based on the studies, AR has been noted to enhance learning motivation and concentration on tasks [27], provide a more authentic learning experience [28], and improve critical thinking, communication, and problem-solving [29]. The price range for smart glasses is wide. However, one can certainly expect smart glasses prices to decline significantly in the coming years, as they start to become broadly adopted and therefore become mass produced consumer products [30]. Mobile devices, such as tablets and smartphones, have become affordable [31], and generally they are used to deliver study materials for large numbers of students [32]. However, at the university level, AR technology is more often used in courses of specific disciplines, with the emphasis on communication, creative media, computer science and information technology [33]. This technology has not yet been commonly utilized in the tertiary medical or health education environments. However, benefits have already been identified from using the AR for medical use, such as for visualizing organs prior to surgery [34], teaching dental students [35], anatomy [36] and pathology education [37] as well as administrating local anaesthesia in paediatric patients [38]. AR has been proven its feasibility also in prehospital care by assisting in electrocardiography [39], ultrasound [40] and triage of patients [41].

From the perspective of pharmacy education, information on the use and outcomes of AR technology is limited. In terms of teaching laboratory skills, Darr et al. [42] designed a virtually facilitated laboratory course including online quizzes. Regarding the use of AR, a virtual medicines module has been created, incorporating AR-facilitated images and audio or video files [31]. A similar tool for teaching pharmacotherapy has been studied by Schneider et al. [18], employing AR images and other media files for teaching purposes. In teaching chemistry laboratory skills, AR-facilitated tools have been tested [3,43]. All these AR technologies have been operated using smartphones or tablets, often utilizing commonly available AR platforms. For the purposes of pharmacy laboratory work, only one prototype tool utilizing AR smart glasses has been described, with its focus on the medication preparation process at a hospital [44]. In addition, a mobile-based AR application has been described to assist in the pharmaceutical compounding laboratories [45]. However, no AR technology oriented towards teaching pharmacy laboratory work has been described that would be suitable for both smart glasses and mobile devices in the university context.

Self-efficacy, a person’s belief of being able to perform a given task successfully, is a major determinant of whether a person will attempt a given task or not, how much effort will be expended, and how much persistence will be displayed while pursuing the task in the face of obstacles [46]. Researchers assume that students’ beliefs in their ability to succeed in chemistry tasks, courses, or activities have a powerful impact on their choice of science-related activities [47,48]. Thus, self-efficacy is proposed to be an important factor influencing attitudes towards chemistry, as well as anxiety related to laboratory studies [48]. Anxiety about using ICT refers to the unpleasant effects of negative emotions and evoked cognitive facts during real or imaginary interactions with technology [49]. Such a negative emotional state can impact technology access and be detrimental to productivity, learning, the social bonds of individuals, and their general well-being [50]. On the other hand, technology self-efficacy is the belief that one has the sufficient abilities and skills to be successful in technology-related tasks [51]. Since self-efficacy in general is related to realized behavior [47], its study is important in trying to understand whether students will be likely to use technology to support their education.

The aim of this pilot study was to develop an AR environment suitable for teaching pharmacy and laboratory skills in a course focusing on microbiology and asepsis. The AR environment was designed to provide instant feedback for university students and simultaneously facilitate students’ learning by utilizing smart glasses or a mobile device. The course was selected due to a high student to teacher ratio and the challenging laboratory work of susceptibility testing where students need extra guidance. The study objective was also to compare smart glasses against a mobile device regarding support for students’ performance in the laboratory in terms of user experience as well as technological anxiety and self-efficacy. The user experience was evaluated in terms of guidance, feedback and learning. The study goal was to acquire data regarding the suitability of AR technology for university-level studies, information on which is scarce from the perspective of higher education, especially pharmacy education.

In detail, the study objectives were:

(1)

To develop an AR environment for laboratory work guidance in pharmacy education, suitable for AR smart glasses and mobile devices;

(2)

To evaluate students’ attitudes towards AR technology in the form of technology orientation and user experience;

(3)

To provide insights for other higher education developers regarding utilizing potential novel digital education tools, in this case AR technology.

2. Materials and Methods

2.1. Participants

A total of 36 students participated in this study during the years 2019–2020. Ten second-year pharmacy students (women n = 8, men n = 2) participated by using AR smart glasses from November–December 2019. Participation in the study was voluntary, and informed consent was obtained from the students. The study was conducted according to the ethical guidelines in Finland. The AR mobile device application was tested by 26 second-year pharmacy students (women n = 22, men n = 4) in December 2020. Testing the mobile device application was voluntary.

2.2. Study Design

This experimental pilot study consisted of four phases, employing pre work ques-tionnaires, practice of using the AR environment, performance of the laboratory work with the aid of AR and post work questionnaires (Figure 1). At first, participants filled in the PRE questionnaire regarding self-efficacy (TSE) beliefs and anxiety (ANX) towards technology. After that, the students were able to practice using the AR environment and its functions for a while on their own. For those using smart glasses, a tutorial video was shown. When the students felt comfortable with the smart glasses or the mobile device, they were instructed to begin the laboratory work and proceed according to the AR environment’s instructions. In general, carrying out the laboratory work with the aid of the AR environment took 1–1.5 h. After the laboratory work, students filled in the POST questionnaire of TSE and ANX towards AR technology as well as the question sheet regarding general feedback and user experience of AR.

2.3. Laboratory Work

The pilot study was carried out during a compulsory laboratory course focusing on pharmacognosy, microbiology and asepsis (course description in Supplementary Material File S1). The student number in the laboratory course has been between 140 and 180, and students are attending the course in several groups. Prior to taking this course, the majority of students has laboratory work experience from laboratory courses in pharmaceutical chemistry and pharmaceutical technology. In the second study year, the students have also taken lecture courses in anatomy and physiology, microbiology and immunology, basics of biosciences, organic chemistry, drug formulations, pharmacology as well as disease theory. During the second and third study year, students participate in courses of pharmaceutical chemistry and analytics, biopharmacy, toxicology, biological drugs and pharmacotherapy and patient counselling. The students also have two compulsory pharmacy practices in retail pharmacy and hospital pharmacy.

AR technology was implemented during a laboratory exercise in antimicrobial susceptibility testing using a broth microdilution method. Susceptibility testing was chosen, as it contained steps and elements that are new for the majority of second-year pharmacy students, bringing about a need for further guidance and instant feedback. Typically, these elements are microbial culturing, serial dilutions of an antimicrobial substance, choosing the correct culture medium in a proper volume, finding appropriate aseptic tools, and using the multichannel pipette with a multi-well reagent reservoir and a well plate. In addition, the course is an introduction to using aseptic techniques in practice for most of the course’s students.

In the broth microdilution method, well plates were filled with culture medium containing inoculated bacteria and dilutions of antimicrobial substance. Susceptibility testing was carried out using a 96-well plate with the addition of resazurin dye as a bacterium cell viability indicator. The aim of the laboratory work was to determine the MIC (minimum inhibitory concentration) value of an antimicrobial substance for the tested bacteria. The tested antimicrobial substances were either ampicillin or penicillin. The tested bacteria were either Staphylococcus aureus, Escherichia coli, Proteus mirabilis, or Bacillus subtilis. The students could choose the tested antimicrobial substance and bacterium themselves.

The laboratory work was carried out during three consecutive days. On the first day, the bacteria to be tested were inoculated to ensure the log phase of the growth in which cell numbers increase. On the second day, serial dilutions of the antimicrobial substance and the resazurin dye dilution were done. In addition, the susceptibility assay was pipetted on the 96-well plate and put into overnight incubation at 37 °C, which is the optimal temperature for the growth of these bacteria. On the third day, the success of the susceptibility testing was visually evaluated based on the color of the cell viability indicator resazurin. After visual evaluation, the MIC value was calculated for the antimicrobial substance. During the laboratory work, the emphasis was on asepsis, correct pipetting order, as well as the rational use of laboratory reagents and equipment. Detailed laboratory work instructions for carrying out the susceptibility testing are presented in Supplementary Material File S2. This study focused on laboratory day two, which is when students particularly need guidance. Carrying out the susceptibility testing on day two takes normally 1–1.5 h.

2.4. Design of the AR Environment

The interactive AR environment was developed in collaboration with researchers at the University of Helsinki and Sciar Company Ltd. (Helsinki, Finland). The AR environment content was designed by the researchers at the University of Helsinki, whereas the Sciar Company Ltd. was responsible of the technical implementation. The same AR environment was utilized in the use of AR smart glasses and mobile devices.

The AR environment utilized Vuzix AR smart glasses (Vuzix M300XL). The glasses presented necessary information right in the field of vision, in parallel with observing the real world. The smart glasses were equipped with a Quick Response (QR) code reader and could be controlled via an external wireless keyboard or integrated buttons on the glasses. As the mobile devices for comparison, mobile phones and tablets compatible with the Android operating system were used. The AR application did not project its virtual elements on top of the view of real world. However, the mobile devices’ video cameras were used to record the QR codes. To utilize mobile devices, students downloaded the Sciar Co-Pilot application from the Google Play Store and used either their own mobile devices or ones provided by the university. Video of the Sciar Co-Pilot application can be found at Supplementary Material Video S1.

The AR environment was designed to provide a step-by-step digital environment with various interactive elements to ensure that the student is working with the correct materials and preparing the correct solutions, while simultaneously promoting their learning. The interactive elements were the gate questions, QR codes, information screens, and feedback, as well as the think-aloud questions. A flow chart of the AR environment is presented in Figure 2, and video of the use of AR smart glasses can be found at Supplementary Material Video S2. To visualize the user’s interface with the AR smart glasses, please find the screen shot of the AR work flow concept, provided by Sciar Company Ltd., in the Supplementary Material Figure S1. For the AR environment, please see also Södervik et al. [52].

In the beginning, each student identified his-/herself with an identifier code that generates individual digital log. The environment then presented them with gate questions, pausing the work until the student responded with the correct values, then letting them continue with the protocol. Gate questions ensured that no mistakes would be made while preparing the solutions, and they checked whether students had made the correct calculations before letting them proceed with the work. In addition, the students had to choose the correct reagents and laboratory tools, most critical for performing successfully in the laboratory. These reagents and tools were tagged with QR codes (

Table 1. Tools and reagents tagged with QR codes, and their justification as part of the laboratory work.

Tool or Reagent	Justification as Part of the Laboratory Work
Antimicrobial substance	Use of correct antimicrobial substance and concentration
Bacterial suspension	Use of diluted bacterial suspension on the 96-well plate
Incubation room	Use of the necessary 37 °C incubation temperature instead of 25 °C
Nutrient broth 50 mL	Economic use of nutrient broth in the preparation of antimicrobial substance dilutions
Nutrient broth 100 mL	Preparation of a sufficient volume of bacterial suspension for the susceptibility testing
Resazurin dye	Use of correct dye and concentration
Serological pipette 1 mL, 2 mL and 5 mL	Use of a sterile and rational-sized pipette for pipetting reagents and dilutions
Sterile water	Use of autoclaved water instead of non-sterile distilled or tap water

). The QR check indicated whether the reagent or tool chosen for the current task was correct or incorrect, thus eliminating mistakes made by the student. In case the choice was wrong, the student had to find the correct reagent or tool before the AR environment would allow them to proceed. In order to avoid leading the students’ answers, some incorrect or unnecessary tools and reagents were marked with inactive QR codes. Additionally, the system provided info-screens and feedback during the exercise. Info-screens contained further information about specific steps of the work or contained hints regarding gate questions. The feedback system was designed to update the student on the progress of the work and to give uplifting comments to motivate them. The AR environment also provided think-aloud questions (

Table 2. Think-aloud questions in the AR environment and their timing during laboratory work.

Think-Aloud Question	Step in the Laboratory Work
What the prerequisites for aseptic working are.	Use of correct antimicrobial substance and concentration
You pipetted from the highest concentration towards the lowest. Why you did so.	Preparation of dilutions of the antimicrobial substance
In the third step of the work, evaluate the growth in the bacterial suspension. Is the suspension turbid or not? What the bacterial suspension should look like and why.	Preparation of dilution of the bacterial suspension
What the most crucial steps are in the next pipetting phase.	Pipetting to the 96-well plate
After you are done with pipetting, how you will dispose of the used pipette tips and why.	Pipetting of the bacterial suspension to the 96-well plate
Why you have kept the multichannel pipetting reservoir in its packaging until now.	Using the multichannel pipetting reservoir
In which order you just pipetted the antibiotic dilutions. Was it from the highest concentration to the lowest or the opposite? Why? Could you have done it in the opposite direction?	Pipetting of the antibiotic dilutions to the 96-well plate
Do you need to change the pipette tips and why. To which column will you pipette the resazurin dye dilution first and why?	Pipetting of the resazurin dye dilution into the 96-well plate

), which encouraged the students to reflect on their work with the knowledge learned previously during the course. To give insights to teachers about students′ knowledge, the answers to the think-aloud questions were recorded with an external camera.

During the laboratory work, the AR environment generated a digital log of the students’ answers to gate questions as well as their choices of reagents and tools. From the digital log, the info-screens the students had visited and the time needed to perform each work step could be discovered.

2.5. Technology Self-Efficacy (TSE), Anxiety (ANX), and Usability of the AR Environment

Each student was given two fill-in forms. One form (PRE) was to be filled in before using the AR environment, and it explored self-efficacy (TSE) beliefs and anxiety (ANX) towards technology on a five-point Likert scale, modified from Bellini et al. The items in the fill-in form were coded in opposite directions according to the constructs they pertained to: TSE scales usually phrase the statements in a way that reinforces the individual’s capabilities, while ANX scales usually assume that capabilities are attenuated [50]. The second form (POST) was filled after using the AR environment, and it consisted of the same TSE and ANX statements. The second questionnaire also contained a question sheet with claims about the AR environment’s usability on a five-point Likert scale. On the usability form, there were nine items measuring students’ experiences regarding guidance (

Table 3. User experiences when using AR smart glasses or a mobile device. Data collected after use of the AR environment (POST). Values are shown as average item scores (± standard error of the mean, SEM) for each individual statement.

	POST Mean (SEM)
	AR Smart Glasses *	Mobile Device
1. The AR application gave guidance that was useful to me.	4.3 (0.15)	4.7 (0.16)
2. The guidance given by the AR application did benefit me while working.	4.3 (0.21)	4.7 (0.17)
3. The AR application gave me enough guidance.	3.9 (0.38)	4.2 (0.18)
4. The AR application gave feedback during right stages of the work.	3.9 (0.28)	3.3 (0.20)
5. The AR application gave feedback that was useful to me.	3.7 (0.15)	3.2 (0.21)
6. The AR application gave me enough feedback.	3.6 (0.34)	3.2 (0.27)
7. Working with the AR application supported my learning.	3.6 (0.34)	4.0 (0.25)
8. I would have learned better without the AR application.	2.6 (0.27)	2.4 (0.24)
9. The AR application checked my work in the right stages.	4.0 (0.30)	4.0 (0.21)

The respondents specified their level of agreement on a five-point Likert scale with the descriptors: (1) Strongly disagree; (2) Disagree; (3) Neither agree nor disagree; (4) Agree; (5) Strongly agree. * See also Ref. [52].

, items 1–3), feedback (items 4–6) and learning (items 7–9). This usability form was developed for the present study. Students were also able to give open-ended feedback regarding the use and comfort of the AR environment.

Descriptive statistics of score values were utilized for presenting the findings. The answers to open-ended questions were analyzed using content analysis.

3. Results

3.1. Usability

In the user experience results (Table 3), the smart glasses were mentioned for their capability of giving sufficient and useful feedback in the right stages of laboratory work. However, mobile devices were seen as promoting learning more than the smart glasses, based on the scores for the statement “Working with the AR application supported my learning” (4.0 and 3.6, respectively). Mobile devices were noted for their capability to give useful guidance that was beneficial to the students during work. In fact, the highest scores (4.7) were obtained from the mobile device statements “The AR application gave guidance that was useful to me” and “The guidance given by the AR application did benefit me during work.”

The responses to the open-ended questions displayed students being generally pleased with the opportunity to test AR technology, and they found it practical as well as pleasant to use. Regarding the AR smart glasses, one student commented, “Impressive technology and equipment. I can imagine that in the future this will be an additional tool in teaching,” whereas the opinion about using a mobile device was, “The mobile device application was a pleasant addition to laboratory work. The use of it made my working coherent and made me feel assured.” Students also pointed out the easiness and fast responsiveness of using QR codes.

However, some users said that the AR environment guided their working too strongly: “I felt like a robot when I was just following the instructions one after another.” Furthermore, the smart glasses were found to be heavy for long-term use, and some students had difficulties finding a suitable position for the smart glasses with regard to their eyes’ accommodation power. In addition, losing the connection between the smart glasses and the keyboard was an issue while moving around in the laboratory. Regarding the mobile device, users expressed concern for the safety of their own devices when using hazardous chemicals or bacteria. In addition, while performing microbiological experiments, poor aseptic quality of the work was mentioned.

Students also suggested improvements to the AR environment for future use. Regarding teaching, users asked for multiple-choice quizzes or further checkpoints for reflection. One proposed technological improvement was making the AR environment iOS-compatible.

3.2. Self-Efficacy (TSE) and Anxiety (ANX) Related to AR Technology

Students’ self-efficacy beliefs (TSE) and anxiety (ANX) towards AR technology before (PRE) and after (POST) the use of AR technology are presented as average values in

Table 4. TSE and ANX towards AR smart glasses in comparison with a mobile device, before (PRE) and after (POST) the use of AR technology. Values are described as average item scores (±standard error of the mean, SEM) for each individual statement.

Item	PRE Average (SEM)		POST Average (SEM)
	AR Smart Glasses	Mobile Device	AR Smart Glasses	Mobile Device
I am motivated to become acquainted with novel technology. (TSE)	4.7 (0.16)	4.0 (0.15)	4.4 (0.18)	4.5 (0.11)
I am trustful that I know how to use the tools within AR. (TSE)	3.6 (0.33)	4.0 (0.18)	3.9 (0.31)	4.3 (0.14)
I am curious to become acquainted with novel technology. (TSE)	4.8 (0.15)	4.2 (0.16)	4.6 (0.16)	4.3 (0.15)
I received sufficient instructions regarding the use of novel technology. (TSE)	4.3 (0.15)	4.7 (0.11)	4.2 (0.25)	4.8 (0.08)
I can operate effectively the functionalities of the AR system. (TSE)	3.5 (0.26)	3.6 (0.14)	3.8 (0.44)	4.5 (0.14)
I am able to use the AR equipment and associated technology. (TSE)	3.5 (0.26)	3.5 (0.19)	3.8 (0.36)	4.5 (0.13)
I am uncomfortable with the fact that AR technology will be really used in our studies. (ANX)	2.3 (0.49)	1.8 (0.19)	1.4 (0.22)	1.6 (0.16)
I question the need to use this new technology in the future. (ANX)	1.9 (0.24)	2.0 (0.14)	1.2 (0.13)	1.9 (0.14)
I am afraid of using the equipment incorrectly. (ANX)	2.9 (0.31)	2.5 (0.22)	2.8 (0.36)	1.8 (0.14)
I do not like the feeling of having to use novel AR technology in my studies. (ANX)	1.6 (0.17)	2.0 (0.19)	1.8 (0.33)	1.7 (0.19)
I am apprehensive about having to use the AR equipment. (ANX)	1.8 (0.29)	1.3 (0.09)	1.6 (0.22)	1.2 (0.08)
Average score of TSE values	4.1 (0.25)	4.0 (0.18)	4.1 (0.14)	4.5 (0.07)
Average score of ANX values	2.2 (0.20)	1.9 (0.16)	1.8 (0.23)	1.6 (0.10)

A five-point Likert scale was used to explore technological self-efficacy (TSE) and anxiety (ANX) towards AR technology. The respondents specified their level of agreement as points: (1) Strongly disagree; (2) Disagree; (3) Neither agree nor disagree; (4) Agree; (5) Strongly agree.

. For detailed results of TSE and ANX items, please see Supplementary Material File S3.

Initially, all scores representing TSE items when using AR smart glasses were above the neutral point (3), indicating that the students were confident about the use of smart glasses. In concordance with this, the item scores representing ANX were below neutral in all cases, demonstrating that the students did not hesitate to use the smart glasses. The highest decrease in individual item scores representing ANX was in the statement “I am uncomfortable with the fact that AR technology will really be used in our studies.” In this case, the item score representing disagreement with the statement changed from the initial average value of 2.3 to 1.4 after the user experience.

Regarding the use of mobile devices, the PRE responses to self-efficacy-related statements were in all cases above neutral (3), and all anxiety-related items were evaluated to be below 3. The direction of Likert item agreement remained the same after laboratory work. The most notable positive change in the scores was seen for the self-efficacy items, “I am able to use the AR equipment and associated technology” and “I can operate effectively the functionalities of the AR system.” For the mobile device, the average TSE score increased from 4.0 to 4.5, whereas for the AR smart glasses, the average PRE and POST scores remained the same (4.1).

When comparing the item scores of smart glasses and mobile devices, a major opposite finding was seen in the self-efficacy item “I am motivated to become acquainted with novel technology.” Here, the POST score increased for mobile devices (from 4.0 to 4.5) and decreased for the smart glasses (from 4.7 to 4.4). The same trend can be seen in the scores for the item “I am curious to become acquainted with this novel technology.” Mobile devices were favored, based on the differences in the PRE and POST scores for the TSE items “I can operate effectively the functionalities of the AR system” and “I am able to use the AR equipment and associated technology.” Regarding the anxiety-related items, a decrease in most POST scores was seen.

4. Discussion

In this pilot study, an AR environment was developed, suitable for giving guidance and immediate feedback in a pharmacy laboratory course. In order to evaluate the suitability for future use, the students’ user experiences, as well as technological self-efficacy and anxiety towards AR technology, were compared by implementing the novel AR environment in smart glasses and mobile devices.

Students saw the AR environment as a useful supplementary tool, but they did not wish AR to completely replace traditional laboratory teaching and instruction, despite the fact that the current generation of students is familiar with advanced technologies [3]. In addition, a study by Nounou et al. [45] supported the benefit of AR mobile application in a pharmaceutical compounding course. Furthermore, smart glasses and mobile devices have shown the potential in knowledge of human organ systems and physiology [53]. This supports the outcomes of earlier studies that found blended learning more preferable for healthcare professionals [5,54]. However, when designed and used effectively, the incorporation of AR technology could promote interest in, while reducing the tension associated with, learning laboratory skills [3]. As in the current study, visual materials provided by AR have been stated to help in carrying out experiments [43,55] by preventing cognitive loads [31]. Indeed, learning gains continue to be the most reported advantage of AR systems in education. In addition, the fact that AR systems increase students’ motivation and academic achievement could eventually reduce the costs associated with grade repetition and early university dropout, and the social problems that these events may cause [56]. Students in this study appreciated the easily accessible and informative laboratory work instructions that improve the mobility in the laboratory and increase confidence [57]. This finding supports the fact, that in educational settings, AR is mainly used to augment information and explain the topic [58]. One must keep in mind, however, that AR may be misinterpreted by some students, resulting in relying on AR content too much [43] and in a “cyborg feel” [59]. Attention should be also paid to not superimposing the information that can distract the students. AR, especially smart glasses, can be also intrusive technology as the device can interrupt the natural interaction with others [58]. However, many students in this study thought that having more interactive features embedded within the AR environment would be valuable in terms of learning. Similar additions, such as pop-up buttons providing information [3], audio files [43] or multiple-choice conceptual questions [56] and video-materials [45] have been suggested for AR tools to support laboratory teaching.

AR smart glasses seemed to raise higher interest among the students than the mobile device. However, the self-efficacy results for mobile devices were higher afterwards, especially related to handling the device. In addition, the anxiety-related items hinted towards mobile devices being a more suitable choice for laboratory teaching. According to Henrysson et al. [60], mobile devices offer an ideal platform for AR applications. They are portable, very cost-effective, and easy to use due to intuitiveness. However, a physical constraint has been detected in the distorting effect that wide-angle mobile phone cameras have compared to the real world as viewed through the eye [61]. In addition, in a comparison between a tablet and smart glasses, the evident advantages of glasses were the stability of tracking, the fact that glasses are hands-free, and safer physical movement in the room without the need to look up [59]. Smart glasses also allow for concentrating on the task at hand without the need to turn the look [62,63].

When it comes to smart glasses, one can expect that wearers of regular glasses may have more difficulties with them because they either cannot wear their spectacles under the smart glasses, or because the smart glasses do not fit that well. This concern was not observed in the user experience feedback forms, contrary to the study by Riedlinger et al. [59] and Thees et al. [64]. Issues to be dealt with are the clarity of, and visual elements in, AR technology. Attention must also be paid to the comfortableness of AR equipment, such as symptoms of dizziness [53] and the heaviness of the equipment which has also been noted by Othman et al. [44]. One notable concern that needs to be addressed is the safety of laboratory chemicals during AR device use. As found by Hu et al. [57], AR devices might be inconvenient while wearing laboratory gloves. This may be overcome by providing dedicated AR devices for laboratory work.

In our study, students’ initial self-efficacy beliefs and anxiety towards AR technology indicated that the participants were confident in, and motivated to try, the new technology. Before using the AR environment, average TSE scores were rather similar, indicating that the students judged themselves as equally competent in using either the AR mobile device application or AR smart glasses. Their experiences on working with AR-assisted technology showed less anxiety after the use of AR technology. Students display variation in terms of their willingness, readiness, and self-efficacy beliefs when it comes to testing new, technology-assisted learning environments, and this can potentially moderate their cognitive processes [50]. However, empirical studies and meta-analyses have shown that low-immersion simulations result in better cognitive outcomes and attitudes towards learning, than traditional teaching methods [65,66,67]. As AR enables the user to see the real world, it supplements reality instead of completely replacing it, thus providing a realistic learning experience for students [68].

Several previous studies have stated that AR is often complicated for students to use [29,69,70], and usability problems have been recognized as one of the biggest barriers to using AR in educational settings [58]. It is important to take care of the technical difficulties of integrating AR into different subjects, so as to not raise the risk of not achieving desired learning outcomes [71]. Characteristics such as small screen size and low network speed and battery capacity [26] have been the issues raised in studies on the educational applications of AR smart glasses and mobile devices. In addition, different color schemes will be necessary for AR applications that are intended to be used in conditions with large disparities in light levels. Changing color schemes depending on light levels to provide a pleasant user experience would be a task for the application developer. However, due to the hardware limitations, some environments would probably pose problems that would be hard to solve merely through application design [72]. Wireless networking of AR is one more matter to be focused on. Location-based and tracking requiring AR applications need wireless networking to access to a remote server to update the current information [73,74]. For 3D object manipulation, the wireless network is essential, when the 3D content registered is stored in the remote server or involving multiple users. Realistic AR experience is also as one of the future directions. For example, haptics is important when AR is applied in the real-word tasks to improve the user’s AR experience like he/she is manipulating the real object [74,75]. Dunleavy et al. [76] argue that the most significant limitation with AR technology exists in the stage of development of nascent software and the managerial complexity of the implementation process. In addition, the compatibility with different mobile devices is still an issue for certain AR applications [71], noted also by Nounou et al. [45].

There are some limitations to this study. The results are based on a rather small and unbalanced number of participants in one particular learning subject in the first stage of an enlarged study; therefore, generalizing the findings to other courses and students at large should be carefully deliberated. In fact, a systematic review of AR-based research in education identified small or medium size samples as a notable issue [58]. Furthermore, the present study would have benefitted from individual students using both technologies—smart glasses and mobile devices. This method would have, however, been too strenuous to the student, as it would have required performing the laboratory work twice. Next, while utilizing the AR environment, the instructional design had not been evaluated in order to be used as a teaching tool. However, this study comprises elements of instructional design evaluation. As participating in the study was voluntary, technology-oriented students may have attended the study more eagerly. In addition, as mobile device use was tested during the COVID-19 pandemic, the results can be affected by students’ receptivity to digital teaching tools. During its use in the study, the mobile device AR environment was still under development, and it was not iOS-compatible, which limited the selection of participants.

5. Conclusions

AR has the potential to offer new learning opportunities, with benefits for both teaching and learning. Various AR platforms are available, presuming careful consideration of affordability, user-friendliness, design, and availability in terms of the aims of using the technology [15]. As a developing technology, AR has barriers to overcome, such as usability and technical issues. However, this study was one of the first that implemented AR for teaching laboratory skills in pharmacy education at the university level. For the first time, such AR environment was designed that is suitable for both smart glasses and mobile devices. Students’ AR technology preferences between smart glasses and mobile devices were measured, with both being accepted with great positivity but neither being clearly preferred over the other. While the results of this pilot study require further investigation, it is clear that AR technology has the potential to create a more inclusive learning environment. For the future, it would be interesting to know whether other AR devices, such as smart glasses with a see-through display, lead to different results. More examples and large-scale longitudinal studies should be developed for educators to see the value of AR technology in teaching pharmacy and laboratory skills.