Next Article in Journal
Making Informed Decisions to Improve Restaurant Image Using a Hybrid MADM Approach: A Case of Fast-Food Restaurants in an Island of East Malaysia
Next Article in Special Issue
Augmented Reality, Virtual Reality, and Game Technologies in Ophthalmology Training
Previous Article in Journal
Toward an Ideal Particle Swarm Optimizer for Multidimensional Functions
Previous Article in Special Issue
The Impact of Student Characteristics for Working with AR Technologies in Higher Education—Findings from an Exploratory Study with Microsoft HoloLens
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Information
Volume 13
Issue 5
10.3390/info13050218
information-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Higher Immersive Profiles Improve Learning Outcomes in Augmented Reality Learning Environments

by
Aldo Uriarte-Portillo
1,
María-Blanca Ibáñez
2,*,
Ramón Zataraín-Cabada
1 and
María-Lucía Barrón-Estrada
1
1
National Technological Institute of Mexico, Technological Institute of Culiacan, Culiacan 80220, Mexico
2
Department of Telematics Engineering, Carlos III University of Madrid, 28911 Madrid, Spain
*
Author to whom correspondence should be addressed.
Information 2022, 13(5), 218; https://doi.org/10.3390/info13050218
Submission received: 29 March 2022 / Revised: 20 April 2022 / Accepted: 20 April 2022 / Published: 22 April 2022
(This article belongs to the Collection Augmented Reality Technologies, Systems and Applications)
Browse Figure
Versions Notes

Abstract

:
Augmented reality (AR)-based learning environments are argued to foster cognitive and emotional involvement. Immersion has been identified as one of the driving forces that promote learning in technology-based learning environments. This study evaluated the learning effectiveness and immersion appeal of an AR-marker-based learning activity targeted at practicing basic chemistry concepts. Data were collected from a cohort of 124 middle school students in Mexico and analyzed using pretest–posttest comparisons and cluster analysis. The results suggest that students with higher immersive learning profiles achieve better learning outcomes compared with those with lower immersive profiles.

1. Introduction

Immersive digital technologies such as augmented reality (AR) provide the user with a highly responsive and fully immersive experience of a constructed learning environment [1]. Powerful immersion involves several levels: sensory, actional, social, psychological, and symbolic [2]. The sensory level relates to visual, audio, and sensory stimuli provided by hardware. The actional level entails the perception by each student that his or her actions have an impact on the virtual environment. Immersive context with real world narratives can trigger symbolic immersion. Finally, psychological immersion is the “mental state of being completely absorbed or engaged in something” [2] (p. 3) and can happen when sensory, actional, and symbolic immersion are achieved [3]. In the educational arena, immersive environments influence attention, motivation, and academic achievement [4,5].
Despite these claims, few studies have investigated the relationship between immersion and learning outcomes in the context of AR environments that use PCs or tablets as displays rather than immersive hardware, and results so far have been conflicting [5,6,7,8]. While some studies have reported positive relations between students’ perception of immersion and learning outcomes [5,6], others did not identify this association in their results [7,8]. The subjective nature of immersion, which can be influenced by individual characteristics, may provide an explanation for the aforementioned contradictions. This study aims to contribute to this research topic by investigating how immersion differences might affect learning outcomes in middle school students using an AR-marker-based learning activity.

1.1. Immersion in Digital Environments

Immersion is considered one of the main driving forces behind student learning [5]. When highly immersed “students quickly enter a state of suspended disbelief, accept the blended real and digital environment, give their attention over to it, and engage in the variety of options available to them to access content related to the topic being addressed” [9] (p. 240).
Immersion is a subjective measure of the vividness offered by a system and the extent to which the system is capable of shutting out the outside world [10]. One of the most-used definitions of immersion is that it is “the participant’s suspension of disbelief that she or he is ‘inside’ a digitally enhanced setting” [11] (p. 66). Immersion can be understood as a technical concept to indicate “the extent to which the computer displays are capable of delivering an inclusive, extensive, surrounding and vivid illusion of reality to the senses of a human participant” [12] (p. 604) but also as a “psychological state characterized by perceiving oneself to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences” [13] (p. 277). Immersion, seen as a technical concept, is dependent on the hardware devices and is related to notions such as interaction, control, and usability. Meanwhile, immersion understood as a psychological phenomenon can be viewed as a graduated psychological process of engagement that may foster flow and/or presence [14]. In this work, we focus on the psychological process of engagement that is triggered by the students’ perception of technical aspects such as interaction, control, and usability.
Immersion is a term commonly used in games and more recently in virtual reality. However, until the pivotal study performed by E. Brown and P. Cairns (2004) [15], understanding the notion of immersion in contexts other than games was a considerable challenge. In their study, E. Brown and P. Cairns describe immersion as the process to achieve a degree of involvement with an activity mediated by technology. The process evolves through time and is controlled by several barriers that should be removed in order to achieve a deeper involvement in the activity. There are three steps to this process: engagement, engrossment, and total immersion. To achieve the engagement level the user needs to overcome the barrier of preference and master the activity by investing time, effort, and attention. Engagement is related to the notion of cognitive absorption [16] which is defined as a state of deep involvement with software which is influenced by two important beliefs: perceived usefulness and perceived ease of use. From engagement the user may be able to become further involved with the activity and become engrossed; in this second step the user’s emotions are directly affected by the activity. The involvement described at this stage mentions that the user becomes less aware of his/her surroundings and less self-aware than previously. This level stage presents some parallels with the state of flow immersion in the fact that attention is needed, sense of time is altered, and sense of self is lost [17]. From engrossment the user may be able to become further involved with the activity. Total immersion requires the highest level of attention and is related to the feeling of presence which refers to a user’s subjective psychological response to a technical system [17]. It is an individual and context user response, related to the experience of “being there” [12].

1.2. Augmented Reality in Education

Current lower-cost and higher-fidelity Augmented Reality (AR) technological developments have led to an explosion of experimentation and development of applications such as gaming, tourism, marketing, and education. The term ‘Augmented Reality’ refers to the superposition of digital information over the real world, that is, added to what the user perceives naturally, creating an improved version of reality [18]. From a technological viewpoint, AR applications must fulfill the following requirements: (1) combination of real and virtual worlds, (2) real time interaction, and (3) accurate 3D registry of virtual and real objects [18]. Two families of AR applications can be identified, namely marker-based and location-based AR. The former requires specific labels to register the position of 3D objects on the real-world image and has been employed in a greater number of interventions than the latter. On the other hand, location-based AR applications use global positioning systems (GPS) to get the accurate location of physical objects. Regarding technological equipment used in AR applications, three main generations of hardware can be identified: the first generation corresponds to the use of desktop devices to interact with AR applications, the second introduces the use of mobile and tablets, and the last generation is based on the use of AR glasses. Each generation has provided a higher level of immersion than the previous one [19].
In the educational arena, augmented reality has the potential to improve not only conceptual understanding and knowledge, but also student skills, such as problem-solving, collaboration, and communication [20]. AR-based learning applications range from STEM education [21,22,23,24,25] to arts and humanities [26,27,28,29]. The targets cover participation going from early childhood education up to higher education and training [30,31,32,33]. The interventions have measured not only cognitive outcomes but also affective factors such as motivation [34,35], engagement [36,37], flow [38,39], presence [40,41], and immersion [42,43,44]. In general, the interventions have shown moderate to high values of affective involvement by the students, while students have shown values that range from low to moderate on variables that measure the cognitive factors. Regarding immersion, location-based AR learning environments have proved to provoke immersion and support learning due to their possibilities of building blended spaces that foster a sense of full absorption in the AR activity [45], while marker-based AR learning environments have fostered immersion when used in learning situations enhanced with activities that combine narrative situations or serious games [42,43]. In this work, we will focus on marker-based AR learning environments without support of narrative or game situations.

1.3. Aim of the Study and Research Questions

In response to the aforementioned issues, the aim of this study was to assess the learning effectiveness of an augmented reality experiential activity for practicing the basic principles of chemistry and the level of immersion achieved by the middle school students who participated in the intervention. The activity was designed according to the curricular objectives and subject matter of the Mexican Chemical curriculum. The following research questions shaped this study:
RQ1: Do students improve their conceptual understanding of basic concepts of chemistry after using the AR-based learning tool?
RQ2: What is the degree of immersion that students achieve by using AR teaching material?
RQ3: How do students’ degrees of immersion relate to learning outcomes in chemistry-based instruction using an AR-based learning tool?
The study is unique in that it investigates the use of AR technology within real school settings for practicing middle school level chemistry. Results from this study can help us to better understand whether AR technology can be effective in promoting student immersion and if it might contribute to a better understanding of the impact of AR on learning outcomes.

2. Methodology

2.1. Participants

In this study, 124 middle school students (age 14–17, M = 14.5, SD = 0.68) from six Mexican schools were evaluated and surveyed. One student did not complete the tests and thus was not considered for the analysis of this study. Among the 123 respondents, 65 (52.8%) were female and 58 (47.2%) were male. The learning activity was mandatory for all students. Informed consent to participate in the activity was requested from all parents or legal guardians of involved participants.

2.2. Learning Application

ReAQ is a marker-based augmented reality learning application constructed by the authors of this study. It was designed with the aim of helping middle school students to practice basic concepts of chemical bonds and reactions. The learning application conforms to the Mexican middle school chemical curriculum.
ReAQ provides AR learning activities to cover the main concepts of five topics: (1–3) study of compounds with a covalent, ionic, or metallic bond; (4) acid–base reactions; and (5) the use of chemical elements necessary to sustain life, such as hydrogen or carbon. Students must correctly answer a minimum number of exercises on each topic. Students receive visual feedback when answering each exercise, if the answer is correct the 3D representation of the compound will appear with a short explanation, otherwise a 3D effect will show the problem as it would appear in the real world. ReAQ might provide the necessary conceptual information to solve any exercise if the student requests it (see Figure 1).

2.3. Instruments

In the following sections we present the instruments we have used to collect, measure, and analyze data related to our research questions.

2.3.1. Conceptual Assessment Test

Knowledge pretests and posttests were conducted and analyzed to evaluate the effectiveness of the learning method on students’ comprehension of basic chemical bonds and chemical reaction concepts. Both tests were composed of ten multiple-choice questions, each worth 1-point towards the final score. Tests were designed by researchers, and they were validated by the schoolteachers participating in the study. A representative question from these tests is listed as follows:
Sample question:
“Name of the chemical compound that is commonly known as salt.
  • Sodium Chloride
  • Sodium Oxide
  • Potassium Chloride
  • Hydrochloric Acid.”

2.3.2. The Augmented Reality Immersion Questionnaire (ARI)

In this study, we translated the ARI questionnaire, created by Georgiou et al. [5] for immersion measurement in marker-based AR settings, to the Spanish language. The ARI questionnaire is composed of 30 questions with 5-point Likert-scale items; it measures three immersion macro factors, each of them composed of 2 factors: engagement is composed of interest (6 questions) and usability (4 questions); engrossment is composed of emotional attachment (5 questions) and focus of attention (6 questions); and total immersion is composed of presence (5 questions) and flow (4 questions). To test the questionnaire reliability, a Cronbach’s Alpha test was performed to evaluate the internal consistency of the survey. Cronbach’s alphas for interest, emotional attachment, focus of attention, presence, and flow ranged from 0.8 to 0.9, which is considered a very good level. However, usability had a low Cronbach’s alpha coefficient (0.5 > α), suggesting that the usability factor has a low internal consistency.

2.4. Procedure

One week prior to the intervention, students received a 70 min lesson related to the concepts involved in the activity and answered a 10 min pre-test to measure their level of subject knowledge.
In the subsequent week, each student received a tablet with the AR-based learning tool installed and the set of markers required for the intervention. Students also received a 10 min tutorial to instruct them about how to use the learning tool. The intervention lasted 50 min. After the completion of the intervention, students filled out a 10 min knowledge posttest questionnaire and the ARI survey in 10 min.

2.5. Data Analysis

Data were analyzed according to distribution using the Shapiro–Wilk test.
Descriptive statistics were used to summarize the results from the survey and the exams. Collected data did not come from normal distributions. Thus, we used non-parametric statistical hypothesis testing. Wilcoxon signed-rank tests were applied to determine statistical differences of two means in paired samples, whereas Wilcoxon tests were used to assess differences of two means in unpaired samples.
To study the relationship between the immersion and learning outcomes, the k-means algorithm was employed as a cluster analysis technique, in which subjects were classified in homogeneous groups, according to similarities in the immersion profiles [44]. Before running the k-means cluster test, we assessed the dataset for its clustering tendency and validated the subsequent use of a clustering tool by using the Hopkins statistic. The Hopkins statistic generated for these data was 0.72, showing that the data have a high tendency to cluster [45]. The optimal number of clusters was determined based on the recommendation of 26 indices handled by the NbClust R library [46].

3. Results

3.1. Research Question 1: Do Students Improve Their Conceptual Understanding of Basic Concepts of Chemistry after Using the AR-Based Learning Tool?

Table 1 shows the descriptive statistics of the pretest and posttest questionnaires.
A Shapiro–Wilk test of normality was performed to check if the samples came from a normal distribution. The results showed that pretest marks (W = 0.964, p-Value = 0.002) and posttest marks (W = 0.964, p-Value < 0.05) may not come from a normal distribution.
A Wilcoxon signed-rank test was performed to check the difference of pretest and posttest means. Results indicated that the mean posttest marks (M = 7.67) were significantly higher than the mean pretest marks (M = 6.5), V = 431, p-Value < 0.001, effect size r = 0.646.
The results suggest that students achieved a deeper understanding of the chemical concepts after using the AR-based learning tool.

3.2. Research Question 2: What Is the Degree of Immersion That Students Achieve by Using AR Teaching Material?

Table 2 shows descriptive statistics for the six subscales that describe immersion in AR-based environments. The highest mean scores were yielded by the interest subscale (M = 3.96) and the emotional subscale (M = 3.77). The lowest mean value was obtained by the usability factor (M = 2.95).
Table 3 shows the mean scores and standard deviations for each of the items that compose the interest factor of the ARI questionnaire [5]. The highest mean scores correspond to items I1 and I5 which state that the augmented reality learning scenario was different to those previously explored by students (I1) and that they found the activity attractive (I5). Over 89% of the students thought that it was mostly true or very true that augmented reality activities were different to learning scenarios previously explored. More than 84% of the respondents indicated that it was mostly true or very true that the activities were attractive.
Table 4 shows the mean scores and standard deviations for items on the emotional attachment subscale of the ARI questionnaire [5]. The highest mean score was yielded by item E3 which states eagerness to complete the learning activities successfully. A total of 80% of the participants indicated that it was mostly true or very true that they were eager to complete the learning activities successfully.

3.3. Research Question 3: How Do Students’ Degrees of Immersion Relate to Learning Outcomes in Chemistry-Based Instruction Using an AR-Based Learning Tool?

The cluster analysis technique k-means algorithm was used to identify the impact of students’ immersive profiles (students’ responses for each ARI factor) on learning outcomes. Two clusters with different immersive profiles were obtained: a high immersion student profile (HI, n = 37 students) and a low immersion student profile (LI, n = 86 students). Students of the HI profile indicated high levels of interest, usability, emotional, focus, presence, and flow ARI factors.
To determine if the immersive profile of the students has any impact on their learning outcomes, we focus on four hypotheses:
Hypothesis 3.1 (H3.1). 
Students with a low immersive profile will obtain better posttest marks than pretest marks.
Hypothesis 3.2 (H3.2). 
Students with a high immersive profile will obtain better posttest marks than pretest marks.
Hypothesis 3.3 (H3.3). 
Students with higher immersive profiles will obtain better marks in pretest than those with lower profiles.
Hypothesis 3.4 (H3.4). 
Students with higher immersive profiles will obtain better marks in posttest than those with lower profiles.
A Shapiro–Wilk test was applied to determine the normal distribution of the data generated in the pretest of students with both immersive profiles. For students belonging to the LI cluster (N = 37, M = 6.05, SD = 1.9, w = 0.959, p-Value = 0.011) and for students of the HI cluster (N = 86, M = 6.68, SD = 1.69, w = 0.962, p-Value = 0.012). In both cases, the test results indicate that the data were not from a normally distributed population.
Similarly, a Shapiro-Wilk test was applied to determine the normal distribution of the data generated in the posttest of students with both immersive profiles. For students belonging to the LI cluster (N = 37, M = 7.16, SD = 1.72, w = 0.924, p-Value = 0.015) and for students of the HI cluster (N = 86, M = 7.89, SD =1.47, w = 0.925, p-Value < 0.01). In both cases, the test results indicate that the data were not from a normally distributed population.
Students with lower immersive profiles obtained higher posttest marks (M = 7.16, SD = 1.72) in comparison with their pretest marks (M = 6,05, SD = 1.9). Results showed that this difference was statistically significant (V = 50, p-Value ≤ 0.001). Therefore, H3.1 is satisfied.
The comparison between pretest (M = 6.68, SD = 1.69) and posttest marks (M = 7.89, SD= 1.47) between students with high immersive profiles indicated that the difference was statistically significant (V = 185, p-Value < 0.001). In particular, HI students got better marks in the posttest. Therefore, H3.2 is satisfied.
Descriptive statistics showed that students with lower immersive profiles (M = 6.05, SD = 1.9) obtained lower pretest marks than those with higher immersive profiles (M = 6.68, SD = 1.69); however, this difference is not statistically significant (W = 1333, p-Value = 0.149). Therefore, H3.3 is not satisfied.
On the other hand, the comparison of posttest marks between students of the HI and LI profile indicated that there was a statistically significant difference between the two groups. In particular, the HI students outperformed the LI students (W = 1222.5, p-Value = 0.03). Therefore, H3.4 is satisfied.
To sum up, the statistical analysis showed that the difference between students’ levels of immersion has an impact on learning outcomes; students with higher immersion had better learning outcomes than students with lower immersion levels.

4. Discussion

In this article, we provide empirical evidence for the relationship between student immersion and learning outcomes in marker-based augmented related settings. The results supported the initial hypotheses of the study: (1) students improve their learning outcomes after using the AR-based tool; (2) students experience immersion towards science learning within AR-marker-based settings; (3) students with higher immersive learning profiles achieve better learning outcomes than those who achieve lower immersive profiles.
When the measures of immersion were analyzed, the results showed that all factors except usability were moderately high. A deeper analysis of the results revealed that students had a high interest in the learning activity mainly due to the novelty factor and that they liked the learning activity. According to researchers in the area, interest is a relevant factor to achieve better learning outcomes [47,48]. AR technology has been identified as a technology that fosters students’ interest in learning activities sometimes because of the novelty effect [49,50]. The results also suggest that the students felt a high emotional attachment to the activity, mainly because they were eager to complete it successfully. In the ARI questionnaire, the emotional attachment is based on success expectation among learners, which in turn allows learners to control their learning processes. Emotional attachment is similar to the confidence factor in the A.R.C.S. motivational model [35], which has been used in several studies and has shown a positive relationship between motivation and learning outcomes in AR-based settings [34,51]. In terms of presence and flow, which best reflect the degree of immersion achieved, our results portray a moderate sensation of immersion [5]. These results are partially aligned with the ARI framework which states that the interest and usability factors are necessary to feel absorbed in an activity and, as consequence, achieve the sense of presence and flow defined as total immersion. Unfortunately, the results regarding the usability factor reflected that the students encountered barriers to using the application, which in turn affected the perception of self-absorption necessary to achieve a better immersion.
The differences in the mean scores from pretest to posttest were used to assess the impact of the AR learning activity on students’ learning outcomes. Findings showed a statistically significant increase in students’ knowledge after their participation in the AR-based learning activity. These results contrast the findings of previous studies in middle school children which showed that AR technology helps to achieve students’ positive affective outcomes but does not necessarily promote better knowledge outcomes [52]. On the other hand, these results meet outcomes of several studies which have concluded that AR-based learning environments have a positive impact on students’ learning outcomes [53,54].
A deeper analysis comparing pre- and post-test scores of students who achieved different levels of immersion in the intervention showed that students improved their knowledge outcome independently of their immersion level; however, those students who achieved higher immersion levels got better posttest scores than those who achieved lower levels of immersion. Our analyses suggest that AR-marker-based learning environments might foster students’ levels of immersion to a high enough extent to have a positive impact on learning outcomes. These results provide empirical evidence that learning outcomes depend on the level of immersion that students achieve in AR-marked based learning environments. Other studies have reported a positive impact of levels of immersion and learning outcomes in location-based augmented reality settings [5,55] on those AR-learning environments enhanced with narrative or in-game learning environments [7,44,45].
This study comprises several limitations which need to be addressed and discussed as directions to be considered for future study. First, it involved short-term retention of the knowledge; it is likely that a long-term retention evaluation would have provided more insight into the effectiveness of the AR application in middle school students. Restrictions in the timetables of the schools involved compelled us to conduct these short-term interventions. Further studies should resolve the aforementioned situation in order to ensure that the learning outcomes have been effectively acquired by the students. Second, collected data were self-reported. It could be interesting to find new ways of measuring the data. Third, more studies are needed to determine if the results obtained are due to the instructional design or the use of AR technology. Finally, the empirical part of the current study has been conducted over the middle educational sector in Mexico. This in turn, would hinder the generalizability of the current results to other educational sectors and other countries as well.

5. Conclusions

The study is unique in that it investigates the use of AR-marker-based technology within real school settings for practicing basic chemistry concepts at the middle school level. The results suggest that immersion in marker-based AR learning environments can be achieved. The findings also suggest that immersion is a complex psychological process involving several factors that contribute to the improvement of learning outcomes. Our work provides empirical evidence suggesting that not only higher levels of immersion, but even lower levels of immersion contribute to the improvement of learning outcomes. Therefore, the design of augmented learning applications should consider immersive factors in order to foster these applications as effective learning tools.

Author Contributions

All authors contributed to the study, conceptualization, and methodology of this work; the software was developed by A.U.-P.; the validation and the formal analysis by M.-B.I. and A.U.-P.; the investigation and resources by M.-B.I., R.Z.-C. and M.-L.B.-E.; writing—original draft preparation by A.U.-P. and M.-B.I.; writing—review and editing by all the authors; supervision by M.-B.I., R.Z.-C. and M.-L.B.-E.. All authors have read and agreed to the published version of the manuscript.

Funding

The authors acknowledge the eMadrid Network, which is funded by the Madrid Regional Government (Comunidad de Madrid) with grant No. P2018/TCS-4307. This work also received partial support from FEDER/Ministerio de Ciencia, Innovación y Universidades—Agencia Estatal de Investigación through project Smartlet (TIN2017-85179-C3-1-R) and received partial support from MCIN/AEI/10.13039/501100011033. The publication is part of the I+D+i project “H2O Learn” (PID2020-112584RB-C31). We also acknowledge the program PRODEP in Mexico that supported this work (511-6/2019-8474).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

We appreciate all the participants who took part in the experiment.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Schott, C.; Marshall, S. Virtual reality and situated experiential education: A conceptualization and exploratory trial. J. Comput. Assist. Learn. 2018, 34, 843–852. [Google Scholar] [CrossRef]
  2. Dede, C.J.; Jacobson, J.; Richards, J. Introduction: Virtual, Augmented, and Mixed Realities in Education. In Virtual, Augmented, and Mixed Realities in Education; Springer: Singapore, 2017; pp. 1–16. [Google Scholar] [CrossRef]
  3. Mills, N. Self-Efficacy in Second Language Acquisition. In Multiple Perspectives on the Self in Sla; Channel View Publications, Ltd.: Bristol, UK, 2014; Volume 73, pp. 6–22. [Google Scholar]
  4. Barab, S.; Dede, C. Games and immersive participatory simulations for science education: An emerging type of curricula. J. Sci. Educ. Technol. 2007, 16, 1–3. [Google Scholar] [CrossRef]
  5. Georgiou, Y.; Kyza, E.A. The development and validation of the ARI questionnaire: An instrument for measuring immersion in location-based augmented reality settings. Int. J. Hum.-Comput. Stud. 2017, 98, 24–37. [Google Scholar] [CrossRef]
  6. Rowe, J.P.; Shores, L.R.; Mott, B.W.; Lester, J.C. Integrating learning, problem solving, and engagement in narrative-centered learning environments. Int. J. Artif. Intell. Educ. 2011, 21, 115–133. [Google Scholar]
  7. Cheng, M.; She, H.; Annetta, L.A. Game immersion experience: Its hierarchical structure and impact on game-based science learning. J. Comput. Assist. Learn. 2015, 31, 232–253. [Google Scholar] [CrossRef]
  8. Hsu, M.; Cheng, M.T. Bio Detective: Student science learning, immersion experience, and problem-solving patterns. In Proceedings of the 22nd International Conference on Computers in Education, ICCE 2014, Nara, Japan, 30 November–4 December 2014; pp. 171–178. [Google Scholar]
  9. Cabiria, J. Augmenting Engagement: Augmented Reality in Education; Emerald Group Publishing Limited: Bingley, UK, 2012. [Google Scholar]
  10. Cummings, J.J.; Bailenson, J.N. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence. Media Psychol. 2016, 19, 272–309. [Google Scholar] [CrossRef]
  11. Dede, C. Immersive Interfaces for Engagement and Learning. Science 2009, 323, 66–69. [Google Scholar] [CrossRef] [Green Version]
  12. Slater, M.; Wilbur, S. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence-Teleoper. Virtual Environ. 1997, 6, 603–616. [Google Scholar] [CrossRef]
  13. Witmer, B.G.; Singer, M.J. Measuring presence in virtual environments: A presence questionnaire. Presence-Teleoper. Virtual Environ. 1998, 7, 225–240. [Google Scholar] [CrossRef]
  14. Jennett, C.; Cox, A.L.; Cairns, P.; Dhoparee, S.; Epps, A.; Tijs, T.; Walton, A. Measuring and defining the experience of immersion in games. Int. J. Hum.-Comput. Stud. 2008, 66, 641–661. [Google Scholar] [CrossRef]
  15. Brown, E.; Cairns, P. A grounded investigation of game immersion. In Proceedings of the Conference on Human Factors in Computing Systems, Vienna, Austria, 24–29 April 2004; pp. 1297–1300. [Google Scholar]
  16. Agarwal, R.; Karahanna, E. Time flies when you’re having fun: Cognitive absorption and beliefs about information technology usage. Mis Q. 2000, 24, 665–694. [Google Scholar] [CrossRef]
  17. Csikszentmihalyi, M. In Flow: The Psychology of Optimal Experience; Harper and Row: New York, NY, USA, 1990. [Google Scholar]
  18. Azuma, R.T. A survey of augmented reality. Presence Teleoper. Virtual Environ. 1997, 6, 355–385. [Google Scholar] [CrossRef]
  19. Mystakidis, S.; Christopoulos, A.; Pellas, N. A systematic mapping review of augmented reality applications to support STEM learning in higher education. Educ. Inf. Technol. 2022, 27, 1883–1927. [Google Scholar] [CrossRef]
  20. Me, F.; Hsu, Y. Mobile augmented-reality artifact creation as a component of mobile computer-supported collaborative learning. Internet High. Educ. 2015, 26, 33–41. [Google Scholar]
  21. Erbas, C.; Demirer, V. The effects of augmented reality on students’ academic achievement and motivation in a biology course. J. Comput. Assist. Learn. 2019, 35, 450–458. [Google Scholar] [CrossRef]
  22. Ibili, E.; Cat, M.; Resnyansky, D.; Sahin, S.; Billinghurst, M. An assessment of geometry teaching supported with augmented reality teaching materials to enhance students’ 3D geometry thinking skills. Int. J. Math. Educ. Sci. Technol. 2020, 51, 224–246. [Google Scholar] [CrossRef]
  23. Ibáñez, M.B.; Di-Serio, A.; Villarán-Molina, D.; Delgado-Kloos, C. Augmented Reality-Based Simulators as Discovery Learning Tools: An Empirical Study; IEEE Transactions on Education: Gainesville, FL, USA, 2015; Volume 58, pp. 208–213. [Google Scholar]
  24. Wojciechowski, R.; Cellary, W. Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Comput. Educ. 2016, 95, 353. [Google Scholar] [CrossRef]
  25. Manisha; Mantri, A. An Augmented Reality Application for Basic Mathematics: Teaching and Assessing Kids’ Learning Efficiency. In Proceedings of the 2019 5th International Conference on Computing, Communication, Control and Automation (Iccubea), Pune, India, 19–21 September 2019. [Google Scholar]
  26. Chang, Y.; Chen, C.; Liao, C. Enhancing English-Learning Performance through a Simulation Classroom for EFL Students Using Augmented Reality-A Junior High School Case Study. Appl. Sci. 2020, 10, 7854. [Google Scholar] [CrossRef]
  27. Paliokas, I.; Patenidis, A.T.; Mitsopoulou, E.E.; Tsita, C.; Pehlivanides, G.; Karyati, E.; Tsafaras, S.; Stathopoulos, E.A.; Kokkalas, A.; Diplaris, S.; et al. A Gamified Augmented Reality Application for Digital Heritage and Tourism. Appl. Sci. 2020, 10, 7868. [Google Scholar] [CrossRef]
  28. Leue, M.C.; Jung, T.; Dieck, D. Google Glass Augmented Reality: Generic Learning Outcomes for Art Galleries. In Information and Communication Technologies in Tourism 20; Tussyadiah, I., Inversini, A., Eds.; Springer: Cham, Switzerland, 2015; pp. 463–476. [Google Scholar]
  29. Lee, M. Rediscovering Neighborhood History Through Augmented Reality. In Proceedings of the 2021 4th IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2021, Online, 15–17 November 2021; pp. 60–64. [Google Scholar]
  30. Chen, C.; Chou, Y.; Huang, C. An Augmented-Reality-Based Concept Map to Support Mobile Learning for Science. Asia-Pac. Educ. Res. 2016, 25, 567–578. [Google Scholar] [CrossRef]
  31. Kapp, S.; Thees, M.; Strzys, M.P.; Beil, F.; Kuhn, J.; Amiraslanov, O.; Javaheri, H.; Lukowicz, P.; Lauer, F.; Rheinlander, C.; et al. Augmenting Kirchhoff’s laws: Using augmented reality and smartglasses to enhance conceptual electrical experiments for high school students. Phys. Teach. 2019, 57, 52–53. [Google Scholar] [CrossRef]
  32. Thees, M.; Kapp, S.; Strzys, M.P.; Beil, F.; Lukowicz, P.; Kuhn, J. Effects of augmented reality on learning and cognitive load in university physics laboratory courses. Comput. Hum. Behav. 2020, 108, 106316. [Google Scholar] [CrossRef]
  33. Uppot, R.N.; Laguna, B.; McCarthy, C.J.; De Novi, G.; Phelps, A.; Siegel, E.; Courtier, J. Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care. Radiology 2019, 291, 570–580. [Google Scholar] [CrossRef] [PubMed]
  34. Ibáñez, M.B.; Uriarte Portillo, A.; Zatarain Cabada, R.; Barrón, M.L. Impact of augmented reality technology on academic achievement and motivation of students from public and private Mexican schools. A case study in a middle-school geometry course. Comput. Educ. 2020, 145, 103734. [Google Scholar] [CrossRef]
  35. Li, K.; Keller, J.M. Use of the ARCS model in education: A literature review. Comput. Educ. 2018, 122, 54–62. [Google Scholar] [CrossRef]
  36. Wen, Y. Augmented reality enhanced cognitive engagement: Designing classroom-based collaborative learning activities for young language learners. EtrD-Educ. Technol. Res. Dev. 2021, 69, 843–860. [Google Scholar] [CrossRef]
  37. Drljevic, N.; Boticki, I.; Wong, L. Investigating the different facets of student engagement during augmented reality use in primary school. Br. J. Educ. Technol. 2022, 1–28. Available online: https://bera-journals.onlinelibrary.wiley.com/doi/abs/10.1111/bjet.13197 (accessed on 28 March 2022). [CrossRef]
  38. Ibáñez, M.B.; Di Serio, A.; Villarán, D.; Delgado, C. Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Comput. Educ. 2014, 71, 1–13. [Google Scholar] [CrossRef] [Green Version]
  39. Hou, H.; Lin, Y. The Development and Evaluation of an Educational Game Integrated with Augmented Reality and Virtual Laboratory for Chemistry Experiment Learning. In Proceedings of the 2017 6th Iiai International Congress on Advanced Applied Informatics (Iiai-Aai), Hamamatsu, Japan, 9–13 July 2017; pp. 1005–1006. [Google Scholar]
  40. Qin, Y. Attractiveness of game elements, presence, and enjoyment of mobile augmented reality games: The case of Pokemon Go. Telemat. Inf. 2021, 62, 101620. [Google Scholar] [CrossRef]
  41. Vrellis, I.; Delimitros, M.; Chalki, P.; Gaintatzis, P.; Bellou, I.; Mikropoulos, T.A. Seeing the unseen: User experience and technology acceptance in Augmented Reality science literacy. In Proceedings of the 2020 IEEE 20th International Conference on Advanced Learning Technologies (Icalt 2020), Tartu, Estonia, 6–9 July 2020; pp. 333–337. [Google Scholar]
  42. Salar, R.; Arici, F.; Caliklar, S.; Yilmaz, R.M. A Model for Augmented Reality Immersion Experiences of University Students Studying in Science Education. J. Sci. Educ. Technol. 2020, 29, 257–271. [Google Scholar] [CrossRef]
  43. Song, H.K.; Baek, E.; Choo, H.J. Try-on experience with augmented reality comforts your decision Focusing on the roles of immersion and psychological ownership. Inf. Technol. People 2020, 33, 1214–1234. [Google Scholar] [CrossRef]
  44. Georgiou, Y.; Kyza, E.A. Bridging narrative and locality in mobile-based augmented reality educational activities: Effects of semantic coupling on students’ immersion and learning gains. Int. J. Hum.-Comput. Stud. 2021, 145, 102546. [Google Scholar] [CrossRef]
  45. Han, J.; Kamber, M.; Pei, J. Data Mining: Concepts and Techniques, 3rd ed.; Morgan Kaufmann Publishers: Burlington, MA, USA, 2012; pp. 1–703. [Google Scholar]
  46. Banerjee, A.; Dave, R.N. Validating clusters using the Hopkins statistic. In Proceedings of the 2004 IEEE International Conference on Fuzzy Systems, Vols 1–3, Proceedings, Budapest, Hungary, 15–29 July 2004; pp. 149–153. [Google Scholar]
  47. Charrad, M.; Ghazzali, N.; Boiteau, V.; Niknafs, A. Nbclust: An R Package for Determining the Relevant Number of Clusters in a Data Set. J. Stat. Softw. 2014, 61, 1–36. [Google Scholar] [CrossRef] [Green Version]
  48. Byers, T.; Imms, W.; Hartnell-Young, E. Comparative analysis of the impact of traditional versus innovative learning environment on student attitudes and learning outcomes. Stud. Educ. Eval. 2018, 58, 167–177. [Google Scholar] [CrossRef]
  49. Jufrida, J.; Kurniawan, W.; Astalini, A.; Darmaji, D.; Kurniawan, D.A.; Maya, W.A. Students’ attitude and motivation in mathematical physics. Int. J. Eval. Res. Educ. 2019, 8, 401–408. [Google Scholar] [CrossRef]
  50. Huang, W. Investigating the Novelty Effect in Virtual Reality on Stem Learning. Ph.D. Thesis, Arizona State University, Tempe, AZ, USA, 2020. [Google Scholar]
  51. Ibáñez, M.B.; Delgado, C. Augmented reality for STEM learning: A systematic review. Comput. Educ. 2018, 123, 109–123. [Google Scholar] [CrossRef]
  52. Georgiou, Y.; Kyza, E.A. Relations between student motivation, immersion and learning outcomes in location-based augmented reality settings. Comput. Hum. Behav. 2018, 89, 173–181. [Google Scholar] [CrossRef]
  53. Ibáñez, M.B.; Di-Serio, A.; Villarán-Molina, D.; Delgado, C. Support for Augmented Reality Simulation Systems: The Effects of Scaffolding on Learning Outcomes and Behavior Patterns; IEEE Transactions on Learning Technologies: Gainesville, FL, USA, 2016; Volume 9, pp. 46–56. [Google Scholar]
  54. Wahyu, Y.; Suastra, I.W.; Sadia, I.W.; Suarni, N.K. The Effectiveness of Mobile Augmented Reality Assisted STEM-Based Learning on Scientific Literacy and Students’ Achievement. Int. J. Instr. 2020, 13, 343–356. [Google Scholar] [CrossRef]
  55. Krueger, J.M.; Bodemer, D. Application and Investigation of Multimedia Design Principles in Augmented Reality Learning Environments. Information 2022, 13, 74. [Google Scholar] [CrossRef]
Information 13 00218 g001 550
Figure 1. Student solving an ReAQ’s activity related to covalent bonds. The instructions list the markers available. The available options allow the student to return to the main menu, reset, ask for help, ask for audio instructions, or go to the next activity. The feedback includes information about the covalent bond that was asked to build in the activity.
Figure 1. Student solving an ReAQ’s activity related to covalent bonds. The instructions list the markers available. The available options allow the student to return to the main menu, reset, ask for help, ask for audio instructions, or go to the next activity. The feedback includes information about the covalent bond that was asked to build in the activity.
Information 13 00218 g001
Table
Table 1. Pretest-Posttest descriptive statistics.
Table 1. Pretest-Posttest descriptive statistics.
MeanSD
Pretest6.501.80
Posttest7.671.58
Table
Table 2. Augmented Reality Immersive descriptive statistics.
Table 2. Augmented Reality Immersive descriptive statistics.
FactorMeanSD
Interest3.960.72
Usability2.950.56
Emotional attachment3.770.76
Focus of attention3.590.91
Presence3.570.83
Flow3.570.88
Table
Table 3. Mean scores and standard deviations of the interest items.
Table 3. Mean scores and standard deviations of the interest items.
ItemMeanSD
I1I liked the activity because it was novel4.180.93
I2I wanted to spend time to participate in the activity3.900.91
I3The topic of the activity made me want to find out more about it3.621.05
I4I wanted to spend the time to complete the activity successfully3.790.98
I5I liked the type of the activity4.170.85
I6The AR application we employed captured my attention4.100.89
Table
Table 4. Mean scores and standard deviations of the emotional attachment items.
Table 4. Mean scores and standard deviations of the emotional attachment items.
ItemMeanSD
E1I often felt suspense by the activity4.050.85
E2I was curious about how the activity would progress3.991.03
E3I was impatient about completing the activity successfully4.080.91
E4I was often excited since I felt as being part of the activity3.551.07
E5I often felt that I was really in charge of the activity3.181.17
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Uriarte-Portillo, A.; Ibáñez, M.-B.; Zataraín-Cabada, R.; Barrón-Estrada, M.-L. Higher Immersive Profiles Improve Learning Outcomes in Augmented Reality Learning Environments. Information 2022, 13, 218. https://doi.org/10.3390/info13050218

AMA Style

Uriarte-Portillo A, Ibáñez M-B, Zataraín-Cabada R, Barrón-Estrada M-L. Higher Immersive Profiles Improve Learning Outcomes in Augmented Reality Learning Environments. Information. 2022; 13(5):218. https://doi.org/10.3390/info13050218

Chicago/Turabian Style

Uriarte-Portillo, Aldo, María-Blanca Ibáñez, Ramón Zataraín-Cabada, and María-Lucía Barrón-Estrada. 2022. "Higher Immersive Profiles Improve Learning Outcomes in Augmented Reality Learning Environments" Information 13, no. 5: 218. https://doi.org/10.3390/info13050218

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop