Next Article in Journal
Machine Learning and Artificial Intelligence for a Sustainable Tourism: A Case Study on Saudi Arabia
Next Article in Special Issue
AI-Enhanced Decision-Making for Course Modality Preferences in Higher Engineering Education during the Post-COVID-19 Era
Previous Article in Journal
Towards an Urban Planning Scenario Model System—A Tool for Exploring Urban Uncertainty: A Case Study of Diaozhen, China
Previous Article in Special Issue
Determinants of Humanities and Social Sciences Students’ Intentions to Use Artificial Intelligence Applications for Academic Purposes
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Virtual and Augmented Reality in Science, Technology, Engineering, and Mathematics (STEM) Education: An Umbrella Review

1
Department of Computer and Information Technology, Purdue University, 401 N. Grant St., West Lafayette, IN 47907, USA
2
Department of Computer Science, Purdue University, 305 N. University St., West Lafayette, IN 47907, USA
*
Author to whom correspondence should be addressed.
Information 2024, 15(9), 515; https://doi.org/10.3390/info15090515
Submission received: 10 July 2024 / Revised: 14 August 2024 / Accepted: 19 August 2024 / Published: 23 August 2024
(This article belongs to the Special Issue Artificial Intelligence and Games Science in Education)

Abstract

:
The application of extended reality (XR) technology in education has been growing for the last two decades. XR offers immersive and interactive visualization experiences that can enhance learning by making it engaging. Recent technological advances have led to the availability of high-quality and affordable XR headsets. These advancements have spurred a wave of research focused on designing, implementing, and validating XR educational interventions. Limited literature focuses on the recent trends of XR within science, technology, engineering, and mathematics (STEM) education. Thus, this paper presents an umbrella review that explores the exploding field of XR and its transformative potential in STEM education. Using six online databases, the review zoomed in on 17 out of 1972 papers on XR for STEM education, published between 2020 and 2023, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. The results highlighted the types of XR technology applied (i.e., virtual reality and augmented reality), the specific STEM disciplines involved, the focus of each study reviewed, and the major findings from recent reviews. Overall, the educational benefits of using XR technology in STEM education are apparent: XR boosts student motivation, facilitates learning engagement, and improves skills, for example. However, using XR in education still has challenges that must be addressed, such as the physical discomfort of the learner wearing the XR headset and technical glitches. Besides revealing trends of using XR in STEM education, this umbrella review encourages reflection on current practices and suggests ways to apply XR to STEM education effectively.

1. Introduction

Extended reality (XR) is a powerful form of interactive visualization that immerses the user into the dataset, allowing the user to select the desired view naturally through head translations and rotations and providing the user with depth perception through distinct images for the left and right eyes with appropriate disparity [1,2,3]. XR is a continuum [4]. At one end of the continuum is virtual reality (VR), in which the user does not see any of their physical surroundings and sees exclusively the virtual environment rendered by the VR headset. In mixed reality (MR), the user sees their physical surroundings, into which 3D computer visualizations are integrated. At the other end of the continuum is augmented reality (AR), where computer visualization is limited to graphical annotations attached to elements of the physical world [5]. XR, in all its forms, has enormous potential in education [6,7,8]. It makes the learner feel present in the dataset, translating it into experiences, and by doing so, grabbing and maintaining their attention [9]. Moreover, XR also amplifies interactive visualization advantages, such as scaffolding learning through embodied cognition [10,11].
We are in the middle of an XR technology revolution, with several trillion-dollar companies firmly invested in it. Now, there are commodity-priced XR headsets that deliver a quality user experience at any point on the VR–AR continuum [12]. All-in-one XR headsets, such as Meta’s Quest 3, pack on-board power, graphics, and inside-looking-out tracking for a completely untethered user experience, providing a large field of view, high-resolution, and high-frame-rate immersive 3D visualization [13,14,15]. On-board cameras acquire the user’s physical surroundings for a video passthrough mode that supports MR and AR without disadvantages inherent to optical passthrough headsets, such as a small active field of view and low brightness [16,17,18].
The potential of XR in education and the recent leap-forward advances in XR technology have ignited research in XR educational intervention design, development, and validation at an explosive pace [19,20]. Therefore, in the current paper, we provide an umbrella review of recent research on XR educational interventions. An umbrella review is a review of meta-analyses and systematic reviews, meaning it only includes the highest degree of evidence [21]. To approach our investigation, we defined the following research question: What are the research trends regarding using extended reality technology for teaching and learning in STEM? Specifically, we aimed to identify the types of extended reality technology used for teaching and learning in STEM, specific STEM disciplines that have used extended reality technology for teaching and learning, the research focus of each study reviewed, and the major findings from each review.

2. Methods

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) method has been widely used for transparent and effective review reporting because it offers a set of guidelines to ensure that reviews are conducted in a clear, complete, and standardized manner [22]. To approach the research question, our study followed the PRISMA framework to conduct an umbrella review to identify the application of extended reality technologies, primarily in science, technology, engineering, and mathematics (STEM) education.

2.1. Search Strategy

Our final Boolean search string was created based on keywords reflecting our research question and specific aims. The keywords were broadly related to four categories: (a) main theme, (b) subject, (c) type of literature, and (d) type of extended reality technology. We showed our sample Boolean search strings and discussed our search strategy with a library expert before finalizing the current search string to ensure its effectiveness. Following the four respective categories of keywords, the final search string was as follows: (Education AND Reality) AND (Science OR Technology OR Engineering OR Mathematics) AND (“Systematic Review” OR “Meta-analysis”) AND (Virtual OR VR OR Augmented OR AR OR Extended OR XR OR Mixed OR MR).
We searched for relevant literature from databases recommended by the library expert. In order to be consistent, the exact search string was used across the following six electronic databases in the full text of literature using advanced search: IEEE Xplore, ACM Digital Library, Compendex, ERIC (i.e., Education Resources Information Center), Education Source, and Web of Science. The publication period was limited to 2020 to 2023 to ensure that only the most recent studies would be considered in this review. All six databases allow the addition of a date filter when applying the search string.

2.2. Inclusion and Exclusion Criteria

Inclusion and exclusion criteria were used to choose the relevant literature. The inclusion criteria were (1) systematic reviews or meta-analyses published between 2020 and 2023, (2) articles in the field of STEM education, (3) articles published as journal articles or conference papers, and (4) articles written in English. Due to their irrelevance to STEM education, the exclusion criteria were non-STEM disciplines and studies that were medicine or health-related in general (e.g., therapy, disorder, and disease).

2.3. Search Procedures

The six databases were searched sequentially on 20 October 2023; the results were downloaded in the .ris or .bib format and imported immediately into Zotero (Version 6.0.27)—an open-source citation management software for researchers—for manual screening [23,24,25]. During the search, the six databases returned 1972 records in total. Since ACM Digital Library only allows a maximum export of 1000 records, the search was performed individually for January 2020–December 2022 and 2023 (using the exact search string) to avoid data loss when exporting. Following the PRISMA framework [22], prior to initial screening, 125 duplicates were removed, one retracted record was removed, and 14 non-English records were removed. Then, abstracts and titles were screened in Zotero based on the inclusion and exclusion criteria; the relevant records were marked. Next, the full texts of relevant studies were downloaded into the box application, which permits collaboration and the secure sharing and storage of files in the cloud. The full texts were further manually evaluated based on the inclusion and exclusion criteria. Studies that were not entirely relevant were eliminated during the process. Finally, the process resulted in the 17 most relevant pieces of literature. Figure 1 presents a flowchart that summarizes our procedures for identifying the final sample based on the PRISMA framework [22].

2.4. Interrater Consensus

The analysis of selected articles for the review involved an interrater consensus between two actively engaged Ph.D. students to ensure reliability and validity. They harmonized their interpretations, analyses, and findings through weekly meetings, fostering a comprehensive understanding and refining the analytical approach. Minor disagreements were resolved during the process. Consensus for validation was pursued to establish interrater reliability, enhancing credibility in the research outcomes. Both researchers filled out a matrix, systematically comparing findings regarding extracted aim, field, research questions, target population, and overall findings, among other pertinent aspects, supplementing the robustness of the analysis. The matrix, devised in collaboration with their advisor professor, provided a structured framework for thorough evaluation and ensured alignment with the research objectives.

3. Results

The findings are organized into two main subsections. The first subsection focuses on systematic or meta-analytic reviews on augmented reality (AR), and the second subsection reports trends from systematic or meta-analytic reviews on virtual reality (VR). The studies in each subsection were summarized with our research question and aims in mind.
AR had the most publications in the selected articles (n = 11). For AR, there were 10 (90.91%) systematic reviews (SRs) compared to one (9.09%) meta-analysis. For VR, there were three (75%) SRs compared to one (25%) meta-analysis. For studies investigating both VR and AR, there was one (50%) SR and one (50%) meta-analysis. Regarding the publication venues of those works, journal articles (n = 14) outnumbered conference papers (n = 3). For AR, there were 10 (90.91%) journal articles compared to one (9.09%) conference paper. For VR, there were three (75%) journal articles compared to one (25%) conference paper. For studies investigating both VR and AR, there was one (50%) journal article and one (50%) conference paper. Only 16 selected articles had research questions ranging from one to eight, with three being the most common number (n = 5, 31.25%). All 17 selected articles listed the number of papers reviewed, ranging from 13 to 319, with 19 being the most common number (n = 3, 17.65%).
As shown in Figure 2, AR was implemented in general STEM and subsequently in science and engineering, specifically in chemistry, mathematics, and physics. VR was used in general STEM and subsequently in science and engineering, specifically in the computer science discipline. Studies investigating both VR and AR were within general science education.
Many articles did not specify the population of interest (n = 11). Six articles explicitly or implicitly specified their population of interest, spanning across various education levels: non-health-profession students from middle school to undergraduate [26]; higher education [27]; primary, secondary, and tertiary education [28]; all education levels [29]; K-16 [30], and K-12 [31]. Overall, AR and VR have been broadly applied to teaching and learning in STEM education.

3.1. Augmented Reality

3.1.1. STEM Disciplines

As suggested in articles concentrating on general STEM education, AR has been applied to many STEM fields and across educational stages. Sırakaya and Alsancak Sırakaya [32] found that physical sciences and life sciences were common disciplines that embraced AR, and marker-based AR was prevalent in K-12 (especially in primary and secondary schools). Sırakaya and Alsancak Sırakaya [32] also revealed that AR studies generally had large sample sizes (i.e., greater than 30), and the data were collected mainly using testing or surveying. Ajit et al. [33] indicated that subjects such as physics and mathematics are common disciplines for AR to be applied in STEM, where more than half of the studies were in the classrooms. Primary and secondary school students (K-12 in general) were popular samples, and tests and surveys were also popular methods to assess the outcomes [33]. On the descriptive side, Hidayat and Wardat [34] showed that AR was used across various STEM subjects (e.g., chemistry and physics), meaning that AR has been widely applied in the STEM field. However, Hidayat and Wardat [34] did not find many studies in subjects such as astronomy and technology.
AR is a versatile tool used in covering a variety of subjects or topics in articles on specific STEM disciplines. Yin et al. [30] and other researchers [28,31] suggested that multiple STEM fields or disciplines, like physics, introduced AR in their education. In answering their research questions about the areas of AR application in engineering and educational activities, Álvarez-Marín and Velázquez-Iturbide [35] found that technical drawing and electronics accounted for more than half of the papers reviewed, followed by construction, manufacturing, electromagnetism, assembling, and so on. The authors also stated that AR activities occurred in laboratories, lectures, and exercises [35]. Ahmad and Junaini [36] acknowledged that geometry was the most popular mathematics topic in the selected articles about AR in mathematics. For their research question about the topics used regarding AR in chemistry, Mazzuco et al. [37] highlighted diverse topics, including molecular structures, chemical reactions, chemical bonds, and more, because AR provided good learning visualizations. Regarding the physics topics covered using AR, Vidak et al. [29] stated that the most common ones were electrical circuits, astronomy, mechanics, and others. Table 1 displays the detailed subjects or concepts taught using AR in STEM education. Table 2 depicts studies investigating both AR and VR.

3.1.2. Study Goals

In the AR-in-general-STEM studies, the goals varied in terms of the depth of the technological side involved. Some studies reported general findings in terms of the characteristics of past studies along with advantages and challenges identified [32,33]. In contrast, one study focused on aspects of technical implementation, focusing on the types of technology and parameters [34], for instance. In the AR-in-specific-STEM-discipline studies, such as in chemistry, reviews focused on the topics delivered, the device types, the learning benefits, and the drawbacks [37]. In the case of physics, the studies focused on the descriptive, geographical, instructional, setting, participants, topical, and technological aspects [29].
Moreover, papers about AR in science education had varying focuses, depending on whether they were interested in the content of the publications or more on the metadata. For example, two reviews combined the educational and technological sides of AR applications, noting trends, theories or pedagogies, and technological features [30,31]. Two others focused more on the effectiveness of AR integration [28,38]. One asked questions about the descriptions on this topic, such as the publication growth and most cited articles [39].
Last, studies’ purposes varied based on their STEM educational fields. One study investigated the application of AR in engineering education, regarding its areas of application in the field, the educational activities, the evaluation, the characteristics, and the interactivity of AR applications in engineering [35]. Others researched how AR was used in mathematics, diving into general mathematics education [36] and mathematical creativity [40]. Table 1 depicts the AR studies.

3.1.3. Major Findings

With a notable trend of increasing AR in STEM publications worldwide [29,30,31,32,33,36,37,39], previous research examined various aspects of AR use and its effectiveness in STEM education, mainly from the educational and technological perspectives. For AR in general STEM education, studies highlighted the benefits on the educational or learning side [32,33]: examples included effective visualization and understanding, improved performance and outcomes, increased motivation and satisfaction, and better class engagement and interaction [32,33]. AR also offered good levels of usability [32,33]. Regarding the technological side, marker-based [33,34] and marker-less AR [34] were prevalent in terms of AR type, noting that (a) marker detection could be a challenge sometimes [32,33]; and (b) animation was a popular digital element [33,34].
For AR and its application in specific STEM fields and disciplines, besides AR’s positive impact on learning [28,31,35,36,37,38,40], past research in the science field (a) mentioned common study designs involving AR—experimental [30,31] and quantitative [31,39]—and (b) pointed out that instructional designers were advised to design AR-based activities with scaffolding in mind [30]. On the technological side, research in the engineering field echoed the use of animation while emphasizing characteristics such as monitor-based display and computing devices [35]. Papers in the mathematics field were about general mathematics [36] and specifically mathematical creativity [40], both representing AR’s practicality and potential to assist learners across different education levels to learn mathematics [36,40]. Further, studies about AR in specific STEM disciplines (e.g., physics) communicated AR’s flexibility, suggesting that AR could be used in classrooms or laboratories with the support of mobile devices [29,37] and software like Unity 3D [29].
In conclusion, the research in this section covered AR use from general STEM to specific STEM disciplines, revealing the educational and technological aspects of working practices while analyzing the advantages, challenges, and potential of AR integration into STEM education.
Table 1. Summary of studies reporting on augmented reality.
Table 1. Summary of studies reporting on augmented reality.
SourceScopeFieldSubjects/ConceptsTarget PopulationPurpose/AimTotal PapersResearch Questions (RQs) as They Originally AppearedOverall Findings
Sırakaya
and Alsancak Sırakaya [32]
SR of articles
until the end of 2018 (from 1980)
STEMPhysical sciences, Life sciences, Earth/Space Sciences, Mathematics, and EngineeringNot specifiedTo show the studies in STEM education that used AR42RQ1. What are the general characteristics of AR-STEM studies? RQ2. What are the advantages identified in AR-STEM studies? RQ3. What are the challenges identified in AR-STEM studies?1. AR-STEM studies have had a growing interest over the years, using quantitative methods in schools (many secondary schools). Marker-based AR was common. 2. AR’s advantages: benefits for the learners, a better learning experience with interaction, and more. 3. The major difficulties were problems with technology (e.g., detecting the marker) and teachers’ hesitance.
Ajit et al. [33]SR of papers published between 2012 and May 2020STEMPhysics (general), Physics (electrostatic, electromagnetism, and elastic collision), Mathematics, Science (electromagnetism), Chemistry (periodic table and molecules), Astronomy, and Natural sciencesNot specifiedTo discuss how AR is connected to STEM and how it benefits learning19RQ1. What are the general characteristics of AR in STEM education? RQ2. What are the benefits of AR in STEM study? RQ3. What are the challenges of AR in STEM study?1. Worldwide interest (primarily in secondary schools with varying sample sizes). Physics was a popular subject to use it. Assessment (e.g., test) was involved. Most studies used marker-based AR with handheld displays. Vuforia and Unity were common tools. 2. Benefits covered benefits to learners (e.g., better academic achievement), improved learning outcomes (e.g., visualizing abstract concepts), and more. 3. Challenges emerged from difficulty detecting the marker, system glitches, physical discomfort, and so on.
Hidayat and Wardat [34]SR of literature published between 2017 and 2022STEMAstronomy, STEM and STEAM, Science, Mathematics, Chemistry, Physics, Biology, Engineering and Architecture, STEM-based Mathematics and Science, and TechnologyNot specifiedTo see how AR helped STEM education42RQ1. What are the types of augmented reality used in STEM learning? RQ2. What are the types of technology employed in implementing STEM learning? RQ3. What are the types of augmented parameters in implementing STEM learning?1. Mainly marker-less (e.g., for chemistry) and marker-based (e.g., for engineering). 2. Camera-based AR was popular, followed by markers, object recognition, and more. 3. Animation and 3D models were the most used.
Irwanto et al. [39]SR of literature published between 2007 and 2022ScienceN/ANot specifiedTo investigate research trends of AR in science319RQ1. What is the annual scientific growth rate of publications on the topic of AR in science education between 2007 and 2022? RQ2. How is the distribution of the documents reviewed with regard to the number of authors in the 2007–2022 period? RQ3. Which countries contributed the most to the publications in the academic journals from 2007 to 2022? RQ4. Which are the most productive journals publishing articles on the applications of AR in science education between 2007 and 2022? RQ5. Which are the most cited articles related to the applications of AR in science education in the 2007–2022 period? RQ6. What were the most preferred research methods in articles on the applications of AR in science education from 2007 to 2022?1. Growth in publication. 2. Two or three authors were common in publications. 3. The United States is the country with the most publications. 4. The Journal of Chemical Education and Computers and Education were the two most productive. 5. The most cited paper was "Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning" by Dunleavy et al., as cited in Irwanto et al. [39]. 6. Quantitative designs have become increasingly popular.
Yin et al. [30]SR of articles published between 2001 and 2020SciencePhysics, Biology, Chemistry, and othersK-16To review of AR in K-16 science89RQ1. What patterns in research publication can be identified from the metadata? RQ2. What pedagogies and instructional affordances have been utilized in AR-supported science education? RQ3. What technological features of AR have been used in science education and how are they evolving over time?1. Over time, there have been many research publications, with many studies in Asia. AR has been most popular in subjects such as physics and biology. 2. Inquiry-based learning along with direct instruction. 3. AR could be location-based or image-based. Human–computer interaction was important in AR instruction. The common presentation mediums are 3D objects and 2D images.
Kalemkuş and Kalemkuş [38]Meta-Analysis of papers published between 2015 and 2020ScienceN/ANot specifiedTo examine the effect of AR in science on student academic achievement16RQ1. What is the general effect of the use of augmented reality applications in science education on the academic achievement of students?1. A moderate effect of using AR in science education on student achievement and positive aspects of encouraging the use of AR in science.
Álvarez-Marín and Velázquez-Iturbide [35]SR of literature published until September 2019EngineeringElectromagnetism, Assembling, Robotics, Topology Technical drawing, Electronics, Production, Nuclear reactor, Construction, and ManufacturingNot specifiedTo check the strengths and weakness of the current state of the art and provide recommendations for future research52RQ1. In which engineering studies has AR been applied? RQ2. In what types of educational activities in engineering education have AR apps been used? RQ3. How have AR apps been assessed in engineering education? RQ4. What are the main characteristics of the AR apps used in engineering education? RQ5. What is the degree of interactivity of the AR apps used in engineering education?1. More than half of the studies were about electronics or technical drawing. 2. Laboratory activities, lecture activities, and exercise class-related activities. 3. Assessments included perceptions of instructors and students (e.g., usefulness) along with academic performance. 4. Enabling technologies and functional characteristics: Smartphones were commonly used, and overlay perception was part of functional characteristics. 5. Limited interactivity. In only a few studies, there was some support.
Hidajat [40]SR of articles published between 2015 and 2023MathematicsN/ANot specifiedTo analyze the past research of AR for mathematical creativity66RQ1. What are the implications and features of using AR for mathematical creativity? RQ2. What is the potential of using AR for mathematical creativity based on the six themes from the NCTM Principles and Standards? RQ3. What are the perceptions of using AR for mathematical creativity, user criteria, measurement tools, and evaluation methods?1. Common good impacts on cognitive performance, knowledge gains, and aspects of self. Main features: head-mounted displays, Unity 3D, and and Vuforia. 2. AR was effective in distinct aspects of mathematical creativity. 3. AR could foster collaboration and was used widely in the college student population. Questionnaires and testing were examples of measurement tools. AR helped with skill development and mathematics learning.
Ahmad and
Junaini [36]
SR of literature published between 2015 and 2019MathematicsStatistics, Probability, Algebra, Geometry, Euclidean Vectors, and othersNot specifiedTo give a clear and comprehensive overview of AR in mathematics learning19RQ1. What are the most popular types of AR development tools used for learning mathematics? RQ2. What are the popular types of AR tools to develop mathematics applications? RQ3. What did the research say on the implementation, development and effectiveness of AR? RQ4. What are the significant benefits of AR in math learning? RQ5. What are the major problems of AR in math learning? RQ6. What are the research approaches used to study AR in math learning? RQ7. What are the methods used to evaluate the effectiveness of AR math tools? RQ8. What are the main topics in mathematics that implement AR for learning?1. Marker-based AR was the most popular, followed by marker-less AR. 2. Unity 3D and Vuforia SDK. 3. Well-demonstrated results. 4. Major benefits included boosted confidence, better visualization, and interactive learning. 5. Difficulties came from visualization and understanding. 6. Quantitative and quasi-experimental seemed common. 7. Pre-test-and-post-test was mostly embraced. 8. Topics covered geometry, algebra, and a few more.
Mazzuco et al. [37]SR of papers published between 2011 and 2020ChemistryKinetic molecular theory, Laboratory safety, Liquids, Nucleophilic addition, Molecular chirality, Molecular geometry, Material composition, Molecular structures, NMR spectroscopy, Molecular hybridization, Organic and inorganic compounds, Solvay and Leblanc process, Organic chemistry, Periodic table, PH and conductivity, Simple compounds, Stereochemistry, Submicroscopic representation, and TitrationNot specifiedTo inform of the use of AR in chemistry education using articles published from 2011 to 202049RQ1. In what topics of chemistry is augmented reality applied? RQ2. Which types of devices are used? RQ3. What are the reported advantages of using augmented reality in chemistry teaching? RQ4. What are the challenges of using augmented reality in chemistry teaching?1. Many distinct topics (e.g., molecular structures), as seen in this row. 2. Smartphones and tablets were mainly utilized. 3. The advantages came from learning domains and technological aspects, such as intellectual skill development and ease of use, respectively. 4. Challenges existed, such as cognitive overload and technical problems.
Vidak et al. [29]SR of literature published from January 2012 to 3 November 2020PhysicsPhysics: Electromagnetic waves, Ray optics, Electrostatics, Magnetism, Thermodynamics and heat, Atomic and molecular physics, Mechanics/mechanical waves, Astronomy, and Electrical circuitsAll physics learnersTo review how AR was used in physics education60RQ1. What instructional techniques and strategies were used for AR-based learning of physics? RQ2. Is the number of studies related to AR in teaching about physics increasing over time? RQ3. How are the articles on AR in teaching physics topics distributed geographically? RQ4. What types of participants and how many of them were included in earlier AR physics education research? RQ5. In what learning environments was the AR physics education research situated? RQ6. What are the most popular software development and hardware technologies for AR-based teaching about physics? RQ7. What physics topics were covered through earlier AR physics education articles?1. The discovery strategy and inquiry-based instructional technique were widely embraced. 2 and 3. The popularity of AR in physics education has expanded across the globe. 4. Many studies were conducted on the K-12 level with fewer than 100 participants. 5. AR was mainly used in the classroom. 6. Unity 3D and Vuforia (for software), and mobile devices (for hardware). 7. Various topics such as electrical circuits and mechanics.
Table 2. Summary of studies reporting on both virtual reality and augmented reality.
Table 2. Summary of studies reporting on both virtual reality and augmented reality.
SourceScopeFieldSubjects/ConceptsTarget PopulationPurpose/AimTotal PapersResearch Questions (RQs) as Originally AppearedOverall Findings
Tsichouridis et al. [28]Meta-analysis of literature between 1998 and 2019 (primarily focused on the past ten years before this paper was published)ScienceZoology, Physics, Programming, Heat conduction, Science, Electricity, Natural sciences, Chemistry, STEM, and Distance perceptionPrimary to tertiary educationTo see the implementation and effectiveness of AR and VR in science education19N/AVR and AR positively impacted learning: they mainly kept high motivation, improved learning outcomes, and helped low-performing learners in primary education; they helped with increased participation and addressed misconceptions in lower-secondary education; students in upper-secondary education showed boosted motivation and enhanced practical skills; in tertiary education, they provided interactive practical experiences, beneficial for physics and space-related concepts. AR and VR created engaging environments that benefitted learning.
Zhang and Wang [31]SR of papers between 2002 and 2021ScienceAstronomy, Biology, Chemistry, Environmental science, Physiology, STEM, Science, Geography, Medical science, and PhysicsK-12To provide insights into AR/VR theories and practices in K-12 science61RQ1. What were the research trends? RQ2. What theories were grounded upon or adopted? RQ3. What types of learning activities have been conducted? RQ4. What research designs were used? RQ5. What types of VR/AR technologies were employed? RQ6. What kind of science education content was involved? RQ7. What were the learning outcomes?1 and 2. Increased publications since 2019, focusing on K-12 physics and biology. >30% studies used constructivism. 3. The emphasis was on inquiry-based learning. 4 and 5. Quantitative and experimental study designs and technologies, such as marker-based AR and tablet PCs, were used. 6 and 7. Many studies aimed at enhancing scientific understanding and investigating cognitive and affective goals. Multiple suggestions for future studies, such as using longer and repeated study designs for robust evaluations.

3.2. Virtual Reality

3.2.1. STEM Disciplines

Several papers have explored the application of VR and immersive VR (IVR) in STEM, each aiming to address specific aspects of VR integration and its impact on education. VR was applied to the general STEM domain [26], specific science subjects including physics, chemistry, and more [27], engineering subjects such as civil and mechanical engineering [41], and many topics about computer science (e.g., security concepts) [42]. Literature has demonstrated that VR can be a flexible tool for improving educational outcomes across different STEM fields [28,31]. Refer to Table 3 for studies on VR technology and Table 2 for studies combining VR with AR in STEM education.

3.2.2. Study Goals

The VR studies had different and meaningful research goals. While some focused on the integration of VR and AR lab environments across education levels, from primary to tertiary education [28], others specifically examined VR and AR in K-12 science education, showing the growing interest in this area [31]. Furthermore, while some studies explored the impact of VR on general learning outcomes in STEM [26] and science education [27], others assessed its effectiveness in fields like engineering [41] and computer science [42].
In addition, some other literature delved into psychological constructs, such as presence [26] and motivation [41], and investigated aspects like active learning techniques [26] and evaluation methodologies [41]. Thus, VR’s effectiveness in education is shaped by both educational content and psychological factors, stressing the need for a thorough approach incorporating active learning techniques and diverse evaluation methods to understand its impact fully.

3.2.3. Major Findings

Literature on VR in STEM education demonstrated that desktop VR often outperformed head-mounted displays, particularly when combined with active learning techniques, resulting in more impactful STEM education outcomes [26]. Effective VR integration would require instructional designs managing cognitive load and supporting select–organize–integrate (SOI) processes tailored to students’ prior knowledge to enhance motivation and engagement [27]. In engineering education, VR’s potential was evident, though challenges like high processing demands and limited immersion would need to be addressed, underscoring the importance of standardized evaluation models for assessing VR’s impact [41]. Specifically in computer science, VR could benefit both interaction and concept teaching, but issues such as cybersickness and the need for customized engagement strategies highlighted the importance of adapting VR to diverse learner needs [42].
The integration of VR and AR across educational levels showed notable benefits: enhancing motivation and learning outcomes in primary education, increasing excitement and understanding in lower secondary education, and supporting practical skills and immersive experiences in upper secondary and tertiary education [28]. Developing realistic and practical VR and AR labs that could bridge virtual and natural environments would be necessary [28]. However, many studies lacked theoretical frameworks [31]. Challenges remained, such as technical issues and methodological limitations, which could lead to inconsistent applications of VR and AR, reducing their potential educational impact [31]. Addressing these challenges would require integrating educational theories, designing effective learning activities, and using comprehensive evaluation methods [28,31].
From these findings, it can be claimed that VR’s success in education depends on careful integration, thoughtful instructional design, and consistent evaluation. Selecting the right VR technology and designing instruction to manage cognitive load while aligning with students’ prior knowledge would be vital for enhancing learning outcomes [26,27]. Additionally, addressing practical challenges and using standardized evaluation models should be essential for ensuring VR’s effectiveness across different educational contexts [41,42].
Table 3. Summary of studies reporting on virtual reality.
Table 3. Summary of studies reporting on virtual reality.
SourceScopeFieldSubjects/ConceptsTarget PopulationPurpose/AimTotal PapersResearch Questions (RQs) as They Originally AppearedOverall Findings
Cromley et al. [26]Meta-analysis of papers published between 2000 and 2020STEMN/AMiddle schoolers to undergraduates (excluding health fields)To foster informed decisions for using VR in STEM18RQ1. Does the research base on learning with virtual reality support this level of use? RQ2. Does VR actually help learning, and if so, for whom and under what conditions? RQ3. To what extent have theory-driven constructs such as presence been tested in studies of VR efficacy for learning? RQ4. Does active learning help learners get more from VR?VR positively impacted learning (g = 0.33). Moderator effects were noted in VR redesign, place settings, learning in science, desktop displays, and across learning outcomes. Desktop VR had more effects than head-mounted displays. Good effects for factual, conceptual, and transfer learning.
Lui et al. [27]SR of papers published between 1 January 2013 and 13 May 2022ScienceMarine ecology, Human anatomy, Medical science, Life science, Geoscience, Biochemistry, Physics, Biology, (Organic) Chemistry, Astronomy, and Environmental scienceHigher educationTo explore learning theories and affordances of immersive VR (IVR) in science29RQ1. How should IVR lessons in higher education be designed to optimize students’ learning outcomes?“Agency” was a prevalent design in IVR. IVR impacts were mixed for students with low prior knowledge. Effective IVR would minimize cognitive loads, improve the process of Select–Organize–Integrate, and use generative learning strategies for active learning.
di Lanzo et al. [41]SR of literature published from 2015 to 2019EngineeringPneumatics, Civil, Electrical, Software, Industrial, and Mechanical engineeringNot specifiedTo understand the use of VR in engineering education17Guiding RQ1. What are the ways in which virtual reality is currently being used within the context of engineering education?Three-dimensional virtual classrooms were effective for teaching complex industrial processes but faced challenges, such as high processing needs and immersion deficiency. VR in engineering lacked a comprehensive assessment model despite showing promise in good learning outcomes.
Pirker et al. [42]SR of literature published after 2013Computer science (CS)Object-oriented programming, Coding, Computational thinking, Innovation and invention and skills, Project-oriented working, System development, Security concepts, and Theoretical CSNot specifiedTo investigate VR in computer science education13RQ1. How relevant is the topic learning and teaching computer science topics with VR in relation to the numbers of research publication? RQ2. What are the learning scenarios and reported learning objectives regarding computer science education? RQ3. What are the reported advantages of using VR for reaching the learning objectives? RQ4. What technologies were used for the VR experiences? RQ5. What forms of locomotion and interaction with the environment are implemented within the VRs? RQ6. What are the distinguished target groups and which engagement strategies were chosen regarding the respective target group? RQ7. What issues and problems were reported within the studies?1. Stable publications on the topic in recent years. 2. Various CS topics taught covering all-cognitive-level learning goals. 3. Notable advantages such as helpful interaction, interesting design, and so on. 4. Desktop VR was popular. 5. Comfort needed to be essential in locomotion methods. 6. Various engagement strategies based on the specific context. 7. Challenges included interface interaction, user acceptance, and cybersickness.

4. Discussion

Our research question aimed to identify the research trends in using extended reality (XR) technology for teaching and learning in STEM. In this section, we will approach our research question through two lenses we deem necessary: (a) similarities and differences among the studies and (b) advantages and limitations of the types of XR in the studies. We will also address our research question by summarizing the overall trends. Table 4 summarizes the lenses in this section.

4.1. Similarities and Differences Among Studies

In three studies on AR in STEM, the authors did not limit their search to a certain education level, so their findings were generalizable to multiple student populations. Regarding inclusion, Sırakaya and Alsancak Sırakaya [32] focused on experimental studies, whereas Ajit et al. [33] and Hidayat and Wardat [34] did not. Although all three papers reported the broad application of AR in STEM, their focuses were not the same. Both Sırakaya and Alsancak Sırakaya [32] and Ajit et al. [33] leaned toward the education side of AR (i.e., what are the outcomes of using AR in STEM). However, Hidayat and Wardat [34] focused on the technological side of AR (i.e., what aspects are essential in implementing AR in STEM). From the technology implementation perspective, Hidayat and Wardat [34] suggested that maker-related technology was popular. From the education application perspective, the major findings from Sırakaya and Alsancak Sırakaya [32] and Ajit et al. [33] were similar: in general, marker-based AR was common in STEM; the upsides of using AR in STEM include increased motivation, satisfaction, and more; the downsides of using AR in STEM include physical and technical difficulties.
The papers about educational research and the application of AR in science offered distinct perspectives. Irwanto et al. [39] followed Cooper’s guidelines and focused on the general characteristics of the trend of AR in science, providing insights into the outlook of the field (e.g., publication trend) spanning over 15 years. Kalemkuş and Kalemkuş [38] investigated a 5-year range of experimental research in the field that evaluated student achievement, suggesting the positive effects of effectively using AR in science. Yin et al. [30] conducted a systematic review of AR practices in K-16 science education over 20 years, revealing aspects such as instructional design and technological features, shedding light on working practices.
Both articles about using AR in mathematics education asked more questions about the education side than the AR technology itself. Ahmad and Junaini [36] focused on general mathematics learning, whereas Hidajat [40] concentrated on mathematical creativity. Ahmad and Junaini [36] reviewed aspects such as educational benefits and issues, and Hidajat [40] revealed angles such as educational implications and potential. Besides the general upsides of using AR in mathematics (e.g., improved academic performance), both reviews pointed out that Unity 3D and Vuforia SDK were the most popular AR tools in learning mathematics. In addition, testing was a common method for evaluating the effectiveness of AR in mathematics. Both articles had no restrictions on the education level, so their findings seemed to be generalizable.
There has been an increasing trend in publications regarding AR application in specific science subjects (e.g., physics and chemistry), which could be used to visualize complex concepts to help students understand them better and also use it in laboratories beyond classrooms [29,37]. Hence, AR has great potential for future development in science subjects. No papers related to AR in technology education were found, and only one paper about AR in engineering education was found, which was from Álvarez-Marín and Velázquez-Iturbide [35]. Future research may address the lack of publications regarding AR in technology and engineering education. There may be a need to reflect on why AR has not been used much in such disciplines.
In this review, physics, chemistry, and mathematics were the most popular disciplines for AR in education applications. There were common topics taught in each discipline. In physics, the topics taught usually lay within areas such as electromagnetism and mechanics [29,33]. In engineering, the topics taught were more hands-on, such as construction [35]; however, engineering topics may overlap with physics, such as electromagnetism [33,35]. In chemistry, the topics taught were fundamental, such as the periodic table and molecular chemistry [33,37]. In mathematics, the topics covered were abstract and conceptual [36]. As a versatile instrument, AR could help cover abstract or hands-on topics, depending on the discipline. In STEM teaching and learning, implementing AR has been supported by existing tools. In terms of technology type, both marker-based AR [29,32,33,34,36] and marker-less AR [34,36] were popular. Software tools used were generally Unity 3D and Vuforia [29,33,36,40]. Many deployed quantitative designs or methods in their research studies [32,36,39]. There may be more tools and designs to be developed for future applications, but as of now, as mentioned here, there are prevalent technology types and tools to be used to help with successful AR implementation in STEM.
Cromley et al. [26], Pirker et al. [42], and Lui et al. [27] suggested that VR may be used to promote active learning and increase participation in STEM education at all education levels. However, these studies also emphasized the importance of defining technology specifications and instructional design to foster effective student engagement. Furthermore, Cromley et al. [26], Pirker et al. [42], Lui et al. [27], and di Lanzo et al. [41] discussed the development and enhancement of problem-solving, practical and technical abilities, and critical thinking skills, bridging the gap between theoretical knowledge and practice. VR has been seen as a relatively effective tool in STEM educational settings.
In addition, Cromley et al. [26], Pirker et al. [42], and Lui et al. [27] recognized the benefits of VR in improving STEM education. Each study emphasized VR’s versatility across various educational settings, its capacity to promote active learning, and a profound grasp of STEM subjects. They also stressed the importance of technology specifications, instructional design, and learner engagement when incorporating VR into educational environments. Pirker et al. [42] and Lui et al. [27] shared a common interest in integrating immersive technologies into educational settings, although they differed in their specific educational focuses. Pirker et al. [42] focused on computer science education, considering secondary to tertiary education levels. In contrast, Lui et al. [27] narrowed their study to higher education, specifically within science education. Although both studies acknowledged and explored the technological and motivational aspects of immersive experiences, Pirker et al. [42] examined trends, advantages, and challenges in immersive VR for computer science education, while Lui et al. [27] focused on crafting effective immersive VR lessons in higher education science, highlighting nuanced design principles and impacts on students with diverse prior knowledge levels from primary to tertiary education. Despite their shared emphasis on tailored engagement strategies, the application of this concept differed in both articles based on particular educational contexts.
The methodological rigor and consistency differed among the VR studies. While Cromley et al. [26] and Pirker et al. [42] offered straightforward, well-defined research questions, di Lanzo et al. [41] needed more specificity in their target audience and evaluation methods. Such methodological variations can hinder their findings’ comparability and general applicability. A uniform evaluation approach could facilitate more evident conclusions and better-informed decisions regarding VR’s integration into STEM education.
Differences exist regarding technological and methodological aspects. In terms of technological emphasis, Cromley et al. [26] highlighted that desktop VR had a more significant impact than head-mounted display VR, whereas Pirker et al. [42] and Lui et al. [27] did not delve deeply into these technological nuances. The structured and less immersive nature of desktop VR allows for better focus on and comprehension of complex topics, facilitating better learning outcomes through more straightforward and less immersive interactions that promote active engagement and effective information processing, avoiding potential distractions, cognitive overload, and possible discomfort that can occur with head-mounted-display VR [43,44,45].
The differences in VR studies suggested differing research priorities. Cromley et al. [26] stressed the significance of technology selection for educational VR applications. Conversely, Pirker et al. [42] and Lui et al. [27] prioritized instructional design and learning outcomes, indicating that technology may not be the sole factor determining VR’s educational effectiveness. Moreover, the studies differed in their focus on methodological rigor and clarity. Pirker et al. [42] stressed VR’s adaptability across various educational settings, while Lui et al. [27] emphasized instructional designs that enhance cognitive processes and learning outcomes. These insights indicated VR’s potential to promote active learning and deepen understanding in STEM. However, Lui et al. [27] also noted variability in student motivation and engagement based on prior knowledge, suggesting a need for tailored learning strategies and instructional designs. By highlighting the importance of considering individual differences in engagement and learning, it has been emphasized how engagement can vary significantly among students based on various factors, including prior knowledge, beyond the use of new technologies [46]. That is, it has been highlighted how learners’ characteristics affect learning outcomes and support the need for considering these characteristics in instructional design, stressing the importance of tailored learning strategies [47], as discussed in the study by Lui et al. [27].
Despite the differences described above, the VR studies all recognized potential challenges related to VR’s integration into education. While di Lanzo et al. [41] mentioned technological issues like high processing power demands and limited immersion, Cromley et al. [26] and Pirker et al. [42] stressed the importance of tailored learning strategies and adaptive instructional designs. A shared concern across these studies was the need for a standardized evaluation model, which may hamper the comparability and clarity of their findings [26,41].
In general, from the teaching side, both AR and VR allow the adoption of various teaching methods (e.g., virtual laboratories, adaptive feedback mechanisms, and assessment support), and they promote active learning, critical thinking, and problem-solving skills from the learning side [28,31]. For instance, Zhang and Wang [31] cited studies that delve into the discussion of the integration of AR into problem-based learning, highlighting its benefits on learning achievement [48]. Furthermore, other studies have discussed the role of visual aids in education, with technologies such as VR and AR, which can significantly support the understanding and retention of abstract concepts concisely [49].
Tsichouridis et al. [28] aimed to assess the effectiveness of AR and VR across various education levels, but Zhang and Wang [31] concentrated on understanding the trends and practices in K-12 science education. Hence, both investigated the potential of VR and AR in science education, but their approaches and findings offered distinct perspectives on the topic. The differences in research questions influenced their choice of methods. Tsichouridis et al. [28] employed a meta-analysis of 19 studies. Notably, the inclusion–exclusion criteria did not eliminate potential biases in their review. Moreover, the methodology section needed more details on the analytical method and reliability metrics, which could have affected the validity and replicability of their findings. In contrast, Zhang and Wang [31] conducted an SR of 61 papers to explore trends in K-12 science education; the criteria for selecting the 61 papers analyzed were not explicitly stated. Additionally, Zhang and Wang [31] did not involve a conceptual framework in their review, which could have impacted the interpretation of the findings. Further, their target populations differed significantly. Tsichouridis et al. [28] considered a wide range of education levels, while Zhang and Wang [31] focused exclusively on K-12 education. This distinction may influence the applicability of their findings to different educational settings.
Regarding future research, Tsichouridis et al. [28] suggested further optimization of VR and AR laboratories. In contrast, Zhang and Wang [31] recommended that teachers be more knowledgeable about the underlying theories and carefully design their lessons. While both recognized the potential benefits, Tsichouridis et al. [28] along with Zhang and Wang [31] acknowledged challenges that need to be addressed for the effective implementation of VR and AR in STEM education.

4.2. Advantages and Limitations of AR and VR

In the studies, AR shows many advantages reflecting its effectiveness: boosted confidence and motivation [36], better interaction in learning [32], and ease of use [37], to list a few. However, AR is not yet a perfect choice due to challenges such as physical discomfort [33] and limited support in interactivity [35]. When considering implementing AR, it is essential to think deeply about the perspectives from the education side (e.g., pedagogies and place for implementation) and the technical side (e.g., supporting technologies and potential challenges).
For example, to specify field-specific (e.g., engineering) or discipline-specific (e.g., physics) application scenarios of AR, various sample papers from the list of the papers reviewed in the current paper have been included here as supplemental resources. In science, AR could help university students develop laboratory skills in science laboratories [50] and improve children’s knowledge of natural science through AR-based puzzle games [51]. Regarding engineering, augmented remote laboratories received positive assessments from teachers and students [52], and AR also ignited engineering students’ learning interest in mechanical drawing [53]. Within mathematics, students had a positive attitude toward implementing AR in classrooms [54]. While AR has been an effective tool in mathematics education, it has faced technical difficulties [55]. When it comes to chemistry, using AR in learning colorimetric titration facilitated learning and reduced the handling risks of chemicals [56]; students enjoyed the interactive environment provided by AR to learn about safety in biochemistry laboratories [57]. In terms of physics, AR could decrease extraneous cognitive processing in physics laboratories [58] and assist in visualizing abstract ideas [59]. AR-embedded problem-based learning activities enhanced physics learning achievement but might generate physical discomfort [48]. The advantages of using AR in various STEM fields or disciplines seem to outweigh the disadvantages.
Findings from the VR studies suggest that it can improve learners’ conceptual understanding, encourage inquiry-based learning, and facilitate the knowledge of scientific principles and concepts by providing students with visual representations to analyze and manipulate. For instance, Cromley et al. [26] detailed how VR can give three-dimensional visual representations, making abstract scientific concepts or principles more tangible and accessible. Moreover, VR allows students to engage in hands-on and interactive experiences, improving their learning, critical analysis, and understanding processes [27]. While Cromley et al. [26] highlighted a positive overall effect of VR on learning outcomes, it is important to interpret these findings cautiously, as effect sizes alone, as uniquely addressed, may not capture the complexity of learning experiences within VR environments. For instance, factors such as learner motivation, individual differences in cognitive processing, and the contextual relevance of VR content may also influence learning outcomes but need to be thoroughly explored in their study. In short, the literature has revealed the improved understanding and retention of scientific concepts through VR, aligning with the technological advantages discussed for this technology in educational settings [60]. Additionally, the literature has delved into how immersive VR environments can facilitate the acquisition of declarative knowledge (i.e., facts and information) that a learner needs to remember and understand [61], as discussed in the study by Pirker et al. [42].
Findings from the studies focusing on VR and AR simultaneously indicate that VR and AR can guide individual learning pathways, supporting the identification of scientific concepts and helping to overcome misconceptions [28,31]. Zhang and Wang [31] emphasized the increasing use of VR and AR in K-12 science education, and Tsichouridis et al. [28] highlighted the positive impact across various education levels, from primary to tertiary education. Tsichouridis et al. [28] noted the effectiveness of VR and AR technologies in addressing misconceptions, enhancing the understanding of scientific knowledge, and boosting student attention by providing students with virtual laboratories to mimic real-world experiments, facilitating learning processes. The literature has shown that VR environments are practical for enhancing students’ comprehension of complex scientific concepts, likely due to the immersive and interactive nature of VR, which allows students to visualize and manipulate abstract concepts in a 3D space [62]. Also, it has been evidenced that AR enhances the learning process by making difficult experiments more accessible and understandable, thus improving students’ grasp of the underlying scientific concepts [63]. Hence, AR and VR have been revealed to increase engagement and retention of scientific knowledge when used [64].
Although both studies about VR and AR found that VR and AR can enhance learning experiences, both studies also identified challenges associated with VR and AR applications. Tsichouridis et al. [28] noted technological glitches and pedagogical problems. In contrast, Zhang and Wang [31] highlighted the importance of teachers understanding the underlying theories and designing practical lessons. Zhang and Wang [31] cited studies focusing on the importance of developing practical lessons, demonstrating the need for instructors to integrate practical and engaging methods into their teaching to facilitate learning [65].
Table 4. Summary of the Discussion’s Lenses Among Studies.
Table 4. Summary of the Discussion’s Lenses Among Studies.
AspectCategorySummary
Similarities and DifferencesSimilaritiesAll the selected STEM-related studies focused on one or more types of XR technology (specifically, AR and VR) and had explicit study goals/purposes/aims/research questions to guide their research. Many studies (a) revealed a growing research interest in the topic, (b) provided examples of STEM disciplines or topics to be covered by their corresponding type of XR technology, (c) appeared to have reliable or generalizable findings within their scope, and (d) gave a sense of the effectiveness of using XR in STEM education primarily in terms of advantages and challenges.
DifferencesThe selected STEM-related studies mainly differed by their research goals/purposes/aims/research questions, making some seem more applied. This fundamental difference likely led to different (a) choices of the type of study conducted (i.e., SR or meta-analysis; quantitative or qualitative in analysis) and (b) year ranges of interest in choosing the studies to be examined.
Advantages of the TechnologyARThe main advantages of using AR in STEM include (a) general adaptability and versatility in the actual use, (b) sufficient support by existing tools [29,33,36,40], and (c) benefits to learners such as boosted confidence in learners [36] and helpful learning interactions [32].
VRThe main advantages of using VR in STEM include (a) general adaptability and versatility in the actual use, (b) the positive impact on learning outcomes [26], and (c) specific benefits to learners such as allowing effective interactions and improving their understanding of hard concepts through good visualizations [42].
OverallUsing XR in STEM can (a) make learning experiences better at different education levels, (b) facilitate the understanding of concepts taught, and (c) improve learners’ skills such as critical thinking and problem-solving [28,31].
Limitations of the TechnologyARThis technology may (a) have issues with marker detection [32] and (b) cause some physical discomfort [33].
VRThis technology may (a) cause some cybersickness [42] and (b) have challenging interactions with interfaces [42].
OverallDue to complexity or unfamiliarity, XR may lead to (a) learning difficulties and (b) ineffective learning results [31].

4.3. Overall Trends

There is an increasing publication trend and growing research interest in using XR in STEM education. This review found that AR and VR are the primary types of XR used in this field. On the one hand, AR is flexible and has been applied to many STEM disciplines, including physics, mathematics, and chemistry. It has a positive impact on enhancing learning experiences and facilitating the understanding of concepts. On the other hand, VR improves problem-solving and collaborative learning abilities in STEM fields (e.g., physics and computer science) by providing immersive and interactive environments. VR aids n skill acquisition and practical training by offering hands-on experiences that mimic real-world situations or phenomena. However, neither technology is without flaws, as they may cause physical discomfort like cybersickness for learners. The selected studies are comprehensive, with some focusing on theoretical aspects (e.g., learning theories) and others on practical aspects (e.g., implementation strategies). Nevertheless, all the studies reflect the effectiveness of their respective XR technology within their research scope. Generally, XR has been utilized across STEM disciplines and different education levels, highlighting its interdisciplinarity, versatility, and adaptability in addressing varying educational needs. XR allows students to bridge the gap between theoretical knowledge and practical application by fostering hands-on practice, exploring and manipulating virtual objects, and conducting experiments. In STEM education, the integration of XR (particularly AR and VR) appears to be highly effective.
Literature on XR in STEM education reveals emerging themes that discuss both the advantages and challenges associated with its integration. A prominent theme is the “enhancement of student engagement and motivation” through AR and VR technologies in STEM. For instance, some studies show that AR applications can significantly increase student interest in STEM by providing interactive content [32,33]. Similarly, other studies found that VR environments foster a strong sense of presence and immersion in STEM learning, which enhances engagement with the material [26,27,28]. These aspects illustrate how XR technologies can effectively capture students’ attention and encourage active participation in their learning processes.
Another important rising trend is the “facilitation of conceptual understanding through good visualizations” of XR in STEM. For AR, some researchers have highlighted how it can make abstract and complex STEM topics more accessible through effective visualizations, such as molecular structures [37] and geometric objects [36], bridging the gap between knowledge and application. For VR, some researchers have emphasized that VR simulations promote teamwork and communication among students to address real-world problems in computer science [42]. These perspectives indicate how XR can transform learning experiences by making difficult concepts more understandable and engaging.
The emerging theme of “skill acquisition” is crucial regarding XR in STEM. Articles have stated that AR can help learners develop general intellectual skills [37] and, further, problem-solving skills [40]. Studies have demonstrated that immersive VR experiences can replicate real-world scenarios, providing students with hands-on training essential for their future careers [26,42]. This practical application of knowledge allows students to practice skills in a controlled environment, preparing them for real-world challenges [27,41]. The capacity to embrace experiential learning through XR is a significant advantage in STEM education, where various skills are needed.
The literature also points out the “challenges associated with XR integration in STEM”. Different authors have underlined issues including physical discomfort [33,42], technical glitches [28,33], and the need for effective design and implementation strategies [30,31]. This theme underscores the importance of being aware of XR’s limitations and developing corresponding strategies to address such challenges. Basically, the literature specifies XR’s impact and potential in STEM education, acknowledging both the benefits and the difficulties.

5. Conclusions, Limitations, Implications, and Recommendations

Overall, XR can help students grasp ideas and engage with their learning processes through immersion and interaction within virtual environments. One important finding here is the increasing trend in publications focusing on applying AR and VR in general STEM education and specific subjects, improving STEM education by promoting active learning and enhancing student engagement. The common fields that would benefit from XR are mathematics, chemistry, physics, biology, and computer science. Applying XR in STEM relies on going beyond in-classroom strategies to analyze and deepen the understanding of abstract scientific concepts and principles. By allowing students to interact in virtual environments, XR can facilitate hands-on learning experiences that build comprehension and practical skills. However, to support student learning to the maximum extent, instructors need to plan and design XR experiences carefully. This task is crucial to aligning with learning objectives and enduring outcomes and supporting comprehensive forms of learning when using XR in STEM education.
We recognize a few limitations to our study. First, we used six databases, but more online databases may be used to search for relevant literature regarding using XR in STEM education. Also, changes in the search string may potentially result in a different sample. Third, the findings about XR from this study may need more generalizability to non-STEM fields. Apart from the limitations, we believe that, by providing insights into the current research trends of applying XR technology (particularly AR and VR) to STEM education, this study has one key implication: the importance of informing educators to develop effective and accessible practices of using XR in STEM education based on the the existing information, opportunities, and challenges presented. Moreover, this umbrella review can stimulate reflection on existing practices and the development of standardized frameworks and metrics to evaluate the educational impact of XR on academic outcomes, helping educators design more impactful educational interventions involving emerging technologies. For future research about XR in STEM education or general education, a few questions may be worth considering: How can we develop a comprehensive framework to guide the implementation of XR in education? How do we build one or more metrics to universally measure the effectiveness of using XR in education on the learning/teaching side? Answering these questions will make applying XR to STEM education or general education less complicated and make it easier to evaluate the outcomes and compare studies of the same or similar nature under the topic.
We have the following recommendations for using XR in STEM education. First, instructors should familiarize themselves with relevant educational theories, carefully design learning activities, and evaluate the impact of XR in their education settings before utilizing them to have most of them support teaching and learning processes. Second, in practice, the implementation of XR is context-dependent. Specifically, AR offers good visualizations to help students contextualize concepts and be practical, and VR provides more immersive learning environments to explore abstract or complex ideas. By choosing the best-fitting technology and addressing challenges such as technical glitches and limitations in research methodologies, instructors can maximize the benefits of XR across various education levels in STEM. Generally speaking, XR can potentially transform STEM education by providing students with interactive and engaging learning experiences at different education levels. Educators are encouraged to integrate XR technologies such as AR and VR into teaching practices, acknowledging their versatility and capacity to create immersive learning experiences to enhance learning, catering to diverse learning styles, promoting inquiry-based learning, and improving students’ motivation/interest and engagement in STEM subjects.

Author Contributions

Conceptualization, Y.Z. and A.J.M.; methodology, Y.Z. and A.J.M.; validation, Y.Z. and M.A.F.-G.; formal analysis, Y.Z. and M.A.F.-G.; investigation, A.J.M., B.B. and V.P.; resources, A.J.M., B.B. and V.P.; data curation, Y.Z.; writing—original draft preparation, Y.Z., M.A.F.-G., A.J.M. and V.P.; writing—review and editing, Y.G., A.J.M., V.P. and B.B.; supervision, A.J.M.; project administration, V.P.; funding acquisition, A.J.M., B.B. and V.P. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the U.S. National Science Foundation under awards No. 2309564 and 2417510. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect those of the National Science Foundation.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data derived from public domain resources.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Stanney, K.M.; Nye, H.; Haddad, S.; Hale, K.S.; Padron, C.K.; Cohn, J.V. Extended reality (XR) environments. In Handbook of Human Factors and Ergonomics; Wiley Online Library: Hoboken, NJ, USA, 2021; pp. 782–815. [Google Scholar]
  2. Alnagrat, A.; Ismail, R.C.; Idrus, S.Z.S.; Alfaqi, R.M.A. A review of extended reality (XR) technologies in the future of human education: Current trend and future opportunity. J. Hum. Centered Technol. 2022, 1, 81–96. [Google Scholar] [CrossRef]
  3. Doolani, S.; Wessels, C.; Kanal, V.; Sevastopoulos, C.; Jaiswal, A.; Nambiappan, H.; Makedon, F. A review of extended reality (XR) technologies for manufacturing training. Technologies 2020, 8, 77. [Google Scholar] [CrossRef]
  4. Zaman, A.; Abir, M.R.; Mursalin, S. Extended reality in education and training: Enhancing trustworthiness. Int. J. Sci. Res. Arch. 2024, 11, 1705–1720. [Google Scholar] [CrossRef]
  5. Rauschnabel, P.A.; Felix, R.; Hinsch, C.; Shahab, H.; Alt, F. What is XR? Towards a framework for augmented and virtual reality. Comput. Hum. Behav. 2022, 133, 107289. [Google Scholar] [CrossRef]
  6. Aguayo, C.; Videla-Reyes, R.; Veloz, T. Entangled cognition in immersive learning experience. Adapt. Behav. 2023, 31, 497–515. [Google Scholar] [CrossRef]
  7. Wang, P.; Wu, P.; Wang, J.; Chi, H.L.; Wang, X. A critical review of the use of virtual reality in construction engineering education and training. Int. J. Environ. Res. Public Health 2018, 15, 1204. [Google Scholar] [CrossRef]
  8. Salah, B.; Abidi, M.H.; Mian, S.H.; Krid, M.; Alkhalefah, H.; Abdo, A. Virtual reality-based engineering education to enhance manufacturing sustainability in industry 4.0. Sustainability 2019, 11, 1477. [Google Scholar] [CrossRef]
  9. Logeswaran, A.; Munsch, C.; Chong, Y.J.; Ralph, N.; McCrossnan, J. The role of extended reality technology in healthcare education: Towards a learner-centred approach. Future Healthc. J. 2021, 8, e79–e84. [Google Scholar] [CrossRef]
  10. Takrouri, K.; Causton, E.; Simpson, B. AR technologies in engineering education: Applications, potential, and limitations. Digital 2022, 2, 171–190. [Google Scholar] [CrossRef]
  11. Khlaif, Z.N.; Mousa, A.; Sanmugam, M. Immersive extended reality (XR) technology in engineering education: Opportunities and challenges. Technol. Knowl. Learn. 2024, 29, 803–826. [Google Scholar] [CrossRef]
  12. Yu, D.; Jiang, W.; Wang, C.; Dingler, T.; Velloso, E.; Goncalves, J. ShadowDancXR: Body gesture digitization for low-cost extended reality (XR) headsets. In Proceedings of the Companion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces, Virtual, 8–11 November 2020; pp. 79–80.
  13. Kini, V.G.; Ganeshrao, S.B.; Siddalingaswamy, P. XR review: A comprehensive analysis of visual function testing and gamification in extended reality environments. IEEE Access 2024, 12, 79714–79730. [Google Scholar] [CrossRef]
  14. Wang, J.; Shi, R.; Zheng, W.; Xie, W.; Kao, D.; Liang, H.N. Effect of frame rate on user experience, performance, and simulator sickness in virtual reality. IEEE Trans. Vis. Comput. Graph. 2023, 29, 2478–2488. [Google Scholar] [CrossRef] [PubMed]
  15. Gamba, I.A.; Hartery, A. The virtual reality radiology workstation: Current technology and future applications. Can. Assoc. Radiol. J. 2024, 75, 08465371241230278. [Google Scholar] [CrossRef] [PubMed]
  16. Hua, H.; Hu, X.; Gao, C. A high-resolution optical see-through head-mounted display with eyetracking capability. Opt. Express 2013, 21, 30993–30998. [Google Scholar] [CrossRef] [PubMed]
  17. Itoh, Y.; Klinker, G. Interaction-free calibration for optical see-through head-mounted displays based on 3D eye localization. In Proceedings of the 2014 IEEE Symposium on 3D User Interfaces, Minneapolis, MN, USA, 29–30 March 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 75–82. [Google Scholar]
  18. Plopski, A.; Itoh, Y.; Nitschke, C.; Kiyokawa, K.; Klinker, G.; Takemura, H. Corneal-imaging calibration for optical see-through head-mounted displays. IEEE Trans. Vis. Comput. Graph. 2015, 21, 481–490. [Google Scholar] [CrossRef] [PubMed]
  19. Meccawy, M. Creating an immersive XR learning experience: A roadmap for educators. Electronics 2022, 11, 3547. [Google Scholar] [CrossRef]
  20. Petrov, P.D.; Atanasova, T.V. The effect of augmented reality on students’ learning performance in STEM education. Information 2020, 11, 209. [Google Scholar] [CrossRef]
  21. Aromataris, E.; Fernandez, R.; Godfrey, C.M.; Holly, C.; Khalil, H.; Tungpunkom, P. Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. Int. J.-Evid.-Based Healthc. 2015, 13, 132–140. [Google Scholar] [CrossRef]
  22. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  23. Zotero. Credits and Acknowledgments. Available online: https://www.zotero.org/support/credits_and_acknowledgments (accessed on 28 June 2024).
  24. Parabhoi, L.; Seth, A.K.; Pathy, S.K. Citation management software tools: A comparison with special reference to Zotero and Mendeley. J. Adv. Libr. Inf. Sci. 2017, 6, 288–293. [Google Scholar]
  25. Stephen McMinn, H. Library support of bibliographic management tools: A review. Ref. Serv. Rev. 2011, 39, 278–302. [Google Scholar] [CrossRef]
  26. Cromley, J.G.; Chen, R.; Lawrence, L.E. Meta-analysis of STEM learning using virtual reality: Benefits across the board. J. Sci. Educ. Technol. 2023, 32, 355–364. [Google Scholar] [CrossRef]
  27. Lui, A.L.; Not, C.; Wong, G.K. Theory-based learning design with immersive virtual reality in science education: A systematic review. J. Sci. Educ. Technol. 2023, 32, 390–432. [Google Scholar] [CrossRef]
  28. Tsichouridis, C.; Batsila, M.; Vavougios, D.; Ioannidis, G. Virtual and augmented reality in science teaching and learning. In The Impact of the 4th Industrial Revolution on Engineering Education; Springer: Berlin/Heidelberg, Germany, 2020; pp. 193–205. [Google Scholar]
  29. Vidak, A.; Šapić, I.M.; Mešić, V. Augmented reality in teaching about physics: First findings from a systematic review. J. Phys. Conf. Ser. 2022, 2415, 012008. [Google Scholar] [CrossRef]
  30. Yin, X.; Li, G.; Deng, X.; Luo, H. Enhancing K-16 science education with augmented reality: A systematic review of literature from 2001 to 2020. In Proceedings of the 2022 8th International Conference of the Immersive Learning Research Network, Vienna, Austria, 30 May–4 June 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–5. [Google Scholar]
  31. Zhang, W.; Wang, Z. Theory and practice of VR/AR in K-12 science education—A systematic review. Sustainability 2021, 13, 12646. [Google Scholar] [CrossRef]
  32. Sırakaya, M.; Alsancak Sırakaya, D. Augmented reality in STEM education: A systematic review. Interact. Learn. Environ. 2022, 30, 1556–1569. [Google Scholar] [CrossRef]
  33. Ajit, G.; Lucas, T.; Louis, K. A systematic review of augmented reality in STEM education. Stud. Appl. Econ. 2021, 39, 1–22. [Google Scholar] [CrossRef]
  34. Hidayat, R.; Wardat, Y. A systematic review of augmented reality in science, technology, engineering and mathematics education. Educ. Inf. Technol. 2023, 29, 9257–9282. [Google Scholar] [CrossRef]
  35. Álvarez-Marín, A.; Velázquez-Iturbide, J.A. Augmented reality and engineering education: A systematic review. IEEE Trans. Learn. Technol. 2021, 14, 817–831. [Google Scholar] [CrossRef]
  36. Ahmad, N.; Junaini, S. Augmented reality for learning mathematics: A systematic literature review. Int. J. Emerg. Technol. Learn. (iJET) 2020, 15, 106–122. [Google Scholar] [CrossRef]
  37. Mazzuco, A.; Krassmann, A.L.; Reategui, E.; Gomes, R.S. A systematic review of augmented reality in chemistry education. Rev. Educ. 2022, 10, e3325. [Google Scholar] [CrossRef]
  38. Kalemkuş, J.; Kalemkuş, F. Effect of the use of augmented reality applications on academic achievement of student in science education: Meta analysis review. Interact. Learn. Environ. 2022, 31, 6017–6034. [Google Scholar] [CrossRef]
  39. Irwanto, I.; Dianawati, R.; Lukman, I. Trends of augmented reality applications in science education: A systematic review from 2007 to 2022. Int. J. Emerg. Technol. Learn. (iJET) 2022, 17, 157–175. [Google Scholar] [CrossRef]
  40. Hidajat, F.A. Augmented reality applications for mathematical creativity: A systematic review. J. Comput. Educ. 2023. [Google Scholar] [CrossRef]
  41. di Lanzo, J.A.; Valentine, A.; Sohel, F.; Yapp, A.Y.; Muparadzi, K.C.; Abdelmalek, M. A review of the uses of virtual reality in engineering education. Comput. Appl. Eng. Educ. 2020, 28, 748–763. [Google Scholar] [CrossRef]
  42. Pirker, J.; Dengel, A.; Holly, M.; Safikhani, S. Virtual reality in computer science education: A systematic review. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology, Virtual Event, 1–4 November 2020; pp. 1–8. [Google Scholar]
  43. Makransky, G.; Borre-Gude, S.; Mayer, R.E. Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments. J. Comput. Assist. Learn. 2019, 35, 691–707. [Google Scholar] [CrossRef]
  44. Lee, E.A.L.; Wong, K.W. Learning with desktop virtual reality: Low spatial ability learners are more positively affected. Comput. Educ. 2014, 79, 49–58. [Google Scholar] [CrossRef]
  45. Klingenberg, S.; Jørgensen, M.L.; Dandanell, G.; Skriver, K.; Mottelson, A.; Makransky, G. Investigating the effect of teaching as a generative learning strategy when learning through desktop and immersive VR: A media and methods experiment. Br. J. Educ. Technol. 2020, 51, 2115–2138. [Google Scholar] [CrossRef]
  46. Azevedo, R. Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educ. Psychol. 2015, 50, 84–94. [Google Scholar] [CrossRef]
  47. Hofer, M. Adolescents’ development of individual interests: A product of multiple goal regulation? Educ. Psychol. 2010, 45, 149–166. [Google Scholar] [CrossRef]
  48. Fidan, M.; Tuncel, M. Integrating augmented reality into problem based learning: The effects on learning achievement and attitude in physics education. Comput. Educ. 2019, 142, 103635. [Google Scholar] [CrossRef]
  49. Pena, B.M.; Gil Quilez, M. The importance of images in astronomy education. Int. J. Sci. Educ. 2001, 23, 1125–1135. [Google Scholar] [CrossRef]
  50. Akçayır, M.; Akçayır, G.; Pektaş, H.M.; Ocak, M.A. Augmented reality in science laboratories: The effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Comput. Hum. Behav. 2016, 57, 334–342. [Google Scholar] [CrossRef]
  51. Lu, S.-J.; Liu, Y.-C.; Chen, P.J.; Hsieh, M.R. Evaluation of AR embedded physical puzzle game on students’ learning achievement and motivation on elementary natural science. Interact. Learn. Environ. 2018, 28, 451–463. [Google Scholar] [CrossRef]
  52. Mejías Borrero, A.; Andújar Márquez, J. A pilot study of the effectiveness of augmented reality to enhance the use of remote labs in electrical engineering education. J. Sci. Educ. Technol. 2012, 21, 540–557. [Google Scholar] [CrossRef]
  53. Cheng, J.; Wang, Y.; Tjondronegoro, D.; Song, W. Construction of interactive teaching system for course of mechanical drawing based on mobile augmented reality technology. Int. J. Emerg. Technol. Learn. 2018, 13, 126–139. [Google Scholar] [CrossRef]
  54. Lin, H.C.K.; Chen, M.C.; Chang, C.K. Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system. Interact. Learn. Environ. 2013, 23, 799–810. [Google Scholar] [CrossRef]
  55. Palancı, A.; Turan, Z. How does the use of the augmented reality technology in mathematics education affect learning processes?: A systematic review. Int. J. Curric. Instr. Stud. 2021, 11, 89–110. [Google Scholar] [CrossRef]
  56. Tee, N.Y.K.; Gan, H.S.; Li, J.; Cheong, B.H.P.; Tan, H.Y.; Liew, O.W.; Ng, T.W. Developing and demonstrating an augmented reality colorimetric titration tool. J. Chem. Educ. 2018, 95, 393–399. [Google Scholar] [CrossRef]
  57. Zhu, B.; Feng, M.; Lowe, H.; Kesselman, J.; Harrison, L.; Dempski, R.E. Increasing enthusiasm and enhancing learning for biochemistry-laboratory safety with an augmented-reality program. J. Chem. Educ. 2018, 95, 1747–1754. [Google Scholar] [CrossRef]
  58. Thees, M.; Kapp, S.; Strzys, M.P.; Beil, F.; Lukowicz, P.; Kuhn, J. Effects of augmented reality on learning and cognitive load in university physics laboratory courses. Comput. Hum. Behav. 2020, 108, 106316. [Google Scholar] [CrossRef]
  59. Faridi, H.; Tuli, N.; Mantri, A.; Singh, G.; Gargrish, S. A framework utilizing augmented reality to improve critical thinking ability and learning gain of the students in physics. Comput. Appl. Eng. Educ. 2020, 29, 258–273. [Google Scholar] [CrossRef]
  60. Parong, J.; Mayer, R.E. Learning science in immersive virtual reality. J. Educ. Psychol. 2018, 110, 785–797. [Google Scholar] [CrossRef]
  61. Webster, R. Declarative knowledge acquisition in immersive virtual learning environments. Interact. Learn. Environ. 2016, 24, 1319–1333. [Google Scholar] [CrossRef]
  62. Trindade, J.; Fiolhais, C.; Almeida, L. Science learning in virtual environments: A descriptive study. Br. J. Educ. Technol. 2002, 33, 471–488. [Google Scholar] [CrossRef]
  63. Strzys, M.P.; Kapp, S.; Thees, M.; Klein, P.; Lukowicz, P.; Knierim, P.; Schmidt, A.; Kuhn, J. Physics holo.lab learning experience: Using smartglasses for augmented reality labwork to foster the concepts of heat conduction. Eur. J. Phys. 2018, 39, 035703. [Google Scholar] [CrossRef]
  64. Huang, K.T.; Ball, C.; Francis, J.; Ratan, R.; Boumis, J.; Fordham, J. Augmented versus virtual reality in education: An exploratory study examining science knowledge retention when using augmented reality/virtual reality mobile applications. Cyberpsychol. Behav. Soc. Netw. 2019, 22, 105–110. [Google Scholar] [CrossRef]
  65. Gil-Quílez, M.J.; Martínez-Peña, B.; De la Gándara, M.; Ambite, M.; Laborda, M. Constructing a model of digestion in a primary school using a theatrical performance. J. Life Sci. 2012, 6, 91–98. [Google Scholar]
Figure 1. PRISMA flowchart for the umbrella review.
Figure 1. PRISMA flowchart for the umbrella review.
Information 15 00515 g001
Figure 2. Type of extended reality technology and the STEM application domain.
Figure 2. Type of extended reality technology and the STEM application domain.
Information 15 00515 g002
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhang, Y.; Feijoo-Garcia, M.A.; Gu, Y.; Popescu, V.; Benes, B.; Magana, A.J. Virtual and Augmented Reality in Science, Technology, Engineering, and Mathematics (STEM) Education: An Umbrella Review. Information 2024, 15, 515. https://doi.org/10.3390/info15090515

AMA Style

Zhang Y, Feijoo-Garcia MA, Gu Y, Popescu V, Benes B, Magana AJ. Virtual and Augmented Reality in Science, Technology, Engineering, and Mathematics (STEM) Education: An Umbrella Review. Information. 2024; 15(9):515. https://doi.org/10.3390/info15090515

Chicago/Turabian Style

Zhang, Yiqun, Miguel A. Feijoo-Garcia, Yiyin Gu, Voicu Popescu, Bedrich Benes, and Alejandra J. Magana. 2024. "Virtual and Augmented Reality in Science, Technology, Engineering, and Mathematics (STEM) Education: An Umbrella Review" Information 15, no. 9: 515. https://doi.org/10.3390/info15090515

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop