Next Article in Journal
Empowering Healthcare: A Comprehensive Guide to Implementing a Robust Medical Information System—Components, Benefits, Objectives, Evaluation Criteria, and Seamless Deployment Strategies
Previous Article in Journal
Meat Texture Image Classification Using the Haar Wavelet Approach and a Gray-Level Co-Occurrence Matrix
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
ASI
Volume 7
Issue 3
10.3390/asi7030050
asi-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Boosting Engineering Education with Virtual Reality: An Experiment to Enhance Student Knowledge Retention

by
Fernando Elemar Vicente dos Anjos
1,2,*,
Adriano de Oliveira Martins
2,
Gislene Salim Rodrigues
2,3,
Miguel Afonso Sellitto
2,* and
Debora Oliveira da Silva
2
1
Federal Institute of Education, Science and Technology of Rio Grande do Sul (IFRS), Production Engineering School, Campus Caxias do Sul, Caxias do Sul 95043-700, Brazil
2
Production and Systems Engineering Graduate Program, UNISINOS—University of Vale do Rio dos Sinos, São Leopoldo 93055-750, Brazil
3
Federal Institute of Education, Science and Technology of Rio Grande do Sul (IFRS), Campus Rio Grande, Mechanical Engineering School, Rio Grande 96401-250, Brazil
*
Authors to whom correspondence should be addressed.
Appl. Syst. Innov. 2024, 7(3), 50; https://doi.org/10.3390/asi7030050
Submission received: 17 April 2024 / Revised: 5 June 2024 / Accepted: 11 June 2024 / Published: 13 June 2024
Browse Figures
Versions Notes

Abstract

:
This article is about experiments investigating teaching and learning processes and their effects on students. Specifically, the laboratory experiment method aims to determine if using virtual reality in classes leads to better learning outcomes, knowledge retention, satisfaction, engagement, and attractiveness compared to traditional teaching methods. The study found that students who used VR (Experimental Group—EG) had significantly better learning outcomes (with an average of 5.9747) compared to the control group (Control Group—CG), who only had traditional classes (with an average of 4.6229). The study employed a Likert scale from 1 to 7. The difference between EG and CG was 29.2%. Furthermore, the study found that students in the EG had higher knowledge retention, satisfaction, engagement, and attractiveness compared to the CG. All measurements were above 6.4 on the same scale. This study is important because it explores innovative teaching methods and their potential to improve learning outcomes, satisfaction, and efficiency. It also opens up avenues for further research on teaching methodologies for undergraduate students.

1. Introduction

The search for more efficient teaching–learning approaches and positive impacts on student learning is ongoing and carried out by researchers, teachers, and professionals in the field of education. In response to this discussion and the need for change, a conference and research article addressed MIT’s program for this purpose [1]. The research article explored the integration of new technologies in engineering education, emphasizing the positive impact of cognitive approaches for the teaching–learning process on the training and performance of engineers. The teaching–learning process plays a crucial role in engineering education, connecting theoretical knowledge with practical application. Cognitive internships organized before professional courses transform students’ engineering knowledge from a superficial understanding to a broader awareness [2]. New approaches employed to attain the anticipated level of efficacy involve the implementation of Education 4.0 principles, which encompass the utilization of advanced and transformative methodologies targeted at higher education [3,4,5,6,7,8].
One of such approaches is the use of virtual reality (VR). VR can be found in various knowledge fields, such as biomedical engineering [9], construction projects [10], chemical [11], 5S training [12], civil engineering [13], manufacturing processes [14,15], physics concepts [16], logistics services [17], robotics [18], industrial design [19], engine assembly [20], spatial geometry [21], urban railway vehicle engineering [22], automotive industry [23], manufacturing sustainability in Industry 4.0 [24], mechanical engineering [25,26], computer engineering [27], smart tourism [28], among others.
The use of VR in engineering education has shown a 60% increase in learning through experiments compared to traditional teaching methods [29]. Additionally, 83% of participants reported greater satisfaction with the use of VR [30]. However, there needs to be more publications on the application of VR in teaching–learning processes, specifically in the field of production engineering, with limited topics such as supply chain management, process flow, layout organization, and lean manufacturing concepts [31,32]. There are several opportunities for future research in areas such as production systems management, production planning and control, material handling, production simulation, production process management, quality control, work organization, risk analysis and prevention, and ergonomics [33], as well as emergent fields such as Industry 5.0 and renewable energy systems management.
The purpose of this study is to assess the effectiveness of VR in enhancing learning outcomes compared to traditional teaching methods. Through an experimental setup involving two distinct groups, the study seeks to determine whether the group exposed to VR exhibits superior learning rates compared to those subjected to traditional teaching. It also aims to measure the extent of knowledge retention across both groups as well as the perceived levels of satisfaction, engagement, and attractiveness among students utilizing VR in contrast to students following conventional teaching approaches.
The study is important because it delves into the area of production sequencing, which has yet to be thoroughly examined in the realm of production engineering processes with VR. Additionally, the utilization of traditional teaching methods in engineering classes, given the new technologies implemented in the manufacturing realm, may lead to lower learning rates and knowledge gaps [33]. Previous studies have shown positive impacts on learning [34,35] and on the satisfaction, engagement, and attractiveness perceived by students subjected to VR approaches [36,37].
The rest of the article is structured in terms of theoretical background, materials and methods, results, discussions, and conclusions.

2. Theoretical Background

Consistently, educators employ technological tools to enhance teaching and learning processes, with the goal of promoting more substantial student learning. Notable examples include gamification [38], PhET simulations [39], Kahoot [40], augmented reality [41,42], the Internet of Things (IoT) [43], flipped classrooms [44,45], and VR [46,47], among others. Teaching strategies have their own characteristics. For instance, industrial gamification applies game elements to enhance learning [48,49], problem-solving, cooperation, motivation, and engagement [38]. PhET simulation is a platform that creates interactive simulations in physics, chemistry, mathematics, earth sciences, and biology to facilitate students’ learning processes, participation, and engagement [50,51]. Kahoot is an educational gaming platform that allows students to play interactive games and quizzes to improve their absorption of subjects and disciplines [40,52].
Along with VR, augmented reality (AR) offers the exciting potential for virtual objects to be seamlessly integrated into the real world, allowing for human interaction and visualization [53]. This cutting-edge technology immerses users in a multi-sensory experience by projecting virtual objects directly into the human eye through optical headsets, AR glasses, and other display devices. Virtual objects turn out to be incredibly realistic, with detailed surface characteristics and high scene resolution [54,55]. VR and AR need to rely on the Internet of Things (IoT) infrastructures, which refer to a network of physical objects embedded with sensors, software, and other technologies that enable them to communicate and share data with other devices and systems via the Internet. IoT integrated with VR and AR has created new possibilities in the realm of education, fostering improved interactions between people and physical objects in a classroom environment [55,56].
The flipped classroom model involves students being introduced to the subject material before coming to class, allowing them to develop initial understanding and concepts. This preparatory work often takes place in virtual teaching environments through various interactive activities such as games, videos, and exercises [44]. VR technology can create fully immersive and interactive environments, offering users a unique and impactful experience that can enhance cognitive performance in various ways, including access, awareness, engagement, creativity, and action [37,57]. Although VR is not widely used in engineering education, it has the potential to create a virtual learning environment that can greatly benefit students’ cognitive performance [33,46].

2.1. VR and Impact on Learning

Previous studies have highlighted VR’s positive effects on the teaching and learning experience, including increased learning and problem-solving abilities [35]. VR has also been shown to enhance the learning experience in dentistry classes and support proper positioning in physical exercises [34,58]. Additionally, it has been found to improve understanding of flat and three-dimensional geometric figures [59] and foster greater creativity in elementary school students [60]. Specifically in engineering education, VR has shown positive learning outcomes compared to traditional teaching methods. Such outcomes include an increased understanding of automation [61], a focus on visualizing projects and systems [62], improved learning in software engineering development training compared to traditional methods [63], and greater understanding in classes on mechanical testing [64]. Specifically for engineering software development, VR may lead to more than 100% improvement in learning [65].
Conversely, certain studies indicate that VR may have a less significant impact on student learning, particularly in areas related to product development [25]. However, in the field of robotics, the findings suggest a greater understanding of certain topics when VR is utilized [26]. While VR is commonly employed in educational settings with various learning outcomes, there are instances where the results are not optimistic. Various studies have evaluated the impact of knowledge retention, although to a lesser extent and with varying results. For example, in laparoscopy surgery training, no differences in skills and knowledge were observed between groups using traditional teaching methods versus VR support after one month [66]. However, there are conflicting findings, with one study showing significant retention of learning by children [67], while another study found that students using VR achieved 80% retention compared to 25% retention for those using traditional teaching methods [68].

2.2. VR and Impact on Student Satisfaction and Engagement

Previous studies have assessed various aspects of incorporating VR into teaching and learning, including user satisfaction, engagement, and attractiveness. For instance, a study found a 76% increase in satisfaction among dentistry students [34], while another reported a 4.28 motivation score (on a Likert scale of 1–5) for science students [37]. Additionally, VR was found to enhance engagement in training for industrial systems [36] and boost motivation in practical nursing studies [69].
Regarding engineering education processes, numerous studies report positive outcomes. For example, research has shown increased engagement in testing mechanical materials [64], greater satisfaction in developing prototypes with VR and additive manufacturing [70], and a 92% student understanding that VR enhances learning and makes the process more attractive, leading to increased engagement in classes [71]. Additionally, a survey revealed that 92% of students feel more satisfied when using VR in teaching and learning processes, particularly in the context of electrical systems [72] and the simulation of welding processes with robots [73].
Conversely, the research conducted by [74] did not find any significant increase in motivation, satisfaction, or any other attribute among participants as a result of using the virtual environment, particularly in the cultural training program for high school students. In the earthmoving simulation, participants noted numerous flaws in the simulator and found it difficult to operate, ultimately impacting their satisfaction [75]. Similarly, difficulties in manipulating the applications were observed during the construction of bridge projects, which hindered the potential benefits of the new application [76].
Based on the search for new disruptive technologies for application in the teaching–learning processes of production engineering, future research opportunities, and the positive impacts related to the learning and satisfaction of participants in the experiments, the following three research questions are presented:
  • RQ1—Is there more significant learning from students who use VR in teaching–learning than from students who only take traditional classes?
  • RQ2—Did the group that used VR in the teaching–learning processes perceive satisfaction, engagement, and attractiveness in the activities, compared to classes that only used the traditional teaching method?
  • RQ3—Does applying VR impact knowledge retention three weeks after using the knowledge test acquired by students?

3. Materials and Methods

The study utilizes a laboratory experiment. Unlike a field experiment, a laboratory experiment operates on a smaller scale and controls a limited number of variables. In contrast, a field experiment would involve true magnitude and require a longer duration, potentially impacting academic records with potential consequences for students. The presence of real agents and their genuine responses distinguishes the laboratory experiment from an in silico experiment, which simulates agent behavior using software, which can lead to errors if the behaviors are not fully known or modeled. The advantage of a laboratory experiment is the ability to make corrections without real-world implications, but it may overlook certain variables and nuances present in a larger-scale experiment. Following the experiment, a mathematical analysis is required to validate results and test hypotheses. The EG and the CG were chosen for comparative analysis [76].
The study was structured into six stages, which are as follows:
  • Definition of research objectives;
  • Development of the VR artifact;
  • Validation of the VR artifact with experts;
  • Application of the VR artifact in controlled groups;
  • Analysis and interpretation of results;
  • Conclusions and proposals for future research.
The introduction describes the first step of the method and presents the cognitive approach to teaching. VR is also discussed, along with some examples of its application and results in learning, satisfaction, attractiveness, and student engagement. The second, third, and fourth steps reference the steps presented in Figure 1. Step 5 analyzed the results and discussed the literature on the topic. Finally, step 6 presented conclusions and proposals for future research.
The steps presented in Figure 1 are described below:
In Stage I, the model’s assumptions were utilized to simulate production scenarios. Such assumptions involved adjusting various inputs such as processing times, production batches, setup times, MTBF, and MTTR. The production system output indicators, such as WIP levels, production lead time, quantities produced, and production costs, were then evaluated.
In Stage II, the Plant Simulation software was utilized to model dynamic systems, allowing for the simulation of input changes and control of output indicators, as well as the generation of 3D VR images. Non-immersive VR was employed without direct interaction with students.
Moving on to Stage III, the researchers presented and validated the VR model with three experts: two engineering professors and a market professional with over a decade of experience in production sequencing. This validation process involved a comprehensive evaluation of the model’s input data, output indicators, and the visual and dynamic aspects of the VR environment.
In Stage IV of the study, an experiment was conducted to assess the impact of a specific teaching method on student learning, satisfaction, engagement, and attractiveness. The sample comprised 68 students from the Production Engineering Course, with 34 students in each group—the control group (CG) and the experimental group (EG). The sample was defined in a non-probabilistic way for convenience [77]. The total student population in the course is 230. All students were invited to participate, and those who agreed formed the two groups based on availability. The control group received ten hours of traditional teaching. In comparison, the experimental group received six and a half hours of traditional teaching and an additional three and a half hours with VR support. Three weeks after the last session, a study was conducted to evaluate the knowledge retention level of each group [68].
Stages V, VI, and VII—three stages were applied for knowledge factual assessments, and four analyses were carried out. RQ1 is investigated by testing hypotheses H1, H2, and H3. RQ2 is investigated by the qualitative part of the questionnaire presented in Appendix B, which belongs to Stage VII of the method. RQ3 is investigated by testing hypothesis H4. The hypotheses are as follows:
  • H1—the previous knowledge of the groups is the same on production sequencing. The hypothesis alternative is H11—there is a difference in the previous knowledge between the groups on production sequencing.
  • H2—the knowledge of the groups on production sequencing did not vary after classes. The hypothesis alternative is H21—there is a difference between the current and previous knowledge of groups on production sequencing after classes.
  • H3—the learning levels of the groups on production sequencing after classes are the same. The hypothesis alternative is H31—there is a difference between the learning levels of the groups on production sequencing after classes.
  • H4—the retention of knowledge by the groups on production sequencing is the same three weeks after classes. The hypothesis alternative is H41—there is a difference between the retention of knowledge of the groups on production sequencing after three weeks.
The p-value must be <0.01 in all evaluations to accept the alternative hypothesis.
A preliminary test was conducted to gauge the existing knowledge of both groups. Following the third meeting, a subsequent test was administered to assess the learning progress of each group. Lastly, three weeks after the conclusion of the last class, a test measured the retention of knowledge among the groups. All tests were scored on a scale from 0.00 to 10.00. Table 1 shows the quantitative method utilized to compare the groups, which was the Student’s t-test, and the findings.
The t-test was selected to determine if the difference in means is statistically significant [77]. The Shapiro–Wilk test was used to assess the normality of the data from each group based on their prior knowledge and retention. To accept the alternative hypothesis, a significance level of α = 1% was applied to all tests. SPSS version 22 was used for the analysis. The results of the normality tests can be found in Appendix A.
In the final stage, the experimental group received a questionnaire consisting of 12 questions. The questions were divided into three blocks concerning issues related to satisfaction (questions 1–4), engagement (questions 5–8), and attractiveness (questions 9–12). The Likert scale, ranging from 1 to 7, was used for the questionnaire. The Likert scale is a widely used tool in research to measure people’s attitudes and opinions. It provides a systematic method for quantifying subjective perspectives, ensuring a balanced range of choices. It includes a central neutral point surrounded by an equal number of options on each side of the scale [78]. The questionnaire and scale are available in Appendix B. Six qualitative questions were utilized to assess students’ perceptions of satisfaction, appeal, and involvement in using VR for teaching and learning. Qualitative responses were categorized according to at least one keyword: ease of comprehension, realism, increased class dynamism, attractiveness, engagement, willingness to participate, scenario-based examples, class interest, reduced boredom, enjoyable experience, enhanced class participation, motivation, attentiveness, and positive impact on happiness.
References [29,30,34,79,80,81] provide empirical evidence on the feasibility and applicability of the questions, ensuring the validity of the questionnaire. Previous empirical studies involving the aspects captured by the questions ensure that the involved issues are likely to be found in such an experiment. The quantitative part of the questionnaire was structured into three blocks that focused on aspects related, respectively, to satisfaction, engagement, and attractiveness of the application. Cronbach’s alpha calculation helped to assess the reliability of the responses. The reliability was confirmed as all alphas are greater than 0.60 [76] (0.73, 0.83, and 0.69, respectively). A limitation of the study is the identification of underlying factors, which is forwarded for further applications with an adequate number of respondents. Appendix B shows the questions, blocks, references, and type of response, quantitative or qualitative.
The Research Ethics Committee approved this research from the Federal Institute of Rio Grande do Sul—IFRS (Brazil) under CAAE 52081521.0.0000.8024.

4. Results

Table 2 provides demographic information on the sample.

4.1. Description of Artifact

The VR artifact was created to integrate key elements into the teaching and learning of production sequencing. It was developed using Plant Simulation software to simulate dynamic systems, specifically a production system, through a VR application. The VR artifact utilized six different scenarios to model the production system, with each scenario adjusting input data and controlling output information [79]. The data input involved processing time per product, setup time, demand, production batch size, available capacity, production flow, and equipment availability indexes (maintenance and reliability). The system was designed to operate over 14 days, producing eight different products, each with its own specific data. We evaluated the quantities produced, demand fulfillment, cash flow generated, average and maximum production lead time, OEE (overall equipment efficiency), number of setups, and in-process stock levels for each scenario. The initial development of the artifact, including simulations and parameter adjustments, took 40 h. During the validation stage, each evaluator received individual presentations. The evaluators noted several positive aspects, including the impact of scenarios on evaluated indicators, the ability to visualize the production system’s operation, the dynamics of inventory in the process, and the ease of adjusting scenario input data.
Additionally, a list was created with six opportunities to modify parameters and the visual aspects of the VR environment. The opportunities included adjusting the production batch size parameter to observe the impact on in-process inventory levels and manufacturing lead time, highlighting the effect on cash flow by increasing work-in-process inventory levels, analyzing the quantity produced exceeding the expected demand for the 14-day simulation, entering a different color for each product in the VR model, incorporating conveyors to enhance material flow visualization in VR, and limiting the number of parts on the conveyor belt awaiting processing to five units to emphasize in-process stock in the buffers. After adjustments, the model was presented to all experts collectively, and all features were reassessed. The experts deemed the model suitable for implementation in achieving the proposed objective. Figure 2 displays the VR images of the production process.

4.2. Results of the Previous Knowledge and Retention Tests

The knowledge of the control and experimental groups regarding the evaluated topic was compared before the first class and after correcting the pre-tests, which assess knowledge of production sequencing. After assessing the results, no statistical differences were found between the groups. The results are detailed in Table 3.
In Table 3, CG and EG achieved prior knowledge scores of 3.5091 and 3.9361, respectively. Therefore, the first alternative hypothesis (H11) is not supported, suggesting similar levels of previous knowledge in production sequencing. Subsequently, the knowledge acquired by each group after attending the classes was assessed, and the results of the learning t-tests are detailed in Table 4 and Table 5.
Table 4 and Table 5, which display the results of the t-test, suggest that the second alternative hypothesis, H21, is supported based on the level of significance (p-value < 0.01). The results indicate a 51.8% increase in the EG and a 31.7% increase in the CG in the average knowledge acquired from the post-test compared to the pre-test. An additional analysis was carried out to determine whether any group showed greater improvement, and the findings are outlined in Table 6.
Table 6 displays the result of the knowledge test administered after classes, with scores of 4.6229 and 5.9747 for the control group (CG) and experimental group (EG), respectively. The t-test yielded a p-value of 0.000012 (less than 0.01). The third alternative hypothesis, H31, is supported. The EG group’s result was 29.2% higher. EG, who had VR-supported classes, achieved more significant and statistically meaningful learning outcomes than the CG. Table 7 analyzes knowledge retention between the groups conducted three weeks after the initial knowledge tests.
The data in Table 7, which evaluate the results for hypothesis 4, show that the experimental group maintained higher results than the control group after three weeks of testing acquired knowledge. The knowledge retention results for the experimental group were, on average, 20.1% higher than those of the control group. The t-test yielded a p-value of 0.0028 (less than 0.01). The fourth alternative hypothesis, H41, is supported. The EG’s results are statistically higher. Additionally, further analysis was conducted on each group’s knowledge and retention test results, and the findings are detailed in Table 8.
Upon reviewing the data in Table 8, it is evident that the CG’s knowledge retention decreased by 4.8% (from 4.6229 to 4.3988). The EG also exhibited a decrease in knowledge retention and acquisition of 11.6% (from 5.9747 to 5.2841) despite maintaining higher knowledge results.

4.3. Assessment of Concerns Related to Satisfaction, Engagement, and Attractiveness

Figure 3 presents the EG’s perception of the application of VR in teaching–learning processes with VR support.
Figure 3 displays the average ratings given by participants in the experimental group for each question in the evaluation. The highest rating was 6.8, while the lowest was 6.2, indicating that, overall, the group that received VR-supported classes strongly believes that VR positively affects student satisfaction, engagement, and attractiveness in teaching. Additionally, Table 9 presents descriptive statistics for the twelve questions, providing information about the experimental group’s perception of VR’s impact on teaching and learning processes.
The results displayed in Table 9 are consistent with the summarized averages in Figure 3. None of the questionnaire responses fell below a score of 4 on the Likert scale of 1–7, indicating that, overall, the feedback was positive. Upon analyzing the 408 responses (N = 34 × 12 questions), we found only 4 neutral responses (scoring 4), and approximately 93% of respondents selected options on scales 6 and 7. Regarding the blocks (1 to 4 for satisfaction, 5 to 8 for engagement, and 9 to 12 for attractiveness), all satisfaction ratings were at least a 5, with engagement receiving 87% of responses between 6 and 7 and attractiveness garnering 98% of responses between 6 and 7.
The findings reveal that students express high levels of satisfaction, engagement, and interest in using virtual reality (VR) for educational purposes. Figure 4 illustrates that integrating VR into educational activities enhances content comprehension by offering a more immersive learning experience. Participants noted that VR usage enhances class interactivity, engagement, and dynamism.

5. Discussion

The initial finding (Table 4 and Table 5) showed that regardless of the teaching method (traditional or traditional with VR support), both groups (EG and CG) learned after participating in classes on production sequencing. However, upon evaluating the group results (Table 6), it was determined that the EG achieved a statistically significant learning outcome (5.9747), which was 29.2% higher than the CG (4.6229). The application of VR in teaching–learning processes generated a real opportunity for more significant student learning, corroborating similar previous studies [34,58,59].
A follow-up assessment was conducted on the groups’ knowledge retention three weeks after the initial knowledge tests [66], revealing a decrease in the average scores for both groups. The control group (CG) achieved a score of 4.3988, while the experimental group (EG) achieved a score of 5.2841. Statistical analysis yielded a p-value of 0.0028, which indicates significant differences between the groups. The data provide strong evidence that the EG has greater knowledge of production sequencing compared to the CG after three weeks, consistent with previous studies [67,68].
Furthermore, regarding knowledge retention, Table 8 illustrates a 4.8% decrease for the CG (from 4.6229 to 4.3988) and an 11.6% decrease for the EG (from 5.9747 to 5.2841). The EG lost more knowledge than the CG after three weeks. Even if the causes are outside the scope of the present study, it prompts further investigation into potential hypotheses, such as specific environmental factors or lapses in focus during the retention test. Finally, what can be seen from the results of the experiment is that students learn regardless of the teaching method applied. However, integrating VR into teaching and learning fosters a cognitive environment [1,2] conducive to deeper learning and enhanced knowledge retention over a three-week period. Further research and experimentation are required to explore the varying levels of knowledge retention among different groups.
In evaluating the satisfaction, engagement, and attractiveness indexes for EG, all average results exceeded 6.0 on a Likert scale of 1–7. The highest average (6.8) was about the class being more enjoyable with VR, and the lowest average (6.2) was about feeling a greater desire to participate through understanding the content developed in class. Table 9 presents the data from the questionnaires that assessed levels of satisfaction, engagement, and attractiveness. More than 90% of the responses had indices between 6 and 7. These results present positive and relevant impacts on student satisfaction, engagement, and attractiveness rates, as indicated by similar studies [17,34,36,69,70,71,72,73].
A recent assessment utilized qualitative analysis of participants’ essay responses (EG). The findings revealed that VR-based classes lead to a deeper understanding of the subject matter, offering a more realistic, dynamic, and engaging learning experience compared to traditional teaching methods. These results are consistent with similar studies [34,35,36,37,70]. The findings indicate that establishing a cognitive teaching and learning environment using VR leads to greater learning and knowledge retention among students compared to traditional teaching methods. Participants find it engaging and satisfying, making it a more attractive option. A critical aspect of these results is the careful construction of the VR environment with expert support and a well-planned experiment. The smooth execution of these steps, along with expert knowledge, is crucial for generating knowledge and creating opportunities to develop solutions to societal needs [80].
Educational settings that prioritize teaching and learning create more effective learning outcomes for students by fostering a cognitive environment, a deeper understanding of the subject matter, and a stronger connection to real-world applications. This approach brings a professional work-like environment into the classroom, promoting dynamic and engaging classes with increased student interaction and participation. Additionally, the use of VR enhances learning and results in higher levels of student engagement, satisfaction, and concentration.

6. Conclusions

This article aimed to compare the learning levels of two groups: one group received traditional teaching methods, while the other group received VR support. The study also evaluated satisfaction, engagement, and attractiveness with the use of VR in the teaching–learning process. The results showed that both groups learned significantly about production sequencing, the topic studied, regardless of the teaching method. However, the group with VR support showed more significant learning after classes and retained a higher level of knowledge after three weeks. On average, the satisfaction, engagement, and attractiveness indices for the VR-supported group were close to 6.56 points (on a Likert scale of 1–7), indicating that the method generated enthusiasm and was well accepted by the students. The evaluation of qualitative questions highlighted concerns such as ease of understanding the topics covered and the appeal of virtualization, making the classes more dynamic and attractive.
The study has limitations. First, the application of VR focused only on production sequencing, so the results may not apply to some other areas of knowledge. Additionally, the study did not account for the participants’ previous experiences, curiosity about the topic, and interest in the activity, which could have affected the retention of information in the VR-supported group. Another limitation is the small number of respondents, which did not allow for the identification of underlying factors in the perceptions of the students. Further studies should focus on knowledge retention over longer periods, such as three or six months after classes. Additionally, developing a standardized method for creating teaching–learning process activities and evaluating their quality and results would be beneficial. Additionally, directing attention toward emerging themes in production and operations management education, such as the shift from Industry 4.0 to human-centric systems (Industry 5.0) [82] and local renewable energy systems [83], may offer intriguing avenues for future research. Finally, employing a large sample of respondents could help improve the measurement model by exploring underlying factors in the students’ responses.

Author Contributions

Conceptualization, F.E.V.d.A., D.O.d.S. and A.d.O.M.; methodology, F.E.V.d.A., D.O.d.S. and A.d.O.M.; validation, G.S.R. and M.A.S.; formal analysis, G.S.R. and M.A.S.; original draft preparation, F.E.V.d.A.; writing—review and editing, F.E.V.d.A., D.O.d.S. and M.A.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received funding from CNPq, the Brazilian Research agency, under the grant number 303496/2022-3.

Data Availability Statement

No further data are available for this research.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Table A1. Tests of the normality of samples.
Table A1. Tests of the normality of samples.
GroupsShapiro-Wilk
Statistics dfSig.
Pre-test resultsCG0.989340.981
EG0.986340.939
Post-test resultsCG0.978340.713
EG0.966340.362
Retention-test resultsCG0.961340.256
EG0.959340.224

Appendix B

Table A2. Questionnaire.
Table A2. Questionnaire.
Question/ReferenceType of Question
1—Did you feel more satisfied with the application of VR in the teaching–learning processes? [34]Quantitative scale
1—Strongly disagreed
2—Disagree
3—Partially Disagreed
4—Neither disagreed nor agreed
5—Partially agreed
6—Agreed
7—Strongly agreed
2—Did the environment with VR influence your willingness to study? [30]
3—Did you feel happier and more excited to use the technology in the teaching–learning process? [79]
4—Did you find visualizing the production system through VR more useful than with a traditional approach? [29]
5—Did you feel more motivated to participate in the class using VR compared to traditional classes? [30]
6—Did the environment with VR help you understand the content of the class better? [29]
7—Do you believe that students would participate more actively in learning using VR? [34]
8—Do you believe that students will be more motivated to participate in classes with a VR application? [80]
9—Do you believe that future VR application classes will be more attractive than traditional classes? [30]
10—Do you believe that the dynamic environment and the learning possibilities of VR make the class more attractive than traditional classes? [30]
11—Do you believe that more VR in the curriculum would make the course more attractive to new students? [31]
12—Do you believe that students will accept participating in more classes on other subjects with VR applications? [79]
13—Highlight some points about your satisfaction (positive or negative) with using the VR artifact in the teaching–learning processes of production sequencing. [34]Qualitative responses
14—Was there any impact (positive or negative) on your motivation to study using VR? [81]
15—Regarding engagement, did you feel more engaged (participative and willing to participate) in class when VR was applied? Please describe. [80]
16—Do you believe that the application of VR as part of the teaching–learning process generates greater attractiveness (student interest) in attending classes? [81]
17—Do you believe that the virtual environment generated by VR provides students with a greater learning opportunity? Why do you think so? [29]
18—Tell us about elements that could be different in this teaching–learning process using VR. [30]

References

  1. Crawley, E.; Hosoi, A.; Long, G.; Kassis, T.; Dickson, W.; Mitra, A. Moving Forward with the New Engineering Education Transformation (NEET) Program at MIT—Building Community, Developing Projects, and Connecting with Industry. In Proceedings of the 2019 ASEE Annual Conference & Exposition Proceedings, ASEE Conferences, Tampa, FL, USA, 16–19 June 2019. [Google Scholar]
  2. Lavi, R.; Bagiati, A. The New Engineering Education Transformation Program at Massachusetts Institute of Technology: The Evolving Design and Implementation of a Programmatic Evaluation Study. In Transdisciplinarity and the Future of Engineering; IOS Press: Amsterdan, The Netherlands, 2022; pp. 658–667. [Google Scholar]
  3. Chaka, C. Is Education 4.0 a Sufficient Innovative, and Disruptive Educational Trend to Promote Sustainable Open Education for Higher Education Institutions? A Review of Literature Trends. Front. Educ. 2022, 7, 824976. [Google Scholar] [CrossRef]
  4. Rodríguez-García, A.; Arias-Gago, A.R. Uso de Modelos Didácticos En España En El Contexto Del COVID-19: Estudio Comparativo Entre Cuerpos Docentes. Rev. Interuniv. Form. Profr. 2022, 97, 209–230. (In Spanish) [Google Scholar] [CrossRef]
  5. Low, M.C.; Lee, C.K.; Sidhu, M.S.; Lim, S.P.; Hasan, Z.; Lim, S.C. Blended Learning for Engineering Education 4.0: Students’ Perceptions and Their Learning Difficulties. Comput. Appl. Eng. Educ. 2023, 31, 1705–1722. [Google Scholar] [CrossRef]
  6. Umbara, D.M.A. Revolutionizing Education 4.0: Evaluating the Role of Robots in Learning Effectiveness. In Proceedings of the E3S Web of Conferences; EDP Sciences: Les Ulis, France, 2024; Volume 482. [Google Scholar]
  7. Liu, Q.; Yu, S.; Chen, W.; Wang, Q.; Xu, S. The Effects of an Augmented Reality Based Magnetic Experimental Tool on Students’ Knowledge Improvement and Cognitive Load. J. Comput. Assist. Learn. 2021, 37, 645–656. [Google Scholar] [CrossRef]
  8. Ibáñez, M.-B.; Delgado-Kloos, C. Augmented Reality for STEM Learning: A Systematic Review. Comput. Educ. 2018, 123, 109–123. [Google Scholar] [CrossRef]
  9. Wilkerson, M.; Maldonado, V.; Sivaraman, S.; Rao, R.R.; Elsaadany, M. Incorporating immersive learning into biomedical engineering laboratories using virtual reality. J. Biol. Eng. 2022, 16, 20. [Google Scholar] [CrossRef] [PubMed]
  10. Kharvari, F.; Kaiser, L.E. Impact of Extended Reality on Architectural Education and the Design Process. Autom. Constr. 2022, 141, 104393. [Google Scholar] [CrossRef]
  11. Sreekanth, N.S.; Varghese, N.; Babu, N.S.C. Virtual Chemistry Lab to VR Chemistry Lab. Resonance 2022, 27, 1371–1385. [Google Scholar] [CrossRef]
  12. Kwok, A.P.K.; Yan, M.; Deng, X.H.; Chen, X.Y.; Huang, Y.T. Exploring the Facilitating and Obstructing Factors of Using VR for 5S Training: An Exploratory Qualitative Study from Students’ Perspectives in an Industrial Engineering Undergraduate Course. Comput. Appl. Eng. Educ. 2022, 30, 1072–1085. [Google Scholar] [CrossRef]
  13. Wang, X.; Chou, M.; Lai, X.; Tang, J.; Chen, J.; Kong, W.K.; Chi, H.-L.; Yam, M.C.H. Examining the Effects of an Immersive Learning Environment in Tertiary AEC Education: CAVE-VR System for Students’ Perception and Technology Acceptance. J. Civ. Eng. Educ. 2024, 150, 2. [Google Scholar] [CrossRef]
  14. Kim, H.; Hartleb, T.; Bello, K.; Aqlan, F.; Zhao, R.; Yang, H. Behavioral Modeling of Collaborative Problem Solving in Multiplayer VR Manufacturing Simulation Games. J. Comput. Inf. Sci. Eng. 2024, 24, 031006. [Google Scholar] [CrossRef]
  15. de Giorgio, A.; Monetti, F.M.; Maffei, A.; Romero, M.; Wang, L. Adopting Extended Reality? A Systematic Review of Manufacturing Training and Teaching Applications. J. Manuf. Syst. 2023, 71, 645–663. [Google Scholar] [CrossRef]
  16. Negahban, A. Simulation in Engineering Education: The Transition from Physical Experimentation to Digital Immersive Simulated Environments. Simulation 2024. ahead of print. [Google Scholar] [CrossRef]
  17. Liu, L. Order Allocation Optimization and Genetic Algorithm in Logistics Service Supply Chain. In Frontier Computing on Industrial Applications Volume 2; Hung, J.C., Yen, N., Chang, J.-W., Eds.; Springer Nature: Singapore, 2024; pp. 194–198. [Google Scholar]
  18. Vassigh, S.; Bogosian, B.; Peterson, E. Performance-Driven VR Learning for Robotics. In Phygital Intelligence; Yan, C., Chai, H., Sun, T., Yuan, P.F., Eds.; Springer Nature: Singapore, 2024; pp. 356–367. [Google Scholar]
  19. Hamurcu, A.; Timur, S.; Rizvanoglu, K. A Study on the Adoption of VR in Industrial Design Education. IEEE Trans. Learn. Technol. 2023, 16, 900–913. [Google Scholar] [CrossRef]
  20. Win, L.L.; Aziz, F.A.; Hairuddin, A.A.; Abdullah, L.N.; Yap, H.J.; Saito, H.; Seyajah, N. Effectiveness on training method using virtual reality and augmented reality applications in automobile engine assembly. ASEAN Eng. J. 2022, 12, 83–88. [Google Scholar] [CrossRef]
  21. Tarasova, L.; Pozdeeva, T.; Baranova, A. Study of the Readiness of Students to Use VR Technologies during a Course in Engineering Geometry and Computer Graphics. In Proceedings of the 2022 VI International Conference on Information Technologies in Engineering Education (Inforino), Moscow, Russia, 12–15 April 2022; pp. 1–5. [Google Scholar]
  22. Yang, J.; Liu, F.; Wang, J.; Kou, Z.; Zhu, A.; Yao, D. Effect of VR Technology on the Teaching of Urban Railway Vehicle Engineering. Comput. Appl. Eng. Educ. 2021, 29, 1163–1175. [Google Scholar] [CrossRef]
  23. Cordero-Guridi, J.-J.; Cuautle-Gutiérrez, L.; Alvarez-Tamayo, R.-I.; Caballero-Morales, S.-O. Design and Development of a I4.0 Engineering Education Laboratory with Virtual and Digital Technologies Based on ISO/IEC TR 23842-1 Standard Guidelines. Appl. Sci. 2022, 12, 5993. [Google Scholar] [CrossRef]
  24. Salah, B.; Abidi, M.; Mian, S.; Krid, M.; Alkhalefah, H.; Abdo, A. VR-Based Engineering Education to Enhance Manufacturing Sustainability in Industry 4.0. Sustainability 2019, 11, 1477. [Google Scholar] [CrossRef]
  25. Schumann, M.; Fuchs, C.; Kollatsch, C.; Klimant, P. Evaluation of augmented reality supported approaches for product design and production processes. Procedia CIRP 2021, 97, 160–165. [Google Scholar] [CrossRef]
  26. Lee, J.; Shvetsova, O. The Impact of VR Application on Student’s Competency Development: A Comparative Study of Regular and VR Engineering Classes with Similar Competency Scopes. Sustainability 2019, 11, 2221. [Google Scholar] [CrossRef]
  27. Cervera, M.; Grandon, N.; Rivera, M.; Besoain, F. Improving the Selection of IQF Raspberries in Processing Lines: A VR Approach for Training and Selecting Personnel. In Proceedings of the 2018 IEEE Biennial Congress of Argentina (ARGENCON), San Miguel de Tucuman, Argentina, 6–8 June 2018; pp. 1–7. [Google Scholar]
  28. Sakulin, S.; Alfimtsev, A. Multicriteria Decision Making in Tourism Industry Based on Visualization of Aggregation Operators. Appl. Syst. Innov. 2023, 6, 74. [Google Scholar] [CrossRef]
  29. dos Anjos, F.E.V.; Rocha, L.A.O.; Oliveira da Silva, D.; Pacheco, R. Impacts of the Application of Virtual and Augmented Reality on Teaching-Learning Processes in Engineering Courses. Int. J. Virtual Pers. Learn. Environ. 2022, 12, 40–46. [Google Scholar] [CrossRef]
  30. Chang, Y.-C.; Chang, K.-H.; Lu, M.-T. A Novel Network Simulation Software for Supply Chain Management Courses. Int. J. Ind. Eng. Theory Appl. Pract. 2018, 25, 230–251. [Google Scholar]
  31. Cudney, E.A.; Corns, S.M.; Farris, J.A.; Gent, S.; Grasman, S.E.; Guardiola, I.G. Enhancing Undergraduate Engineering Education of Lean Methods Using Simulation Learning Modules within a Virtual Environment. In Proceedings of the ASEE Annual Conference and Exposition, Conference Proceedings, Vancouver, BC, Canada, 26–29 June 2011. [Google Scholar]
  32. Gamlin, A.; Breedon, P.; Medjdoub, B. Immersive VR Deployment in a Lean Manufacturing Environment. In Proceedings of the 7th International Conference on Interactive Technologies and Games, Nottingham, UK, 16–17 October 2014; pp. 51–58. [Google Scholar]
  33. dos Anjos, F.E.V.; Rocha, L.A.O.; Silva, D.O.d.; Pacheco, R. Virtual and Augmented Reality Application in Production Engineering Teaching-Learning Processes. Production 2020, 30, e20190088. [Google Scholar] [CrossRef]
  34. Mansoory, M.S.; Azizi, S.M.; Mirhosseini, F.; Yousefi, D.; Moradpoor, H. A Study to Investigate the Effectiveness of the Application of VR Technology in Dental Education. BMC Med. Educ. 2022, 22, 457. [Google Scholar] [CrossRef] [PubMed]
  35. Huang, H.; Hwang, G.-J.; Jong, M.S.-Y. Technological Solutions for Promoting Employees’ Knowledge Levels and Practical Skills: An SVVR-Based Blended Learning Approach for Professional Training. Comput. Educ. 2022, 189, 104593. [Google Scholar] [CrossRef]
  36. Slezaka, R.J.; Keren, N.; Gilbert, S.B.; Harvey, M.E.; Ryan, S.J.; Wiley, A.J. Examining VR as a Platform for Developing Mental Models of Industrial Systems. J. Comput. Assist. Learn. 2023, 39, 113–124. [Google Scholar] [CrossRef]
  37. Liu, R.; Wang, L.; Koszalka, T.A.; Wan, K. Effects of Immersive VR Classrooms on Students’ Academic Achievement, Motivation and Cognitive Load in Science Lessons. J. Comput. Assist. Learn. 2022, 38, 1422–1433. [Google Scholar] [CrossRef]
  38. Murillo-Zamorano, L.R.; López-Sánchez, J.Á.; López-Rey, M.J.; Bueno-Muñoz, C. Gamification in Higher Education: The ECOn+ Star Battles. Comput. Educ. 2023, 194, 104699. [Google Scholar] [CrossRef]
  39. Pontes Pedrajas, A. Uso Didáctico de Un Laboratorio Virtual Para Favorecer La Progresión de Los Modelos Mentales de Los Estudiantes Sobre Circuitos de Corriente Eléctrica. Bordón. Rev. Pedagog. 2022, 74, 145–160. (In Spanish) [Google Scholar] [CrossRef]
  40. De Souza Castro, V.; Ronaldo Bezerra Oliveira, S. An Analysis of Application the Kahoot! Tool in a Gamified Approach to Face-to-Face and Emergency Remote Teaching and Learning of Software Engineering. In Proceedings of the 2022 IEEE Frontiers in Education Conference (FIE), Uppsala, Sweden, 8–11 October 2022; pp. 1–8. [Google Scholar]
  41. Ou Yang, F.-C.; Lai, H.-M.; Wang, Y.-W. Effect of Augmented Reality-Based Virtual Educational Robotics on Programming Students’ Enjoyment of Learning, Computational Thinking Skills, and Academic Achievement. Comput. Educ. 2023, 195, 104721. [Google Scholar] [CrossRef]
  42. Allen, A.J.; White, A.B.; Bacon, D.R.; Beck Dallaghan, G.L.; Jordan, S.G. Commentary on Ultrasound Instruction in Undergraduate Medical Education: Perspective from Two Students. Adv. Med. Educ. Pr. 2023, 14, 1–7. [Google Scholar] [CrossRef]
  43. Latinovic, T.; Sikman, L. Impact of COVID-19 on the Information Technology for Online Education, Education 4.0, Challenges and Solutions. In Advanced Technologies, Systems, and Applications VII; Springer: Cham, Switzerland, 2023; pp. 707–716. [Google Scholar]
  44. Vera, F.; García-Martínez, S. Creencias y Prácticas de Docentes Universitarios Respecto a La Integración de Tecnología Digital Para El Desarrollo de Competencias Genéricas. Rev. Colomb. Educ. 2022, 84, 1–16. [Google Scholar] [CrossRef]
  45. Ni, Z.H.; Huang, J.; Yang, D.P.; Wang, J. Nursing Students’experience of Flipped Classroom Combined with Problem-Based Learning in a Paediatric Nursing Course: A Qualitative Study. BMC Nurs. 2024, 23, 88. [Google Scholar] [CrossRef]
  46. Li, W.; Zhu, J.; Dang, P.; Wu, J.; Zhang, J.; Fu, L.; Zhu, Q. Immersive VR as a Tool to Improve Bridge Teaching Communication. Expert. Syst. Appl. 2023, 217, 119502. [Google Scholar] [CrossRef]
  47. Wong, K.-P.; Zhang, B.; Qin, J. Unlocking Potential: The Development and User-Friendly Evaluation of a VR Intervention for Attention-Deficit/Hyperactivity Disorder. Appl. Syst. Innov. 2023, 6, 110. [Google Scholar] [CrossRef]
  48. Musengimana, J.; Kampire, E.; Ntawiha, P. Rwandan Secondary School Students’ Attitudes in Learning Chemistry: Explored with Task-Based Instruction. Heliyon 2022, 8, e10509. [Google Scholar] [CrossRef] [PubMed]
  49. Ayasrah, F.T.M.; Alarabi, K.; Al Mansouri, M.; Fattah, H.A.A.; Al-Said, K. Enhancing Secondary School Students’ Attitudes toward Physics by Using Computer Simulations. Int. J. Data Netw. Sci. 2024, 8, 369–380. [Google Scholar] [CrossRef]
  50. Maraza-Quispe, B.; Traverso-Condori, L.C.; Torres-Gonzales, S.B.; Reyes-Arco, R.E.; Tinco-Túpac, S.T.; Reyes-Villalba, E.; Carpio-Ventura, J.R. Impact of the Use of Gamified Online Tools: A Study with Kahoot and Quizizz in the Educational Context. Int. J. Inf. Educ. Technol. 2024, 14, 132–140. [Google Scholar] [CrossRef]
  51. Raja, V.; Calvo, P. Augmented Reality: An Ecological Blend. Cogn. Syst. Res. 2017, 42, 58–72. [Google Scholar] [CrossRef]
  52. Dutta, R.; Mantri, A.; Singh, G. Evaluating System Usability of Mobile Augmented Reality Application for Teaching Karnaugh-Maps. Smart Learn. Environ. 2022, 9, 6. [Google Scholar] [CrossRef]
  53. Arpaia, P.; Duraccio, L.; Moccaldi, N.; Rossi, S. Wearable Brain–Computer Interface Instrumentation for Robot-Based Rehabilitation by Augmented Reality. IEEE Trans. Instrum. Meas. 2020, 69, 6362–6371. [Google Scholar] [CrossRef]
  54. Bogdanovic, Z.; Milicevic, A.; Stojanovic, D.; Labus, A.; Despotovic-Zrakic, M.; Radenkovic, B. Open Innovation Strategies in Engineering Education. In Proceedings of the 2023 IEEE 33rd International Conference on Microelectronics, MIEL 2023, Niš, Serbia, 16–18 October 2023; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2023. [Google Scholar]
  55. Jones, S.; Dawkins, S.; Julian, M. Understanding VR. In Understanding VR: Challenging Perspectives for Media Literacy and Education; Routledge: London, UK, 2022; pp. 106–108. ISBN 9780367337032. [Google Scholar]
  56. Lin, H.-T.; Tsai, H.-J.; Li, Y.-I.; Hu, W.-P. BMC Medical Education; BioMed Central Ltd.: London, UK, 2022. [Google Scholar]
  57. Su, Y.-S.; Cheng, H.-W.; Lai, C.-F. Study of VR Immersive Technology Enhanced Mathematics Geometry Learning. Front. Psychol. 2022, 13, 760418. [Google Scholar] [CrossRef] [PubMed]
  58. Hui, J.; Zhou, Y.; Oubibi, M.; Di, W.; Zhang, L.; Zhang, S. Research on Art Teaching Practice Supported by VR (VR) Technology in the Primary Schools. Sustainability 2022, 14, 1246. [Google Scholar] [CrossRef]
  59. Xu, K. Electrical Automation Teaching Based on VR VR Technology. In Proceedings of the 2021 2nd Asia-Pacific Conference on Image Processing, Electronics and Computers, Dalian, China, 14–16 April 2021; ACM: New York, NY, USA, 2021; pp. 925–928. [Google Scholar]
  60. Skarka, W.; Wylezoł, M.; Januszka, M.; Rzydzik, S.; Targosz, M. Improving the Ability of Future Engineers by Using Advanced Interactive 3D Techniques in Education. Adv. Transdiscipl. Eng. 2015, 2, 647–656. [Google Scholar] [CrossRef]
  61. Akbulut, A.; Catal, C.; Yıldız, B. On the Effectiveness of VR in the Education of Software Engineering. Comput. Appl. Eng. Educ. 2018, 26, 918–927. [Google Scholar] [CrossRef]
  62. Srinivasa, A.R.; Jha, R.; Ozkan, T.; Wang, Z. Virtual reality and its role in improving student knowledge, self-efficacy, and attitude in the materials testing laboratory. Int. J. Mech. Eng. Educ. 2021, 49, 382–409. [Google Scholar] [CrossRef]
  63. Inayat, I.; Inayat, Z.; Amin, R. Teaching and Learning Object-Oriented Analysis and Design with 3D Game. In Proceedings of the 2016 International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan, 19–21 December 2016; pp. 46–51. [Google Scholar] [CrossRef]
  64. Lowry, B.; Johnson, G.G.R.J.; Vergis, A. Merged VR Teaching of the Fundamentals of Laparoscopic Surgery: A Randomized Controlled Trial. Surg. Endosc. 2022, 36, 6368–6376. [Google Scholar] [CrossRef] [PubMed]
  65. Amprasi, E.; Vernadakis, N.; Zetou, E.; Antoniou, P. Effect of a Full Immersive VR Intervention on Selective Attention in Children. Int. J. Instr. 2022, 15, 565–582. [Google Scholar] [CrossRef]
  66. Laseinde, O.T.; Adejuyigbe, S.B.; Mpofu, K.; Campbell, H.M. Educating Tomorrows Engineers: Reinforcing Engineering Concepts Through VR (VR) Teaching Aid. In Proceedings of the 2015 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Singapore, 6–9 December 2015; IEEE: New York, NY, USA, 2015; pp. 1485–1489. [Google Scholar]
  67. Lee, J.S. Implementation and Evaluation of a VR Simulation: Intravenous Injection Training System. Int. J. Environ. Res. Public Health 2022, 19, 5439. [Google Scholar] [CrossRef]
  68. Schauer, A.M.K.; Fillingim, K.B.; Pavleszek, A.; Chen, M.; Fu, K. Comparing the Effect of Virtual and In-Person Instruction on Students’ Performance in a Design for Additive Manufacturing Learning Activity. Res. Eng. Des. 2022, 33, 385–394. [Google Scholar] [CrossRef] [PubMed]
  69. Cherner, Y.; Karim, A.; Khan, A.; Mullett, G. Web Based Interactive Virtual Laboratories For Electrical Engineering and Manufacturing Education. In Proceedings of the 2010 Annual Conference & Exposition Proceedings, Louisville, KY, USA, 20–23 June 2010; pp. 15.1358.1–15.1358.30. [Google Scholar]
  70. Chaturvedi, S.; Prabhakaran, R.; Yoon, J.; Abdel-Salam, T. Engineering Laboratory Instruction in Virtual Environment—“eLIVE”. Adv. Eng. Educ. 2011, 2, n4. Available online: https://files.eric.ed.gov/fulltext/EJ1076106.pdf (accessed on 13 March 2024).
  71. Grodotzki, J.; Ortelt, T.R.; Tekkaya, A.E. Remote and Virtual Labs for Engineering Education 4.0: Achievements of the ELLI Project at the TU Dortmund University. Procedia Manuf. 2018, 26, 1349–1360. [Google Scholar] [CrossRef]
  72. Chien, S.-Y.; Hwang, G.-J. A Question, Observation, and Organisation-Based SVVR Approach to Enhancing Students’ Presentation Performance, Classroom Engagement, and Technology Acceptance in a Cultural Course. Br. J. Educ. Technol. 2022, 53, 229–247. [Google Scholar] [CrossRef]
  73. Duckworth, L.; Sulbaran, T.; Strelzoff, A. Usability of a Collaborative VR Environment Earthwork Exercises. In Proceedings of the 2012 ASEE Annual Conference & Exposition Proceedings, ASEE Conferences, San Antonio, TX, USA, 10–13 June 2012; pp. 25.1405.1–25.1405.12. [Google Scholar]
  74. Sampaio, A.Z.; Martins, O.P. The Application of VR Technology in the Construction of Bridge: The Cantilever and Incremental Launching Methods. Available online: https://linkinghub.elsevier.com/retrieve/pii/S0926580513001830 (accessed on 5 May 2019).
  75. dos Anjos, F.E.V.; Rocha, L.A.O.; Pacheco, R.; da Silva, D.O. Teaching-Learning Strategies to Production Planning and Control Concepts: Application of Scenarios to Sequencing Production with VR Support. Int. J. Learn. Teach. Educ. Res. 2021, 20, 108–125. [Google Scholar] [CrossRef]
  76. Hair, J., Jr.; Page, M.; Brunsveld, N. Essentials of Business Research Methods; Routledge: New York, NY, USA, 2019. [Google Scholar] [CrossRef]
  77. Dresch, A.; Lacerda, D.P.; Antunes, J.A.V.J. Design Science Research; Springer International Publishing: Cham, Switzerland, 2015; ISBN 978-3-319-07373-6. [Google Scholar]
  78. Baierle, I.C.; Schaefer, J.L.; Sellitto, M.A.; Fava, L.P.; Furtado, J.C.; Nara, E.O.B. MOONA software for survey classification and evaluation of criteria to support decision-making for properties portfolio. Int. J. Strateg. Prop. Manag. 2020, 24, 226–236. [Google Scholar] [CrossRef]
  79. Inayat, I.; Amin, R.; Inayat, Z.; Salim, S.S. Effects of collaborative web based vocational education and training (VET) on learning outcomes. Comput. Educ. 2013, 68, 153–166. [Google Scholar] [CrossRef]
  80. Shirazi, A.; Behzadan, A.H. Content Delivery Using Augmented Reality to Enhance Students’ Performance in a Building Design and Assembly Project. Adv. Eng. Educ. 2015, 4, 1–24. [Google Scholar]
  81. Shirazi, A.; Behzadan, A.H. Design and assessment of a mobile augmented reality-based information delivery tool for construction and civil engineering curriculum. J. Prof. Issues Eng. Educ. Pract. 2014, 141, 04014012. [Google Scholar] [CrossRef]
  82. Borchardt, M.; Pereira, G.; Milan, G.; Scavarda, A.; Nogueira, E.; Poltosi, L. Industry 5.0 beyond technology: An analysis through the lens of business and operations management literature. Organizacija 2022, 55, 305–321. [Google Scholar] [CrossRef]
  83. Adami, V.; Júnior, J.; Sellitto, M. Regional industrial policy in the wind energy sector: The case of the State of Rio Grande do Sul, Brazil. Energy Policy 2017, 111, 18–27. [Google Scholar] [CrossRef]
Asi 07 00050 g001
Figure 1. Stages of development and application of the VR artifact.
Figure 1. Stages of development and application of the VR artifact.
Asi 07 00050 g001
Asi 07 00050 g002
Figure 2. Images generated by the artifact in VR.
Figure 2. Images generated by the artifact in VR.
Asi 07 00050 g002
Asi 07 00050 g003
Figure 3. Satisfaction, engagement, and attractiveness indices of using VR for EG on a scale from 1 to 7.
Figure 3. Satisfaction, engagement, and attractiveness indices of using VR for EG on a scale from 1 to 7.
Asi 07 00050 g003
Asi 07 00050 g004
Figure 4. Keywords reported in qualitative questions by participants in the experimental group.
Figure 4. Keywords reported in qualitative questions by participants in the experimental group.
Asi 07 00050 g004
Table 1. Statistical tests applied to CG and EG to evaluate the experiment.
Table 1. Statistical tests applied to CG and EG to evaluate the experiment.
EventStatistical Test
Assessment of prior knowledge- Shapiro–Wilk to assess data normality
- Independent sample t-test to assess whether the means obtained by the groups are similar
Assessment of learning improvement- Shapiro–Wilk to assess data normality
- Paired difference t-test to assess whether the mean obtained by the groups generated learning
Test to compare knowledge between groups- Shapiro–Wilk to assess data normality
- Independent sample t-test to assess whether the means obtained by the groups are similar
The test compares the level of knowledge retention- Shapiro–Wilk to assess data normality
- Independent sample t-test to assess whether the means obtained by the groups are similar
- Assessment of the proportion of knowledge retained by each group
Table 2. Characterization of the groups that participated in the experiment.
Table 2. Characterization of the groups that participated in the experiment.
GroupsControlExperimental
GenderMale1925
Female159
The average age of the participants28.9 years28.7 years
Previously familiar with the topic of production sequencing67
Table 3. Result of the student t-test in the prior knowledge test.
Table 3. Result of the student t-test in the prior knowledge test.
GroupNAverage95% CI Lower95% CI UppertSig. (2 Ends)Average DifferenceDifference Standard Error
ResultsCG343.50913.09713.9212−1.3110.1940.52220.47497
EG343.93653.41714.4556
Table 4. The t-test result was used to assess learning in the CG.
Table 4. The t-test result was used to assess learning in the CG.
NAverageAverage DifferenceStandard DeviationDifference Standard ErrortSig. (2 Ends)
Pre-test343.5091−1.11380.098040.1681−6.6250.000000155
Post-Test344.6229
Table 5. The t-test result is used to evaluate the learning of the EG.
Table 5. The t-test result is used to evaluate the learning of the EG.
NAverageAverage DifferenceStandard DeviationDifference Standard ErrortSig. (2 Ends)
Pre-test343.9365−2.03821.52750.2618−7.7800.0000000058
Post-Test345.9747
Table 6. Learning similarity between groups.
Table 6. Learning similarity between groups.
GroupNAverage95% CI Lower95% CI UppertSig. (2 Ends)Average DifferenceDifference Standard Error
ResultsCG344.62294.19185.0540−4.7410.000012−1.351760.28515
EG345.97475.58656.3629
Table 7. Knowledge retention levels after three weeks of experiments.
Table 7. Knowledge retention levels after three weeks of experiments.
GroupNAverage95% CI Lower95% CI UppertSig. (2 Ends)Average DifferenceDifference Standard Error
ResultsCG344.39883.97634.8212−3.10.0028−0.885310.28557
EG345.28414.88525.6829
Table 8. Comparison between the results of knowledge acquisition and knowledge retention.
Table 8. Comparison between the results of knowledge acquisition and knowledge retention.
AssessmentGroupResults
Test of Knowledge AcquiredControl4.6229
Experimental5.9747
Knowledge Retention TestControl4.3988
Experimental5.2841
Table 9. Descriptive statistics of responses on EG’s perception of using VR.
Table 9. Descriptive statistics of responses on EG’s perception of using VR.
Descriptive Statistics
QuestionMinimumMaximumAverageStandard DeviationSkewness
Q1576.530.615−0.950
Q2576.530.615−0.950
Q3576.410.701−0.786
Q4576.790.479−2.360
Q5476.470.748−1.509
Q6476.180.936−0.843
Q7576.350.774−0.724
Q8576.590.557−0.929
Q9476.710.629−2.795
Q10576.740.511−1.816
Q11576.740.511−1.816
Q12676.710.462−0.946
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Anjos, F.E.V.d.; Martins, A.d.O.; Rodrigues, G.S.; Sellitto, M.A.; Silva, D.O.d. Boosting Engineering Education with Virtual Reality: An Experiment to Enhance Student Knowledge Retention. Appl. Syst. Innov. 2024, 7, 50. https://doi.org/10.3390/asi7030050

AMA Style

Anjos FEVd, Martins AdO, Rodrigues GS, Sellitto MA, Silva DOd. Boosting Engineering Education with Virtual Reality: An Experiment to Enhance Student Knowledge Retention. Applied System Innovation. 2024; 7(3):50. https://doi.org/10.3390/asi7030050

Chicago/Turabian Style

Anjos, Fernando Elemar Vicente dos, Adriano de Oliveira Martins, Gislene Salim Rodrigues, Miguel Afonso Sellitto, and Debora Oliveira da Silva. 2024. "Boosting Engineering Education with Virtual Reality: An Experiment to Enhance Student Knowledge Retention" Applied System Innovation 7, no. 3: 50. https://doi.org/10.3390/asi7030050

Article Metrics

Back to TopTop