1. Introduction
The rapid development of information technology leads educators to apply advanced technologies, such as AR/VR, into competency-based learning [
1,
2,
3,
4,
5,
6]. In the 21st century, students need to acquire the foundation of learning processes, such as problem-solving, critical thinking, creativity, innovation, metacognition, communication, etc. to survive in the modern world [
7]. The new K-12 education curriculum guidelines implemented in the 2019 academic year in Taiwan take the key competencies as the backbone of the coherence and integration of basic education courses and thereby guide the development of curriculum and teaching in various fields. Learning materials have a great influence on self-learning for students; therefore, how to implement teaching content that conforms to the idea of competency-based instructions is one of the most important issues.
Problem-solving is the foundation of learning processes and is a cognitive process that requires the ability to define and identify a problem, find a solution, and implement the solution in relatively new situations [
8]. Problem solving is often challenging for students because they do not understand the problem-solving process. In [
8], Yu, K.C. et al. reveal context-based learning may effectively enable students to establish a complete problem-solving process. Many studies reveal virtual reality technology has demonstrated positive educational outcomes can engage and motivate users and potentially assist cognitive processing and knowledge transfer in some subjects [
2,
3,
4,
5,
6]. Within the paper [
6], authors conclude for high school students, the use of VR for subjects associated with chemistry was positive in terms of motivation and confidence since knowledge is required to handle these technologies. Currently, problem-solving has been identified as an important educational skill for primary school students [
9]. VR technology enables the possibility of making abstract concepts concrete while increasing students’ engagement and motivation in the learning process [
10,
11]. In the paper [
3] proposed by Ronaghi, M.H., it shows VR-based training courses, as well as holding traditional training courses, has been effective on sustainable behavior and has been effective in learning sustainable behavior.
Sustainability is one of the global challenges, and awareness of environmental protection is even more urgent in education. Environmental sustainability covers a wide range of issues from regional locations to the globe. Global issues include concerns about greenhouse gas reduction, climate change, and renewable energy, while regional issues are soil erosion, water management, soil quality, and air and water pollution. Thus, understanding the interaction between human and environmental ecology is one of the important topics on environmental education. In the study, a competency-based VR learning course on environmental education is implemented. One-group pre- and post-test designs were adopted to evaluate the effects of the VR learning course on students’ behavioral intentions for problem solving.
1.1. Interactive Design in VR Content
Interactive design refers to a mechanism by which designers interact with products or services. Human-computer interaction design based on user experience should consider the user’s background, user experience, and feelings during operations to design a product that gives a satisfactory user experience.
Using human intuition and experience can effectively reduce users’ thinking and ego depletion. A user can understand and use a design immediately without consciously thinking about how to do it. Thus, intuitive design can help users to immerse themselves in the experience of content without the burden for actions in the virtual environment [
12]. An overview of some key terms relevant to interaction design in virtual reality were popularized in the book [
13]. According to Nielsen [
14], ten usability heuristics for digital interface design are proposed to create better experiences:
- (1)
Visibility of system status;
- (2)
Match between the system and the real world;
- (3)
User control and freedom;
- (4)
Consistency and standards;
- (5)
Error prevention;
- (6)
Recognition rather than recall;
- (7)
Flexibility and efficiency of use;
- (8)
Aesthetic and minimalist design;
- (9)
Help users recognize, diagnose, and recover from errors;
- (10)
Help and documentation.
We follow these guidelines to design VR interactive content for better user experiences. In this work, we make efforts to follow these guidelines. In particular, the two guidelines of (1). visibility of system status and (2). match between the system and the real world have implications for both interaction design and scenarios.
1.2. Design of Competency-Based Learning Materials
The new K-12 education curriculum guidelines in Taiwan take the key competencies as the backbone of the coherence and integration of basic education courses and thereby guide the development of curriculum and teaching in various fields. Different from many traditional learning methods using summative testing, competency-based learning focuses on the idea that students may only identify and demonstrate the mastery of individual learning outcomes [
15,
16]. Learning materials have a great influence on self-learning for students; thus, how to implement teaching content that conforms to the idea of competency-based instruction is one of the most important issues.
A framework to design the experience for sustainable behaviors on exploring the use of virtual reality was proposed by Giulia et al. [
4]. It summarized the VR experiences could be explorations, games, simulations, and so on. Additionally, the objectives of VR experiences are related to the three behavioral dimensions, including emotional, rational, and practical. The use of VR experiences to support design teaching–learning activities can be related to the practical dimension. These designed solutions, presented and tested in VR, can explore strategies involving the rational and emotional ones.
As described in the paper [
5], different teaching methods affect competency development and effectiveness in different ways. Novel VR tools require appropriate teaching methods to foster students’ abilities to develop competencies. Additionally, in [
17] it also proposes approaches of modeling and implementations to make the design of competency-based educational content easier and serve as the recommendation of a set of resources concerning a skill to be acquired.
2. Aim
In the study, it proposes a competency-based VR learning material on environmental education, which emphasizes students’ exploration and recording about surrounding objects and provides formative assessments in the final exploration phase. Further, a VR learning course using the competency-based learning material is implemented and then evaluates whether the effect on students’ behavioral intentions of problem solving has been improved in the junior high school.
3. Materials and Methods
3.1. Hardware Components
The user application is based on Oculus Quest 2, which has a better price–performance ratio, easy maintenance, and classroom preparation. Users can query their learning records from the web server by web browser with the user ID and the school ID. These learning records can also be used as materials for students to explain and elaborate their understanding of concepts in further teaching activities. The details are described later.
3.2. Software Components
Good and flexible software development tools are helpful for the efficiency of system implementation and integration. In the development of the VR interactive learning materials, related development tools include the Unity 3D engine Ver. 2019.1 LTS [
18], Autodesk Maya [
19], MRTK v2.7 [
20], Oculus Integration SDK Package [
21], etc. Additionally, the XAMPP [
22], an easy to install Apache distribution, extends web information server with integrated services, including PHP development environment, MySQL Database services, etc.
3.3. VR Content Design
The ADDIE model [
23], including analysis, design, development, implementation, and evaluation steps, has been used as a framework for designing and developing educational and training programs for many years. The clearly defined steps help educators create effective teaching–learning materials.
In the analysis phase of the work, the target group is the students in the junior high school. There are petrochemical factories, orchards, pig farms, and other fields near their living environment. The purpose of the learning materials is to increase local students’ interests in understanding the classification of pollution sources and understanding the interaction between humans and the living environment.
In the content design phase, the types of environmental pollution sources are classified into five categories: eutrophication, heavy metal pollution, dioxin pollution, solid waste, and volatile organic compounds, VOCs, in the VR interactive material, as shown in
Figure 1. Students must judge and classify the pollution sources collected by themselves into these classification kinds to complete the task, and the interactive learning system will feedback appropriate information in time. Additionally, it also supports students to query their learning records by web browsers and further discuss and share their learning results in the extended teaching–learning activities. It presents an example of the student’s learning records in
Figure 2.
The system architecture of the VR content is illustrated, as shown
Figure 3. The Unity Player application is developed for Oculus Quest 2, which is an all-in-one VR system and gains a better price–performance ratio in the current period. The database server is mainly used to maintain users’ learning records, including users’ test scores, users’ taking pictures in each scene, and recording times. Students using the Unity Player applications can upload their learning results to the cloud service.
3.4. Functional Architecture
In
Figure 4 derived from
Figure 1, it explains the VR learning objects in each different scene will be constrained to its preset pollution types. For example, in the house scene, it focuses on water pollution, air pollution, and solid waste. In the interactive design of VR learning material, we use the familiar camera element as a collection tool for users to explore in the virtual world. Using familiar visual elements can reduce burdens of human cognition for functionalities, but it is undeniable there are still many hardware interface design limitations that need to be improved, such as VR handle’s operations and so on. For unfamiliar user interfaces, users need a longer adaptation period.
4. Experimental Design and Data Description
4.1. Background and Experimental Design
One-group pre-test–post-test design is adopted in this study, and the participants’ questionnaire responses are analyzed. In the questionnaire, the main purpose is to evaluate students’ perceptions of course identification, analyze differences of the student ability scale, identify students’ intentions for problem-solving, and assess the students’ impacts on the VR teaching-learning activities. The process of the experimental test for VR experience is shown in
Figure 5.
In this study, the experiment was conducted in September 2022, and the subjects were 29 first-year students at a junior high school in Kaohsiung City, Taiwan. There were 10 girls and 19 boys in this experimental class. The experimental activity of VR experience was held in school club classes. Before the VR experience activity, the teacher gave a brief activity description. Each student had approximately 25 min of VR experience time. Before and after the experience, participants filled out the pre-test and post-test questionnaires within 10 min each.
4.2. Experimental Method
The VR learning content consists of five scenes, including petrochemical factory, orchard, house, pig farm, and the laboratory. The laboratory scene is created for assessment after the learning experiences of the students. Other built scenes are related to the local students’ living environments. The scripts of these scenarios lead students to encounter the related environmental pollution problems in their lives.
As we know, the impact of pollution sources on the environment is complex and not classified into a single type of pollution. Therefore, we set different pollution weights for specified objects as references for the game exploration scoring. For example, as shown in
Figure 6, each of the pollution objects in different scenes are assigned with different weighted values for five classified pollution types, including eutrophication, heavy metal pollution, dioxin, acid rain, and solid waste.
A total of 29 first-year junior high school students participated in the VR learning experience. Students were divided into two groups. Due to equipment limitations, the two groups were tested at different time periods. First, the teacher explained the content of the VR experiment and conducted a pre-test of the survey. Next, students had turns with the VR experiences. Finally, a post-test questionnaire was conducted. For each pre- and post-test survey, students needed to complete the questionnaire in 10 min.
4.3. Results Based on Questionnaires
In the experimental test, a total of 29 first-year junior high school students participated in the pre- and post-tests. For each pre- and post-test survey, students completed the questionnaire surveys in 10 min. The test results are described as follows.
4.3.1. Pre-Test and Post-Test Data Overview
VR Course Identity
One hundred percent of students agreed the “VR Learning Course” is helpful for personal learning in the pre- and post-tests, as shown in
Figure 7. On a scale of 1 to 4, the average scores for the pre-test and post-test in the class were 3.38 and 3.84, respectively. The details are presented in
Table 1.
Intentions for Problem-Solving
It was shown 100% of students had good intentions for problem-solving in the pre-test and post-test, as shown in
Figure 8. On a scale of 1 to 5, the average scores for the pre-test and post-test in the class were 4.12 and 4.59, respectively. The detail are presented in
Table 2.
4.3.2. Data Description
Total post-test results of the VR course identity and problem-solving intentions were more highly significant (
p < 0.05) than the pre-test results in
Table 1 and
Table 2. The means and standard deviations were calculated for further discussions.
Part 1: VR Course Identity
Item values in the questionnaire represent the degrees of VR course identification. Item values < 1.74 are considered as disagree; values between 1.75 and 2.49 as a little bit disagree; values between 2.5 and 3.24 as a little bit agree and scores more than 3.25 as agree.
Based on the analysis results, as shown in
Table 1, the reliability of the survey for both student’s pre- and post-test responses for VR course identity ≥ 0.9 (number of questions = 13); thus, it can be concluded the survey results developed for VR course identity has excellent internal consistency and high reliability. Additionally, 100% of the students in the class agree the “VR learning course” is helpful for self-learning. The average scores of the pre- and post-tests in the class were 3.38 and 3.84 on a scale of 1 to 4, respectively.
Part 2: Students’ Behavioral Intentions Regarding Problem Solving
Item values in the questionnaire represent students’ intentions regarding their problem-solving. Item values < 1.99 are considered as “very bad”; values between 2.0 and 2.99 as “not good”; values between 3.0 and 3.99 as “good”; and scores more than 4 as “very good”.
Based on the analysis results, as shown in
Table 2, the reliability of the survey for both pre- and post-test for students’ behavioral intentions regarding problem solving ≥ 0.9 (number of questions = 28); thus, it can be concluded the survey results developed for students’ behavioral intentions regarding problem solving has excellent internal consistency and high reliability. Additionally, the result shows 100% of the class has good or very good problem-solving intentions. On a scale of 1 to 5, the average values of pre- and post-tests in the class were 4.12 and 4.59, respectively.
4.4. Impact on Learning Outcomes
In
Table 3, the
p-value is much less than 0.05, which proves it has a significant effect on the course identification after the implementation of the VR learning course.
In
Table 4, the
p-value is much less than 0.05, which proves it has a significant effect on students’ behavioral intentions for problem solving after the implementation of the VR learning course.
5. Discussions
Many studies [
1,
2,
3,
4,
5,
6] reveal virtual reality technology has demonstrated positive educational outcomes. In VR learning courses, students’ use of VR teaching materials has a positive impact on their learning outcomes. In this case report, the aim is not only to develop competency-based VR learning materials but also to evaluate students’ VR course identity and problem-solving behavioral intentions. First, in the interactive design of competency-based VR learning materials, we followed human intuitions and experiences to reduce users’ thinking and ego depletion for a better user experience. In the implementation of interactive VR learning material, we use the familiar camera element as a collection tool for users to explore in the virtual world. Using familiar visual elements can reduce burdens of human cognition for functionalities, but it is undeniable that there are still many hardware interface design limitations that need to be improved, such as VR handle’s operations and so on. For unfamiliar user interfaces, most users need a longer adaptation period. Secondly, using the learning records, teachers can encourage students to explain and elaborate their understanding of concepts in new teaching–learning activities.
Additionally, we assessed the impact of the VR learning course on students’ VR course identity and behavioral intentions for problem-solving using the proposed competency-based VR learning materials. An increase in learning intent will have a positive impact on learning outcomes. The results of the questionnaire reveal 100% of the first-year students at a junior high school all agree the “VR learning course” is helpful for personal learning. Additionally, they have a high course identity with VR learning courses, and their behavioral intentions for problem-solving have been improved significantly.
6. Conclusions
Different teaching methods affect competency development in different ways, and different types of learning materials have a great influence on students’ self-learning [
5]. In the VR competency-based learning experiment, we attempted to improve the VR course identity and problem-solving intentions and stimulate the learning of problem-solving abilities through VR experience.
In this case report, the interactive competency-based VR learning materials are implemented following human intuitions and experiences to reduce users’ thinking and ego depletion. It also analyzes and discusses the learning impact results before and after the VR experience in junior high school. In the VR experimental results, it was found the first-year students at a junior high school have a high course identity with VR learning courses, and their behavioral intentions for problem-solving improved significantly. It is expected through the VR content, students could learn to observe and pay much attention to their surroundings to understand the interaction between humans and the environment. Next, we will conduct a quasi-experimental study using the proposed competency-based VR materials to further analyze participants’ responses with VR course identity and learning intentions of problem-solving abilities.
The VR immersive experience can effectively enhance motivation and engagement of students’ learning. Different from traditional teaching ways, the VR courses require the development of their own teaching methods and learning material designs. However, the evaluation of learning effectiveness for competency-based VR course plans have always been one of the important research topics for educations. Critical thinking and problem solving are some of the most important development skills for 21st century learning outcomes. Educators should think about how to cultivate students to develop these skills in the learning processes. Thus, it is becoming increasingly important to develop competency-based lesson plans.
Author Contributions
All authors contributed to the study, conceptualization, and methodology of this work; the software was developed by F.-J.L.; the validation and the formal analysis, F.-J.L. and C.-C.Y.; the investigation and resources, F.-J.L. and C.-C.Y.; data curation, C.-C.Y.; writing—original draft preparation, F.-J.L.; writing—review and editing, F.-J.L.; project administration, F.-J.L.; funding acquisition, F.-J.L. All authors have read and agreed to the published version of the manuscript.
Funding
This research was supported by the 2020 Plan of VAR Teaching Materials Development and Promotion of the Ministry of Education of Taiwan.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by theEthics Committee of National Cheng Kung University (protocol code NCKU HREC-E-111-277-2 and date of approval 21 June 2022).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data presented in this study are available on request from the corresponding author.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Araiza-Alba, P.; Keane, T.; Chen, W.S.; Kaufman, J. Immersive virtual reality as a tool to learn problem-solving skills. Comput. Educ. 2021, 164, 104121. [Google Scholar] [CrossRef]
- Xin, Y. Influence of Learning Engagement on Learning Effect under a Virtual Reality (VR) Environment. Int. J. Emerg. Technol. Learn. 2022, 17, 226–237. [Google Scholar] [CrossRef]
- Ronaghi, M.H. The effect of virtual reality technology and education on sustainable behavior: A comparative quasi-experimental study. Interact. Technol. Smart Educ. 2022. [Google Scholar] [CrossRef]
- Scurati, G.W.; Bertoni, M.; Graziosi, S.; Ferrise, F. Exploring the Use of Virtual Reality to Support Environmentally Sustainable Behavior: A Framework to Design Experiences. Sustainability 2021, 13, 943. [Google Scholar] [CrossRef]
- Hee Lee, J.; Shvetsova, O.A. The Impact of VR Application on Student’s Competency Development: A Comparative Study of Regular and VR Engineering Classes with Similar Competency Scopes. Sustainability 2019, 11, 2221. [Google Scholar] [CrossRef] [Green Version]
- Santos Garduño, H.A.; Esparza Martínez, M.I.; Portuguez Castro, M. Impact of Virtual Reality on Student Motivation in a High School Science Course. Appl. Sci. 2021, 11, 9516. [Google Scholar] [CrossRef]
- Scott, C.L. The Futures of learning 2: What Kind of Learning for the 21st Century? In Education, Research and Foresight: Working Papers; UNESCO: Paris, France, 2015; 14p, Available online: https://unesdoc.unesco.org/ark:/48223/pf0000242996 (accessed on 6 October 2022).
- Yu, K.C.; Fan, S.C.; Lin, K.Y. Enhancing students’problem-solving skills through context-based learning. Int. J. Sci. Math. Educ. 2015, 13, 1377–1401. [Google Scholar] [CrossRef]
- Cornoldi, C.; Carretti, B.; Drusi, S.; Tencati, C. Improving problem solving in primary school students: The effect of a training programme focusing on metacognition and working memory. Br. J. Educ. Psychol. 2015, 85, 424–439. [Google Scholar] [CrossRef] [PubMed]
- Araiza-Alba, P.; Keane, T.; Matthews, B.; Simpson, K.; Strugnell, G.; Chen, W.S.; Kaufman, J. The potential of 360-degree virtual reality videos to teach water-safety skills to children. Comput. Educ. 2021, 163, 104096. [Google Scholar] [CrossRef]
- Persky, S.; McBride, C.M. Immersive virtual environment technology: A promising tool for future social and behavioral genomics research and practice. Health Commun. 2009, 24, 677–682. [Google Scholar] [CrossRef] [PubMed]
- Andrews, R. Understanding Intuitive Design. Senior Capstone Projects 844. 2019. Available online: https://digitalwindow.vassar.edu/senior_capstone/844 (accessed on 6 October 2022).
- Norman, D. Design of Everyday Things: Revised and Expanded Edition 2013; Basic Books: New York, NY, USA, 2013. [Google Scholar]
- Nielsen, J. Ten Usability Heuristics 2005. Available online: https://www.nngroup.com/articles/ten-usability-heuristics/ (accessed on 6 October 2022).
- EdSurge News. New to Competency-Based Learning? Here’re Five Ways to Assess It; EdSurge: Burlingame, CA, USA, 2017. [Google Scholar]
- From Formative Assessment to Tracking Student Mastery: The Road to Competency-Based Instruction. 16 July 2015. Available online: https://www.gettingsmart.com (accessed on 9 March 2022).
- Sedra, M.; Bennani, S. Competency Based Approach: Modeling and Implementation. Int. J. Emerg. Technol. Learn. 2020, 15, 230–238. [Google Scholar] [CrossRef]
- Unity 3D Engine. Available online: https://unity.com/ (accessed on 6 October 2022).
- Autodesk Maya. Available online: https://www.autodesk.com.tw/products/maya/ (accessed on 6 October 2022).
- Microsoft Mixed Reality Toolkit for Unity. Available online: https://docs.microsoft.com/zh-tw/windows/mixed-reality/mrtk-unity/?view=mrtkunity-2021-05 (accessed on 6 October 2022).
- Oculus Integration SDK. Available online: https://developer.oculus.com/downloads/package/unity-integration/ (accessed on 6 October 2022).
- XAMPP. Available online: https://www.apachefriends.org/zh_tw/index.html (accessed on 6 October 2022).
- Molenda, M. In search of the elusive ADDIE model. Perform. Improv. 2003, 42, 34–36. [Google Scholar] [CrossRef]
| Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).