The Effects of an Immersive Virtual-Reality-Based 3D Modeling Approach on the Creativity and Problem-Solving Tendency of Elementary School Students

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper discusses the idea of design issues related to construction of effective VR tools for education. The underlying idea is to provide usability study for immersive teaching environment. Professionally prepared questionnaire targeted group of students that were representative for primary school level. There are a number of already available methods of  virtual communication tools built on particularly attractive feature of VR the enables student to fully submerge into a virtual environment. Moreover over 20 years ago studies carried by Baukal and Ausburn showed that the retention rates for VR learning reached over  75 % comparing to 10% for reading and less than 50% for lecture style learning (Baukal C.E., Ausburn F.B., Ausburn L.J.: A Proposed Multimedia Cone of Abstraction: Updating a Classic Instructional Design Theory. Journal Of Educational Technology, 9(4), pp.15-24 (2013)

 

The originality of the authors’ approach is in their contribution to evaluation of randomly selected students from each of the experimental (IVR-3D) and control (CVR-3D) groups. First group of students participated in the immersive teaching process with the use of Virtual Reality. The proposed metaverse teaching space and curriculum process is based on the social factors as one of the most valuable evaluation criteria of  user experience design for human creativity and spatial intelligence. VR is characterized by distinctive feature of being totally ‘immersive’, where user is immersed in the virtual scenery that is shared, persistent and accessible. Although participation in virtual simulation may cause potential problems when the system is applied to the meaningless or abstract images, it works quite well with images that are recognized and are familiar to the user (hence the choice of recognized VR “backgrounds”). According to the paper students looked primarily for ‘fun’ and ‘experience’, that is a highly subjective feature closely related to quality of interactions and is extremely hard to obtain in purely  VR (hence the popularity of Cinematic VR or 360 Videos that bring the viewer to filmed real-life situations).   

 

Author Response

Thank you for your suggestion, in response to the mention of the VR learning retention issue you mentioned we think this article is very valuable and has been added to the literature review on 2.3 Immersive Virtual Reality in Education.

The additions are as follows:

Meanwhile, one study confirmed that virtual reality has a retention rate of more than 75% of learning outcomes, far exceeding the retention rate of traditional reading and lecture-based learning(Baukal et al., 2013).

Reviewer 2 Report

Comments and Suggestions for Authors

1. The 3D modeling course lasted only 7 weeks, with 45-minute weekly classes. This relatively short duration may not be sufficient to fully capture the long-term effects of the IVR-based 3D modeling approach.

2. The paper mentions that both groups received a 40-minute introduction to the basic 3D modeling activities and completed a pretest questionnaire before the intervention. The order of these activities could have influenced the students' subsequent performance and attitudes.

3. The paper does not mention whether the students' participation in the 3D modeling course was voluntary or mandatory. If it was voluntary, there could be self-selection bias, where students with a higher interest or aptitude in 3D modeling may have chosen to participate.

4. The study was conducted in a controlled classroom setting, which may not fully reflect the real-world application of the IVR-based 3D modeling approach. Evaluating the method in more natural, everyday learning environments could improve the ecological validity of the findings.

5. The study used self-report questionnaires to assess creative thinking, problem-solving skills, and cognitive load. Self-report measures can be subject to biases and may not fully capture the constructs of interest. The paper does not provide detailed information on the validation of the Chinese versions of the assessment scales used, which could raise questions about their reliability and validity in the current context.

6. The paper only reports on the interviews conducted with the experimental group students. Obtaining feedback from the control group students could provide a more balanced perspective on the strengths and limitations of the two approaches. The interview questions were not provided, making it difficult to assess the depth and breadth of the qualitative data collected.

7. The paper does not provide any information on the specific behaviors and interactions of the students during the 3D modeling activities, such as time spent on tasks, number of iterations, or types of errors made. Collecting and analyzing such process data could help elucidate the underlying mechanisms and factors contributing to the observed outcomes.

8. The pre-test scores for creative thinking and problem-solving tendencies were relatively high, with mean scores around 2.0 (out of 3.0) and 3.6 (out of 5.0) respectively. This suggests the possibility of ceiling effects, where there was limited room for improvement, particularly for the control group.

9. The study only examined the immediate post-intervention effects. It did not include any follow-up assessments to determine if the benefits of the IVR-based 3D modeling approach were sustained over time. Potential confounding factors: The study did not control for prior 3D modeling experience, spatial ability, or general academic performance, which could have influenced the observed outcomes.

10. The qualitative interview data was only collected from the experimental group, providing limited insight into the perceptions and experiences of the control group students.

Author Response

1.We agree with your view that the duration of the experiment this time may not be sufficient to fully assess the long-term development of students in creative thinking and problem-solving skills, due to some constraints of the current situation. In future research, we plan to extend the duration of the course to more accurately assess the impact of the IVR teaching method on students' long-term learning outcomes. We will include these limitations and prospects in the conclusion section.

2.Thank you for your attention; it is essential that both groups of students have an understanding of the activity. Regarding the issue you pointed out about the sequence of the introduction and pre-test possibly influencing the students' subsequent performance and attitudes, we indeed considered this in our research. In the design of this study, all participants received a uniform 40-minute introduction to basic 3D modeling activities before starting the intervention learning activities, followed by a pre-test questionnaire. This design was implemented to ensure that both groups of students had the same level of foundational knowledge and preliminary understanding of 3D modeling at the start of the experiment, thus minimizing the potential impact of prior knowledge on learning outcomes.

3.Thank you for pointing out an important detail that was missing from our research report. The issue you mentioned regarding whether student participation in the 3D modeling course was voluntary or mandatory could impact the broad applicability and interpretation of the research findings.In our study, the students' participation was incorporated as part of the school curriculum, therefore it can be considered mandatory. This means that all students, regardless of their personal interest or previous skill levels, were involved in the course and related research activities.

The revised information is below:

Seventy-seven sixth-grade students from an elementary school in China participated in this 3D modeling course. The students' participation was incorporated as part of the school curriculum. The course took 7 weeks, with a 45-minute class each week, and was designed to teach students the fundamentals of 3D modeling and to develop their creative thinking and problem-solving skills. Since three students did not complete the program, only 38 male and 36 female questionnaires were collected, totaling 74, and the average age of the students was 11.37 years old. All the students had no previous experience with 3D modeling or immersive virtual reality. The study was approved by the Research Ethics Committee of the affiliated institution. The students were also informed that they could stop participating in the experiment without any consequences if they experienced feelings such as nausea and dizziness during the virtual reality activities.

4.Thank you for your comments on the setup of our research environment. While conducting research in a controlled experimental environment can help us more precisely control variables, thus accurately assessing the educational effects of IVR technology, this setup may not fully capture the true reactions and behaviors of students in a natural learning environment.

To address this issue and enhance the ecological validity of our research, we plan to expand the sample population and diversify participant backgrounds in the future research section of our conclusions. Additionally, longer-term observations and a variety of data recording methods will also help in assessing students' responses and behaviors in natural environments. The modifications are placed in the shortcomings and perspectives of the study, see the conclusions section.

5.Thank you very much for your concern regarding the assessment methods used in our study. Indeed, while self-report questionnaires are an effective tool for data collection, they may be subject to biases such as social desirability and self-perception biases. We plan to incorporate qualitative data and objective measurement methods in future research to enhance the scientific validity of our conclusions. See the Conclusion section of the article for modifications. Additionally, regarding the reliability and validity of the Chinese version of the questionnaire, the questionnaires used in this study have been revised and utilized by Chinese scholars (Hwang et al., 2013; Lai & Hwang, 2014; Lin & Wang, 1994), and these questionnaires have achieved acceptable levels of reliability and validity in these studies.

6.Thank you for your suggestion. In this study, interviews with the control group are elaborated in Table 5. However, the placement may not be very clear. We will make further markings in the method steps section to clarify this.

7.Thank you very much for your detailed review and suggestions. Your point about the lack of data on specific behaviors and interactions of students during the 3D modeling activities is very apt. Indeed, such process data are crucial for deeply understanding the dynamic changes during the learning process and their impact on learning outcomes. Due to the immaturity of immersive virtual reality conditions at the time of the experiment, we encountered some technical difficulties in recording process data. We will address this limitation and provide future directions in the deficiencies and outlook section of our research.

8.Thank you for pointing out the issue of a possible ceiling effect due to high scores in creative thinking and problem-solving abilities. This phenomenon could indeed affect our assessment of the effectiveness of the teaching methods, particularly if the pre-test scores are already relatively high, as this could limit the scope for measuring significant improvements. This may be related to the potential drawbacks of using self-report questionnaires. In future research, we will incorporate qualitative data and objective measurement methods to enhance the scientific validity of our conclusions.

9.Thank you for pointing out the issue that our study did not include a long-term follow-up assessment. This indeed represents a significant limitation in our current research design. In future studies, we plan to extend the duration of the course to more accurately assess the impact of IVR teaching methods on students' long-term learning outcomes. As for potential confounding factors, we conducted a simple survey of participants before the experiment started. Regarding their experience with 3D modeling and immersive virtual reality, all students lacked long-term or frequent usage, which we did not fully consider. The potential impact of students' spatial abilities on the observational results is something we overlooked. Thanks for the suggestion, we'll point that out in the study limitations. We think increasing the diversity of participants in future studies may help to address this issue.

10.Thank you for your suggestion. In this study, interviews with the control group are elaborated in Table 5. However, the placement may not be very clear. We will make further markings in the method steps section to clarify this.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Students in primary school could benefit from the development of creative thinking and problem-solving abilities through VR 3D modeling. This study develops an IVR-based 3D modeling approach and explores its impact on students' creative thinking, problem-solving tendency, and cognitive load. The testing phase was conducted in an elementary school with a sample of 77 sixth-grade students. The findings show that IVR-based 3D modeling approach significantly enhances students' creative thinking and problem-solving abilities, while reducing students' cognitive load.

---

I commend the authors for their interesting article, with a relevant topic for the scientific literature, clear in the motivation, contribution, methodology, testing and results. The paper is easy to read, well-written and the narrative is well-developed within the paper flow.

Good luck with your future research!

Author Response

Thank you for recognizing my research.
I wish you all the best.

Reviewer 4 Report

Comments and Suggestions for Authors

1. Why do some data analysis methods use MANOVA (such as: creative thinking), and some data use ANCOVA (such as: problem-solving skills)? Is there any reason why different statistical analysis methods should be used? It is recommended that it should be clearly explained, or even The data of this study are used as an example to illustrate the applicability of each analysis method.

2. In addition to the background of the participants, does this study have any other limitations? For example, if the equipment is fully equipped, can the same results be obtained for large classes (more than 100 people)?

3. It is recommended to add more content in the Discussion section on how to interpret experimental results, especially for those results that did not achieve the expected results. This should include in-depth analysis of experimental design or system mechanisms, external influencing factors, and future research directions.

4. It is recommended to conduct more comparisons with existing literature, especially those related to the application of immersive virtual reality technology in the field of education, which will help position the contribution of this study.

Author Response

1.Thank you for the specific questions regarding the statistical analysis methods we used. Indeed, we have employed various statistical methods to analyze different types of data in our study, based on considerations of the data structure and our research objectives. Here is a detailed explanation of why we chose MANOVA and ANCOVA, as well as their respective applicabilities:

In our study, we used MANOVA to analyze data on creative thinking because creative thinking includes multiple sub-dimensions, which are treated as several related dependent variables. MANOVA allows us to consider multiple dependent variables simultaneously, which helps in assessing the overall impact of IVR on students’ creative thinking, while also examining the interactions between different dimensions.

ANCOVA was used for the analysis of problem-solving abilities because we needed to control for potential confounding variables (such as students' prior problem-solving abilities), which could affect subsequent performance. By using ANCOVA, we can adjust these baseline scores to more accurately estimate the net effect of the IVR intervention.

We will include this explanation in the 4.4. Data analysis methods section of the paper to clarify.

2.Thank you for raising further questions about potential limitations in our study. Your mention of considerations regarding equipment provision and large classroom environments is very important, as these factors could indeed affect the generalizability of the research findings and the feasibility of practical application. Due to the high cost of immersive virtual reality equipment, which requires regular maintenance and necessary technical support, the effectiveness of experiments in large classroom settings may be impacted. We have included additional information about other limitations in the deficiencies and prospects section of our conclusions.

The revised information is below:

There are some limitations of this study that need to be noted. First, the participants in this study were all sixth-grade students in the same elementary school in China, which means that it may not be appropriate to extrapolate the results to other subjects with different learning environments, family backgrounds, and age groups. Secondly, the study lacked procedural data to support the procedural user behavior of immersive virtual reality, which is a very worthwhile direction for future research and exploration. The measurement of this study was based on subjects' self-reported data with some perceived bias[84]. In addition, we acknowledge that the limited duration of the experiment may not sufficiently assess the long-term development of students in creative thinking and problem-solving abilities due to some practical constraints.

In future research, we plan to extend the duration of the course to more accurately assess the impact of the IVR teaching method on students' long-term learning outcomes. Studies can be expanded to include different schools, age groups, and social backgrounds to increase the representativeness of the sample. This will help us better understand the role of immersive virtual reality technology in different educational settings. Additionally, understanding the process-specific behaviors of students in immersive virtual reality environments will provide valuable insights into educational interventions, and collecting and analyzing process data will help to reveal underlying mechanisms that will provide a deeper understanding.

3.Thanks to your suggestion, we have added an explanation of the experimental results in the Discussion section, in particular an analysis of the reasons why the experimental results did not reach the expected results.

4.Thanks to your suggestion, we have added literature related to the application of immersive virtual reality technology in education to highlight the contribution of the research. Please see 2.3 Immersive Virtual Reality in Education in the Literature Review for modifications.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Accept as is

Comments on the Quality of English Language

Accept as is