The Effect of Elementary School Soccer Instruction Using Virtual Reality Technologies on Students’ Attitudes toward Physical Education and Flow in Class

Round 1

Reviewer 1 Report

This paper is a resubmission of a draft that I have previously reviewed. I still believe that the paper has potential to make an interesting contribution to ubicomp and to the research on learning technologies for sport applications. The authors have implemented some recommendations in this draft, but I believe the current draft was too rushed and falls short of a serious round of major revision. In particular, the paper needs a Related Work section to introduce relevant work in VR and learning technologies, and VR for sport training. This could greatly help to streamline the narrative and to highlight the specific contribution of this research. So far, in all honesty, it is very difficult to assess the specific contribution of this paper, because of the lack of relevant literature. 

Author Response

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. I look forward to working with you to move this manuscript closer to publication in Sustainability. 
The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith. 
Thank you for your consideration. I look forward to hearing from you. 

Author Response File: Author Response.pdf

Reviewer 2 Report

 Experiments have not been conducted correctly to show the usefulness of the proposed method. The experiments you have conducted are only subjective evaluation experiments based on questionnaires for each group. This means that no fair experimentation has been conducted for each approach. This is because it has not been verified whether the items required for soccer education are actually realized for the experimental group. For example, it is necessary to carry out quantitative experiments on the actual improvement of the student's soccer ability, such as the individual soccer skills of the students and the strength of the team. This should be done in the real environment. The reason for this point is that similar results could be obtained by experimenting with soccer video games.

 You have to clearly describe the same or different points between what you did with VR technology and what you did in the actual class. For example, it is possible to give an emphasis effect that is not possible in reality with the one using VR technology. These are very important findings if they give students confidence and as a result improve their soccer ability.
However, it is completely unknown from the paper what kind of VR technology is used in the proposed method. This is not reproducible in the experiments described in the paper, as the results depend on the VR application used. It is necessary to carry out rigorous experiments to show what kind of VR effect has what kind of effect.

 I have decided that re-experiment is necessary for the above two points. However, if the above points can be improved, it is worth accepting.

Author Response

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. I look forward to working with you to move this manuscript closer to publication in Sustainability. 
The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith. 
Thank you for your consideration. I look forward to hearing from you. 

Author Response File: Author Response.pdf

Reviewer 3 Report

The present manuscript is well-written and addresses an interdisciplinary and sustainable research question in an education scenario which is methodologically investigated using modern techniques of 3D visualization (immersive virtual environments). I would like to make some suggestions which could help to improve the final version of the manuscript.

• The use of 3D data and game engines is inherently linked to space and geography. In your introduction, the authors also mention this linkage in the introductory chapter: “This allows learners and teachers to interact with each other beyond the limits of geographical space.” (ll. 42-43). There is an ongoing debate that is focused on the potentials of VR that could even extend the possibilities to explore the real physical space, such as integrating multimedia into VR (without losing the feeling of presence (immersion)) or taking views from perspectives which could not be reached in the real world. It would be worth mentioning the idea that physical space is extended and use the geography example. These two references could be worth considering:

Edler, D., Keil, J., Wiedenlübbert, T., Sossna, M., Kühne, O., Dickmann, F. (2019): Immersive VR Experience of Redeveloped Post-Industrial Sites: The Example of "Zeche Holland" in Bochum-Wattenscheid. In: KN - Journal of Cartography and Geographic Information, 69 (4): 267-284. https://doi.org/10.1007/s42489-019-00030-2

Hruby, F., Castellanos, I. & Ressl, R. (2020): Cartographic Scale in Immersive Virtual Environments. KN J. Cartogr. Geogr. Inf. https://doi.org/10.1007/s42489-020-00055-y

 

• Could you be a bit more precise when it comes the connection between the introduction and your six research hypotheses? If it is not possible to derive all these hypotheses from cited literature, could you mention that you follow an explorative research approach?

 

• In your conclusion, you justify soccer as case study in the following way: ”Soccer was chosen as the sport because the program was well developed by the VR company, and elementary school students’ participation in soccer was high.” (ll. 333-334). Is there no other reason why soccer would be a suitable case study? Is there only the aspect of availability?

 

• Please add units in your figures.

Author Response

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. I look forward to working with you to move this manuscript closer to publication in Sustainability. 
The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith. 
Thank you for your consideration. I look forward to hearing from you. 

Author Response File: Author Response.pdf

Reviewer 4 Report

In my opinion, a correlation between Study objectives and research hypotheses is necessary.  

Relevance: the paper seems to be relevant for both researchers/teachers and practitioners. The described procedures / examples seems to be technically sound.

The paper is significant due to the following arguments:

 the authors' original contribution is very clear explained

 the used methods are well described

From a quality point of view, the manuscript is technically correct. The entire manuscript is well written. 

Author Response

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. I look forward to working with you to move this manuscript closer to publication in Sustainability. 
The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith. 
Thank you for your consideration. I look forward to hearing from you. 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I appreciate the authors' work on editing the introduction to clarify the contribution of this paper. As I mentioned before, I believe the experimental methods are sound. I still think, however, that it would be useful to have a related work section (even if a short one), following the introduction, to discuss attempts to design or implement other technological interventions in related settings (e.g., gym, dance, sports in general). This would help clarifying why the use of VR is different from other technological attempts, and why VR is particularly suited for soccer education. A few examples:

• Trajkova, M., & Cafaro, F. (2018). Takes Tutu to ballet: designing visual and verbal feedback for augmented mirrors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(1), 1-30.
• Misha Patel and Aisling Ann O'Kane. 2015. Contextual Influences on the Use and Non-Use of Digital Technology While Exercising at the Gym. In <i>Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems</i> (<i>CHI '15</i>). Association for Computing Machinery, New York, NY, USA, 2923–2932. DOI:https://doi.org/10.1145/2702123.2702384
• Chan, J. C., Leung, H., Tang, J. K., & Komura, T. (2010). A virtual reality dance training system using motion capture technology. IEEE transactions on learning technologies, 4(2), 187-195.
• Ochi, D., Kameda, A., Takahashi, K., Makiguchi, M., & Takeuchi, K. (2016). VR technologies for rich sports experience. In ACM SIGGRAPH 2016 Emerging Technologies (pp. 1-2).
• Vallerand, R. J., & Losier, G. F. (1999). An integrative analysis of intrinsic and extrinsic motivation in sport. Journal of applied sport psychology, 11(1), 142-169.

Author Response

Dear reviewer,

We wish to re-submit the manuscript titled “The Effect of Elementary School Soccer Instruction Using Virtual Reality Technologies on Students’ Attitudes toward Physical Education and Flow in Class.” The manuscript ID is Sustainability-1118067.

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. We look forward to working with you to move this manuscript closer to publication in Sustainability.

The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith.

Thank you for your consideration and I look forward to hearing from you.

Take care.

Regards
Hyun Suk Lee

Author Response File: Author Response.pdf

Reviewer 2 Report

My concerns on the following two points have not been improved.

(i) There is a serious problem with the reproducibility of the experiment in your paper. You are only experimenting with a few classes. Nevertheless, your paper does not provide details on how the lecture was conducted. These mean that there is a risk that the experimental results will vary depending on the method and content of the lecture. This makes your experiment unreliable. Please solve the above problem by describing each method exactly to improve the reproducibility of the experiment, or by treating it as a general theory using many various lectures and VR applications.

(ii) Your paper proposes a method focusing on soccer in physical education. Your paper seeks to evaluate the effectiveness of the PE lecture. If so, the items required in the lecture should be clarified and the experiment should be conducted accordingly. Of course, I also understand the importance of psychological evaluation. I think psychological evaluation is part of that. However, when targeting soccer Instruction, not only psychological evaluation but also physical evaluation should be performed to show the usefulness of the proposed method. Of course, there is no need for the proposed method to surpass the conventional method in all cases. If you claim that physical evaluation is less important in physical education, please provide evidence.

Author Response

Dear reviewer,

We wish to re-submit the manuscript titled “The Effect of Elementary School Soccer Instruction Using Virtual Reality Technologies on Students’ Attitudes toward Physical Education and Flow in Class.” The manuscript ID is Sustainability-1118067.

We thank you for your thoughtful suggestions and insights. The manuscript has benefited from these insightful suggestions. We look forward to working with you to move this manuscript closer to publication in Sustainability.

The manuscript has been rechecked and the necessary changes have been made in accordance with your suggestions. The responses to all comments have been prepared and are attached herewith.

Thank you for your consideration and I look forward to hearing from you.

Take care.

Regards
Hyun Suk Lee

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

This is a well-designed and well-explained study; however, the standard deviation of the female participants in both groups on "interest" and "confidence" measure is much higher than the males.  Are there any thoughts on this issue?

Reviewer 2 Report

In your paper, you investigates the effects of VR technology on physical education. I think its purpose is meaningful. However, there are insufficient points in the paper with respect to the novelty, reproducibility, and usefulness of the proposed method.

Your experiment is just the result of verifying only one of the many VR applications. You are abstractly described as VR technology for one application, but there are various technologies in VR. I think the quality of the application affects the results obtained in the experiment. Readers do not know what technology is used in your VR application and what is being achieved. What kind of technology has what kind of effect on physical education is important. In the experiment you should verify them.

First of all, it is necessary to describe what the goals of physical education should be achieved. If you focus on soccer education, you should describe how the experimental and control groups are trying to achieve them. Experiments should be conducted to verify how much of what is needed for soccer education has been achieved.

Reviewer 3 Report

This paper reports a study that assesses the impact of VR technologies for PE (in particular, soccer practice). I think the paper makes an interesting contribution to ubicomp and learning technology. The following are my suggestions for revisions.

* The target population should be better clarified in the introduction: the introduction mentions “teenagers,” but the study participants were children from two elementary schools. From which grade/age group?

* Similarly, PE is a broad discipline. The soccer example is introduce late in the text (page 5). It would help if, at the beginning of section 2.1, the authors could mention the soccer scenario, or add few examples of the activities that children typically do at school and that they can be improved thanks to their technological intervention (the VR system).

* With three-way ANOVA, I assume the authors actually mean three-way mixed ANOVA, is it correct? As I understand the experiment (and this could also be added in the text for clarity), it included one repeated measure (pre-test vs. post-test), and two sets of conditions (VR/traditional and male/female). Overall, the experimental design includes many experimental conditions (maybe too many? I wonder about the limited role of gender in the analysis). Table 7 shows a lack of a three way effect, which I do not think is a problem per se, as long as it is better disclosed in the text. I.e., I would expect to read something like: “There was not a statistically significant three-way interaction between …, F(…) = …, p=…. We continued the analysis because there was a statistically significant two-way interaction between … time and groups …”

* I’d suggest removing the actual numbers from the statistical analysis from the abstract, to provide a higher-level overview of the major results and contribution.

* The first sentence in the Introduction (“ICBM technology, which refers to the Internet of Things, Cloud, Big Data, and Mo- 25 bile technology, signals the arrival of a hyper-world that we cannot understand with our 26 existing thinking”) is cryptic and seems irrelevant for this work.

* Section 1.1 could be moved out of the Introduction and renamed Problem Statement. Additionally, although the list of hypothesis is reasonable, it would be interesting if the authors could elaborate on the rationale (i.e., prior work, relevant literature) for these hypothesis.

* Overall, the PE application context and the challenges describe make me think of a relevant paper from the dance technology literature (it could be referenced to strengthen the point that sports and PE education can benefit from technology): Milka Trajkova, Francesco Cafaro, and Lynn Dombrowski. 2019. Designing for Ballet Classes: Identifying and Mitigating Communication Challenges Between Dancers and Teachers. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). Association for Computing Machinery, New York, NY, USA, 265–277. DOI:https://doi.org/10.1145/3322276.3322312