Design and Evaluation for Immersive Virtual Reality Learning Environment: A Systematic Literature Review

Round 1

Reviewer 1 Report (Previous Reviewer 2)

Thanks to the authors for revising the manuscript. The paper is significantly improved, and all weak points regarding data collection timeframe, utilized sources, findings/results accuracy, and presentation are addressed. Still, in the paper, there is an extra heading (between table 2 and the ‘discussion ‘ section) that must be removed in the proofreading phase.

Author Response

Thank you for pointing this out. We agree with this comment. Therefore, we have revised the manuscript based on comments provided. Correction was made in Page 11.

Author Response File: Author Response.pdf

Reviewer 2 Report (Previous Reviewer 1)

well-done.

Author Response

Thank you for providing constructive comments for us to improve the manuscript titled “Design and Evaluation for Immersive Virtual Reality Learning Environment: A Systematic Literature Review“. We appreciate the time and effort that you and the reviewers have dedicated to providing your valuable feedback on our manuscript. We are grateful to your positive comments on our paper.

Author Response File: Author Response.pdf

Reviewer 3 Report (New Reviewer)

The manuscript (review article) is already in a mature status. However, there are a couple of points the authors should consider when preparing their next manuscript version. Strengthening of the level of detail in the conclusion as well as a more detailed discussion of cognitive aspects would help to gain additional benefit for this interesting manuscript. The points are ordered by relevance, beginning with the strongest impact:

1)      A review paper should go beyong a systematic review and analysis of the current (international) status of a certain research field. The review and analysis are ok (improvements are suggested below), but I miss a detailed presentation about what follows from the analysis. Could you identify research gaps with focused research topics within the various disciplines dealing with methods, technologies and applications in the immersive VR spectrum? This concluding passage does not provide a clear take-home-message for your readership:

 

“Our analysis results hadindicated that the application of virtual reality in both Natural Science and Health Science had received the most attention. On the other hand, cultural and historical education such as Cultural Heritage and Archaeology had acquired the least amount of interest for the educators to apply virtual reality technology in their teachings. This bestowed us an opportunity to delve in deeper regarding the application of virtual reality technology for educational purposes. If we are able to pay more attention to these fields, it will certainly be a large contribution to talent training.“

 

2)      You leave out a detailed inclusion of the important aspect of cognition in immersive VR (learning) environments. In section 4.1., you indicate such effects (“When users engulf themselves in immersive virtual reality learning domain, they will need to reconstruct their cognition of the virtual world“). However, what does the state-of-the-art literature state about the impact of user cognition when processing immersive VR environments. (see e.g. https://doi.org/10.1038/s41598-018-29029-x & https://doi.org/10.3390/ijgi10030150 & https://doi.org/10.3390/ijgi10030150)

 

3)       It would be worth adding in the dicussion and conclusion section which potentials you see for the field of spatial cognition (VR environments are inherently spatial) in future research.

Author Response

Thank you for providing constructive comments for us to improve the manuscript titled “Design and Evaluation for Immersive Virtual Reality Learning Environment: A Systematic Literature Review“. We appreciate the time and effort that you and the reviewers have dedicated to providing your valuable feedback on our manuscript. We are grateful to the reviewers for their insightful comments on our paper. We have been able to incorporate changes to reflect most of the suggestions provided by the reviewers. We have highlighted the changes within the manuscript. Here is a point-by-point response to the reviewers’ comments and concerns.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report (New Reviewer)

The re-submission considers the points suggested by the reviewer. The argumentation in the response letter is logical. Against this background, I would like to recommend the current version of the manuscript for publication in this journal.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

A well-developed and comprehensive review.

Reviewer 2 Report

This paper presents a systematic literature review concerning the design and evaluation of immersive Virtual Reality (systems/environments) as (design) learning environments. The overall format of the paper is acceptable. However, some serious issues regarding data collection, results’ presentation/accuracy, and references need to be addressed, corrected, and revised.

First, there is a question for the authors; what was the strategy behind using another open-access journal’s search engine (IEEE) and attempting to publish the work in another open-access journal (MDPI)? It is highly recommended to include relevant works published by MDPI.

Second, what was the concept behind choosing the 2017-2021 time frame for searching and exploring the data?

Since the paper will be accessible for readers in 2023 (if approved and published), missing the published relevant works in 2022 will be a considerable gap and weak point regarding presenting up-to-date research and studies. I highly recommend expanding the data collection timeframe to the end of 2022 and including as many relevant works as references or search results. 

In addition, with a quick check of the study’s references, it is clear that many of the relevant published works between 2017 and 2021 in the criteria of VR systems/environments utilization for design learning /pedagogy do not exist and detected in your search and not included in collected data and references. As a guide, a few relevant published works between 2017 and 2021, and 2022 are listed at the end of the review report, and the inclusion and citation of all of them are highly recommended.

 

On the other hand, although the paper proposes that the timeframe for searching and collecting the relevant works is limited to 2017 until 2021, some publications that do not belong to this timeframe are included. Check table 1 (Hong et al. 2015) and table 2 (Fernandes et al. 2016).

 

There are two sections as result sections (3 & 4) in the paper; which one is correct and which must be renamed or removed?

 

In the introduction section, it is mentioned that Data analyses and Discussion are sections 4 and 5, respectively, while none exist in the paper.

 

In the sub-section “2.3.2. Research Design,” the description of figure 6 and the presented data is confusing, and its accuracy is doubtful. It is mentioned that between 61 studies, 4 studies are not included and removed, but the sum of the presented numbers in the bar chart is 70. 

The format of the subsections’ titles is not matched, and many have a big gap between the number and text which must be fixed. (e.g., check and compare  the title of subsections 3.1 and 4.1)

The list of recommended references can be found below:

 

 

           Paes, D.; Arantes, E.; Irizarry, J. Automation in Construction Immersive Environment for Improving the Understanding of Architectural 3D Models : Comparing User Spatial Perception between Immersive and Traditional Virtual Reality Systems. Autom. Constr.2017, 84, 292–303, doi:10.1016/j.autcon.2017.09.016.

 

           Srivastava, P.; Rimzhim, A.; Vijay, P.; Singh, S.; Chandra, S. Desktop VR Is Better Than Non-Ambulatory HMD VR for Spatial Learning. Front. Robot. AI2019, 6, 1–15, doi:10.3389/frobt.2019.00050.

 

           Trunfio, M.; Rossi, S. Advances in Metaverse Investigation: Streams of Research and Future Agenda. Virtual Worlds2022, 1, 103–129, doi:10.3390/virtualworlds1020007.

 

           Morgado, L.; Allison, C.; Beck, D.; Penicheiro, F. Immersive Learning Research; 2018; Vol. 24; ISBN 9783319417684.

 

           Hou, N.; Nishina, D.; Sugita, S.; Jiang, R.; Oishi, H.; Kindaichi, S.; Shimizu, A. A Study on Possibility of Using VR Space in Design Education Part 1: Verification of VR Space Effectiveness by Learning Experiment of Scale Feeling. J. Environ. Eng.2021,86, doi:10.3130/aije.86.670.

           Azarby, S.; Rice, A. Understanding the Effects of Virtual Reality System Usage on Spatial Perception: The Potential Impacts of Immersive Virtual Reality on Spatial Design Decisions. Sustainability2022, 14, 10326, doi:10.3390/su141610326.

 

           Horvat, N.; Škec, S.; Martinec, T.; Lukacevic, F.; Perišic, M.M. Comparing Virtual Reality and Desktop Interface for Reviewing 3D CAD Models. Proc. Int. Conf. Eng. Des. ICED2019, 2019-Augus, 1923–1932, doi:10.1017/dsi.2019.198.

 

           Gomez-Tone, H.C.; Chávez, M.A.; Samalvides, L.V.; Martin-Gutierrez, J. Introducing Immersive Virtual Reality in the Initial Phases of the Design Process—Case Study: Freshmen Designing Ephemeral Architecture. Buildings2022,12, doi:10.3390/buildings12050518.

 

           Gómez-Tone, H.C.; Escapa, J.B.; Escapa, P.B.; Martin-Gutierrez, J. The Drawing and Perception of Architectural Spaces through Immersive Virtual Reality. Sustain.2021, 13, 1–25, doi:10.3390/su13116223.

 

           Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A Systematic Review of Immersive Virtual Reality Applications for Higher Education: Design Elements, Lessons Learned, and Research Agenda. Comput. Educ.2020, 147, 103778, doi:10.1016/j.compedu.2019.103778.

 

           Soliman, M.; Pesyridis, A.; Dalaymani-Zad, D.; Gronfula, M.; Kourmpetis, M. The Application of Virtual Reality in Engineering Education. Appl. Sci.2021,11, 1–14, doi:10.3390/app11062879.

 

           Bashabsheh, A.K.; Alzoubi, H.H.; Ali, M.Z. The Application of Virtual Reality Technology in Architectural Pedagogy for Building Constructions. Alexandria Eng. J.2019,58, 713–723, doi:10.1016/j.aej.2019.06.002.

 

           Abu Alatta, R.; Freewan, A. Investigating the Effect of Employing Immersive Virtual Environment on Enhancing Spatial Perception within Design Process. Archnet-IJAR2017, 11, 219–238, doi:10.26687/archnet-ijar.v11i2.1258.

 

           Lach, E.; Benek, I.; Zalewski, K.; Skurowski, P.; Kocur, A.; Kotula, A.; Macura, M.; Pamuła, Z.; Stankiewicz, M.; Wyrobek, T. Immersive Virtual Reality for Assisting in Inclusive Architectural Design. In Proceedings of the Advances in Intelligent Systems and Computing; 2020; Vol. 1061, pp. 23–33.

 

           Azarby, S.; Rice, A. Scale Estimation for Design Decisions in Virtual Environments: Understanding the Impact of User Characteristics on Spatial Perception in Immersive Virtual Reality Systems. Build. 2022, Vol. 12, Page 14612022,12, 1461, doi:10.3390/BUILDINGS12091461.

 

           Williams, J.E.; Orooji, F.; Aly, S.J. Integration of Virtual Reality (VR) in Architectural Design Education: Exploring Student Experience. In Proceedings of the ASEE Annual Conference and Exposition, Conference Proceedings; 2019.

 

           Zhang, Y.; Liu, H.; Kang, S.C.; Al-Hussein, M. Virtual Reality Applications for the Built Environment: Research Trends and Opportunities. Autom. Constr.2020, 118.

 

           Kamińska, D.; Sapiński, T.; Wiak, S.; Tikk, T.; Haamer, R.E.; Avots, E.; Helmi, A.; Ozcinar, C.; Anbarjafari, G. Virtual Reality and Its Applications in Education: Survey. Inf.2019, 10, doi:10.3390/info10100318.

 

           Taub, M.; Sawyer, R.; Smith, A.; Rowe, J.; Azevedo, R.; Lester, J. The Agency Effect: The Impact of Student Agency on Learning, Emotions, and Problem-Solving Behaviors in a Game-Based Learning Environment. Comput. Educ.2020,147, 103781, doi:10.1016/j.compedu.2019.103781.

 

           Cook, M.; Lischer-Katz, Z.; Hall, N.; Hardesty, J.; Johnson, J.; McDonald, R.; Carlisle, T. Challenges and Strategies for Educational Virtual Reality: Results of an Expert-Led Forum on 3D/VR Technologies across Academic Institutions. Inf. Technol. Libr.2019, 38, 25–48, doi:10.6017/ital.v38i4.11075.

 

           Bjørn, P.; Wulff, M.; Petræus, M.S.; Møller, N.H. Immersive Cooperative Work Environments (CWE): Designing Human-Building Interaction in Virtual Reality. Comput. Support. Coop. Work CSCW An Int. J.2021, 30, 351–391, doi:10.1007/s10606-021-09395-3.

 

           Azarby, S.; Rice, A. User Performance in Virtual Reality Environments: The Capability of Immersive Virtual Reality Systems in Enhancing User Spatial Awareness and Producing Consistent Design Results. Sustainability2022, 14, 14129, doi:10.3390/su142114129.

 

           Aydin, S.; Aktaş, B. Developing an Integrated VR Infrastructure in Architectural Design Education. Front. Robot. AI2020, 7, doi:10.3389/frobt.2020.495468.

 

Al-Gindy, A.; Felix, C.; Ahmed, A.; Matoug, A.; Alkhidir, M. Virtual Reality: Development of an Integrated Learning Environment for Education. Int. J. Inf. Educ. Technol.2020, 10, 171–175, doi:10.18178/ijiet.2020.10.3.1358.

 

 Kelly, J.W.; Doty, T.A.; Ambourn, M.; Cherep, L.A. Distance Perception in the Oculus Quest and Oculus Quest 2. Front. Virtual Real.2022, 3, 1–7, doi:10.3389/frvir.2022.850471