Teaching Methodology for Understanding Virtual Reality and Application Development in Engineering Major

Round 1

Reviewer 1 Report

The paper is well orgnaized and presents a new analysis for teaching approach. However, the follwing points are recommended to be discussed to improve the paper. 

1- References avoid to use multiple references at once, ex 1-3   4-8 , cite each reference alone

2. the introduction can be improved by discusion of other teaching process like flipped class methods...

3. What are the skills that the method want to improve. 

4. Discuss effect of proposed method on students cognitive skills. 

5. What is the effect on the soft skills of students. 

6. paper can be improved by adding a section on  ABET requirements or any other standards. 

 

Author Response

Response to Review 1 Comments

Thank you for giving me the opportunity to submit a resubmitted draft of my manuscript titled Teaching Methodology for Understanding Virtual Reality and Application Development in Engineering Major to Sustainability. We appreciate the time and effort that you and the reviewers have dedicated to providing your valuable feedback on my manuscript. We are grateful to the reviewers for their insightful comments on my paper. We have been able to incorporate changes to reflect most of the suggestions provided by the reviewers. We have highlighted the changes within the manuscript.

Here is a point-by-point response to the reviewers’ comments and concerns.

We found a crack in the submitted manuscript template form. For this reason, we have downloaded and rewritten at latest sustainability form. Some line numbers in the revised version may not match the numbers created in the review, but we would like to inform you that the content of the text was created with the version before the revision.

In addition to the above comments, all spelling, grammatical errors and journal forms pointed out by the reviewers have been corrected.

We look forward to hearing from you in due time regarding our submission and to respond to any further questions and comments you may have.

The paper is well orgnaized and presents a new analysis for teaching approach. However, the follwing points are recommended to be discussed to improve the paper. 

Comment 1: References avoid to use multiple references at once, ex 1-3   4-8 , cite each reference alone

Revised reference number: 1, 2, 3, 4,5,6,7,8,14,15,16, 17, 20 ,21, 22, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 ,41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51

Response: Thank you for pointing this out. As per your advice, we separated some multiple references.

Comment 2: the introduction can be improved by discusion of other teaching process like flipped class methods...

Response: Thank you for giving me a good comment. Rather than focusing on the teaching method, we concentrate on the structure of the course, which consists of three stages. We lectured Webex-based non-face-to-face courses during the courses because face-to-face courses were impossible due to the Covid-19 pandemic. Concentrating on your feedback, we will research reconstructed improved courses that take advantage of that course and previous VR-based learning methods and compares them with a variety of teaching methods, including flipped learning in the future.

Comment 3: What are the skills that the method want to improve. 

Response: The method proposed in this paper proposes a class method that deals with both hardware-related technical background and software technology. Conventional course design methods did not combine hardware-based techniques for understanding optical structures for understanding head-mounted displays with software techniques for developing VR-based content. However, previous studies have confirmed that such biased classes do not meet the requirements of current VR-related education and industry. We redesigned courses that dealt with general VR technology in line with the latest trends and verified the effectiveness of evaluating and proposing to students.

Comment 4: Discuss effect of proposed method on students cognitive skills. 

Response: The cognitive benefit is known to imply that students perceive better memorization, comprehension, and application of pre-existing content. This paper confirmed that there was a significant difference between the students who took the course and those who took it. For this reason, we have supplemented the Step 1 and Step 2 courses to ensure background knowledge instead of prior learning in the first half of the class.

Comment 5: What is the effect on the soft skills of students. 

Response: The software learning process developed in this paper is different from other classes. Unlike the traditional course method, we understand hardware technology and learn about software technology. In the case of VR technology, unlike conventional PC and mobile environments, we use a unique device called HMD. In particular, to utilize HMD to create an immersive virtual environment, the ability to configure the immersive virtual environment (for example, Modeling, Rendering, and Interaction) is required. And because these technologies are closely related to hardware, understanding hardware characteristics and learning software techniques can produce better learning outcomes and convey cognitive benefits.

Comment 6: paper can be improved by adding a section on ABET requirements or any other standards. 

Response: Thank you for pointing this out. Once Revision is complete, you will find the process of getting ABET accredited.

 

Reviewer 2 Report

The current work aims to propose a design for a VR course so that learners can utilise VR in education. The topic is of great importance in the era of digitalisation and the modernisation of education. The experimental work conducted by the authors is highly appreciated. However, the manuscript suffers a low-quality of presentation, thus a loss of the scientific value and the reusability of its findings. The backbone of the academic work is its methodology which allows readers to understand its map and design. Once the methodology is improved, the value of this work will better shine.

In the “Introduction” section, the authors jumped quickly to the conclusion that VR-related courses do not meet the current industrial and educational needs. This conclusion sounds strong before conducting a relevant literature review. The authors insert the “metaverse” in the presentation of the paper’s content, which is not that suitable … The last two closing paragraphs provide some detailed explanation of the content of the upcoming sections, which is not necessary as it is not yet justified. This definitely needs to be checked. This section is meant to show the motivation; however, it compares project-based learning to problem-based learning and then speaks about VR. This section needs improvement and better structuring. Many things can be deleted without affecting the manuscript.

The “Literature” section provides a background on the use of VR for various education scenarios, yet there are some phrases that express some exaggeration such as “VR is a typical technology that has led to such changes [33].”.  Some literature works such as [47], [48] and [49] are explained in 5 lines for each without any significant outcome related to VR use. References [50] and [52] are explained with almost 10 lines each, which is too much. The authors did not define the research gap that is not covered in the literature. Consequently, it is not understandable how they built their research methodology! In light of the 'objective' defined in the “Introduction” (and not as it should be in the literature review), this section has to be rewritten, structured and only the relevant literature works should be included with a concise explanation of their contribution.

The “Methodology” section in its current form is a mixture of both a methodology and a case study. The authors have to define a clear methodology in which they define targets, targeted data, the type of data analysis, comparisons to similar works (if applicable), etc. Those elements are not found in the present manuscript as a concept but as an application. Thus, it is necessary to clarify them.  

Please provide further explanation of the concept proposed in Figure 2. Also, for Table 2, please explain the codes: TM, CCT, etc.  Why not also use the same codes in Figure 7?

Due to the absence of a clear methodology, there seems to be a good experimental work that is poorly presented. The evaluation through the questionnaire follows the qualitative approach which involves a thematic analysis in addition to the choice of sample size (the authors refer to this later in the limitation), what is inferred from the calculated standard deviation? … what is shown in excerpts does not contribute much, so they should be reduced.

To show the benefit of this work, there should be takeaways the reader/teachers/trainers can rely on when designing VR training, in addition to an “empirical” procedure/method/scenario to be followed.

Below are some remarks on the language, structure and text coherence. 

• Page 1, Lines 26-28: please rephrase and double-check this sentence: “In particular …  are required”.

• Page 2, Line 82: please explain the term “TA-led content creation”

• Pages 1-2, Lines 82-90: please stick to the passive voice instead of using the pronoun “we”. 

• Page 3, Line 100: The sentence “This change has brought about” does not look right.

• Page 3, Line 108: the phrase “In the recent STEM field” is not common/correct.

• Page 3, Line 111: what sort of design is in the phrase “Research to advance the design”.

• Page 3, Lines 112-113: “Upon reviewing the literature, we found a report about the positive effects of education through VR”, why is not this report referenced? 

• Page 3, Line 115: please define HMD in the text (head-mounted displays) and CAVE (Cave Automatic Virtual Environment).

• Page 4, Line 181: please define the acronym OOP (Object Oriented Programming).

• Page 5, Line 214: please revise the phrase “they guide them to propose”.

• Page 5, Line 228: please define the acronym IR (Infra-Red?).

• Page 7, Line 276: please add “T” to “BL”.

• Page 8, the caption of Figure 4: please capitalise “bloom”. 

• Page 9, Lines 322- 323: please check the sentence “Table 1 shows the results of a survey of students before attending courses regarding whether they meet the prerequisites and their VR experience”

The manuscript holds value, but has to be more concise.

Author Response

Response to Reviewer 2 Comments

Thank you for giving me the opportunity to submit a resubmitted draft of my manuscript titled Teaching Methodology for Understanding Virtual Reality and Application Development in Engineering Major to Sustainability. We appreciate the time and effort that you and the reviewers have dedicated to providing your valuable feedback on my manuscript. We are grateful to the reviewers for their insightful comments on my paper. We have been able to incorporate changes to reflect most of the suggestions provided by the reviewers. We have highlighted the changes within the manuscript.

Here is a point-by-point response to the reviewers’ comments and concerns.

We found a crack in the submitted manuscript template form. For this reason, we have downloaded and rewritten at latest sustainability form. Some line numbers in the revised version may not match the numbers created in the review, but we would like to inform you that the content of the text was created with the version before the revision.

Overall response: Thank you for your detailed review. It is difficult to respond to all questions during the short review period, but we have tried our best to respond to your review comments, so please understand this part.

[Introduction section]

Response: The term 'metaverse' can actually be used in various fields other than immersive technology represented by VR technology. If we use the term 'metaverse', as in your comment, it can confuse the reader as to which domain the class proposed in the text covers. For this reason, we have removed the word 'metaverse' from the Introduction and Methodology figures and Section 3-1 [line number: 98, 238].

Also, paragraphs that did not dare mention the introduction section in the last two paragraphs have been moved to Methodology. These improvements reduced the overall volume of the introduction section without affecting what we wanted to offer [line number: 90-107 to 195-212].

[Literature & Methodology section]

Response: In the text, we agree that the bibliography is overly verbose. In response to your feedback, I have removed the over-explaining sentences in some of the cited references.

Through the papers cited in the Literature, we wanted to mention: All existing VR-related classes are biased toward specific technologies which do not meet current industrial and educational needs. To focus on the content mentioned above, we have focused on what technologies the cited papers deal with. They then briefly described the results their work achieved in the cited reference [line number: 163-166, 168-174].

We think the weak point of our paper is that it does not explain the research gap with previous research. We performed both quantitative and qualitative analyses. This does not mean that the Questionnaire item we used has not been validated. Many papers have used these items to compare educational or VR experience effects. We decided that a quantitative analysis would be difficult to have statistical significance due to the small number of people who participated in the experiments to validate the proposed method. Also, since we did not use the same categories and questionnaire, we decided it was inappropriate to compare them on the same line.

For this reason, the discussion in the manuscript focuses on qualitative analysis through in-depth interviews rather than quantitative analysis. Finally, we chose a method that complements the proposed method by advancing self-feedback based on experimental data, considering that it is difficult to compare with other previous studies. In future research, we will use more universal evaluation methods to ensure the credibility of the papers to complement the problems that have arisen in this paper.

 

Additional Comment

• Page 1, Lines 26-28: please rephrase and double-check this sentence: “In particular …  are required”.

Response: Thank you for pointing this out. We have revised the mentioned sentence to reflect your opinion. [line number: 20-22]

• Page 2, Line 82: please explain the term “TA-led content creation”

Response: The term "content creation training" was explained in section 3.2. But we think we provided confusion by abbreviating the TA's full name instead of expressing it in the text. For this reason, TA is written on that page as the full name. [line number 202-203]

• Pages 1-2, Lines 82-90: please stick to the passive voice instead of using the pronoun “we”. 

Response: Thank you for pointing this out. We have revised the mentioned sentence to reflect your opinion. [line number: 195-212]

• Page 3, Line 100: The sentence “This change has brought about” does not look right.
• Page 3, Line 108: the phrase “In the recent STEM field” is not common/correct.

Response: Thank you for pointing this out. We have revised the mentioned sentence to reflect your opinion.

• Page 3, Line 111: what sort of design is in the phrase “Research to advance the design”.

Response: The design we wanted to describe in that sentence was "course design". We have added a "course" so as not to confuse the reader. [line number: 127]

• Page 3, Lines 112-113: “Upon reviewing the literature, we found a report about the positive effects of education through VR”, why is not this report referenced? 

Response: The reason we didn't add a reference to that sentence was because it meant the reference mentioned in the previous sentence. Removed the "Upon~" link to avoid confusing the reader. [line number: 128]

• Page 3, Line 115: please define HMD in the text (head-mounted displays) and CAVE (Cave Automatic Virtual Environment).

Response: We think we provided confusion by abbreviating the full name of the HMD instead of expressing it in the text. For this reason the HMD was created on that page as Full name. [line number 131-132]

• Page 4, Line 181: please define the acronym OOP (Object Oriented Programming).

Response: OOP is mentioned in the Literature section. [line number: 189]

• Page 5, Line 214: please revise the phrase “they guide them to propose”.

Response: Thank you for pointing this out. We have revised the mentioned sentence to reflect your opinion. [line number: 252-254]

• Page 5, Line 228: please define the acronym IR (Infra-Red?).

Response: Infra-red is the correct IR used in the text. We provided confusion by abbreviating the IR's full name instead of expressing it in the text. For this reason, the IR is written on that page as the full name. [line number: 268]

• Page 7, Line 276: please add “T” to “BL”.

Response: We have confirmed that the T is missing from the words in the text. For this reason, we have revised this term. [line number: 316]

• Please provide further explanation of the concept proposed in Figure 2. Also, for Table 2, please explain the codes: TM, CCT, etc. Why not also use the same codes in Figure 7?

Response: Thank you for your good comment. We have identified that Tables 2 and 3 of this paper are somewhat unfriendly to the reader. We wrote the full name and abbreviation for the Category in the Questionnaires section and Table 2. [line number: 374-427, page number: 12, 13, 15, 16,17]

• Page 8, the caption of Figure 4: please capitalise “bloom”. 

Response: Like your comment, we're a Taxonomy created by a researcher named 'Bloom', so we agree to capitalize the first letter of words. Therefore, we capitalized 'Bloom' in the captions of Figures 4 and 8.. [page number: 9, 18]

• Page 9, Lines 322- 323: please check the sentence “Table 1 shows the results of a survey of students before attending courses regarding whether they meet the prerequisites and their VR experience”

Response: Thank you for pointing this out. We have revised the text to clarify what we want to convey. [line number: 365]

 

Reviewer 3 Report

This study presents a teaching methodology for understanding Virtual Reality and application development in engineering major.

The manuscript presents in an adequate manner the research methodology. There are enough and recent references cited.

Comments:

Please remove the final statement from the abstract (lines 11-13). It should contain the main conclusion and not a repetition of the paper aim.

The authors can provide more details about the nine students who attended the course (age, specialization).

Which are the prerequisite information and VR experience that the authors considered for the initial evaluation? How initial VR experience was analysed?

Please explain the meaning for TM, CCT, IP, DEV, ENG, MOT, COG, PLE used in Table 2, Table 3 and so on.

Discussion section should discuss the obtained results and correlate them with the previous studies. This section should include the limitations and future work.

Please remove the lines 602-605.

Please replace the phrase ”In this paper, we propose…” (lines 645-647) with a general conclusion.

Author Response

Response to Reviewer 3 Comments

Thank you for giving me the opportunity to submit a resubmitted draft of my manuscript titled Teaching Methodology for Understanding Virtual Reality and Application Development in Engineering Major to Sustainability. We appreciate the time and effort that you and the reviewers have dedicated to providing your valuable feedback on my manuscript. We are grateful to the reviewers for their insightful comments on my paper. We have been able to incorporate changes to reflect most of the suggestions provided by the reviewers. We have highlighted the changes within the manuscript.

Here is a point-by-point response to the reviewers’ comments and concerns.

We found a crack in the submitted manuscript template form. For this reason, we have downloaded and rewritten at latest sustainability form. Some line numbers in the revised version may not match the numbers created in the review, but we would like to inform you that the content of the text was created with the version before the revision.

The paper is well orgnaized and presents a new analysis for teaching approach. However, the follwing points are recommended to be discussed to improve the paper. 

Comment 1: Please remove the final statement from the abstract (lines 11-13). It should contain the main conclusion and not a repetition of the paper aim.

Response: Thank you for pointing this out. We agree with you and have deleted that sentence. [line number: 20-22]

Comment 2: The authors can provide more details about the nine students who attended the course (age, specialization).

Response: Thank you for your good comment. We considered providing information about class participants. However, we wanted students to genuinely provide positive or negative feedback to their courses through the in-depth interviews. However, the students were reluctant to mention their specificity and demanded anonymity. For this reason, it seems complicated to disclose information such as age. But we think the information you mentioned, specialization, can be replaced with the prerequisite. We ask for your understanding in this situation.

Comment 3: Which are the prerequisite information and VR experience that the authors considered for the initial evaluation? How initial VR experience was analysed?

Response: This methodology was validated in the department of electrical engineering curriculum during our graduate course. Prerequisites are categorized based on the "Introduction to display" subject in the department of electrical engineering bachelor's course. And the VR experience differentiated whether the user had the experience of actually experiencing it.

Comment 4: Please explain the meaning for TM, CCT, IP, DEV, ENG, MOT, COG, PLE used in Table 2, Table 3 and so on. 

Response: Thank you for your good comment. We have identified that Tables 2 and 3 of this paper are somewhat unfriendly to the reader. We wrote the full name and abbreviation for the Category in the Questionnaires section and Table 2. [line number: 374-427, page number: 12, 13, 15, 16,17]

Comment 5: Discussion section should discuss the obtained results and correlate them with the previous studies. This section should include the limitations and future work. 

Response: Thank you for pointing this out. Limitations and Future works wrote in Section 5 of the existing manuscript. The weakness of our paper is correct in that there is no comparison of results with previous studies. We performed both quantitative and qualitative analyses. This does not mean that the Questionnaire item we used has not been validated. Many papers have used these items to compare educational or VR experience effects. We decided that a quantitative analysis would be difficult to have statistical significance due to the small number of people who participated in the experiments to validate the proposed method. Also, since we did not use the same categories and questionnaire, we decided it was inappropriate to compare them on the same line.

For this reason, the discussion in the manuscript focuses on qualitative analysis through in-depth interviews rather than quantitative analysis. Finally, we chose a method that complements the proposed method by advancing self-feedback based on experimental data, considering that it is difficult to compare with other previous studies. In future research, we will use more universal evaluation methods to ensure the credibility of the papers to complement the problems that have arisen in this paper.

Comment 6: Please remove the lines 602-605. 

Response: Thank you for pointing this out. It was confirmed that unwanted sentences were added together in the process of converting the files. Like your comment, we deleted that sentence immediately. [line number: 666-668]

Comment 7: Please replace the phrase ”In this paper, we propose…” (lines 645-647) with a general conclusion.. 

Response: Thank you for pointing this out. We have modified the last sentence of the conclusion to look like the general conclusion form. [linen number: 711-714]

In addition to the above comments, all spelling, grammatical errors and journal forms pointed out by the reviewers have been corrected.

We look forward to hearing from you in due time regarding our submission and to respond to any further questions and comments you may have.

 

Round 2

Reviewer 2 Report

The reviewer thanks the authors for the modifications they made. The limitation of time is not an obstacle as the journal is flexible and further extensions can be requested.

Up to the moment, the long comparison and explanations of problem-based learning and project-based learning do not contribute much to the research problem formulation. Please reduce this.

Why is it needed to analyse thoroughly the work of Ref#47? please shorten it.
Please add a summary paragraph at the end of the literature review outlining the knowledge gaps obtained from the literature review.

The conclusion should better highlight the findings and recommendations that build on the findings.

The paper's English and punctuation need to be revised.

Author Response

Overall response: The reviewer thanks the authors for the modifications they made. The limitation of time is not an obstacle as the journal is flexible and further extensions can be requested.

Up to the moment, the long comparison and explanations of problem-based learning and project-based learning do not contribute much to the research problem formulation. Please reduce this.

Response: Thank you for pointing this out. We partially reduced the content related to project/problem-based learning in the Introduction section. In this process, we classified and rewritten references that refer to the same features and problems.

[Line number: 49-81]

Why is it needed to analyse thoroughly the work of Ref#47? please shorten it.

Response: We revised the specific process of Reference 47 and revised it to mention only the results of the study.

[Line number: 175-183]

Please add a summary paragraph at the end of the literature review outlining the knowledge gaps obtained from the literature review.

Response: Thank you for pointing this out. As in your comment, we have deleted some unnecessary sentences and added a paragraph that summarizes the previous studies and explains the problems.

[Line number: 231-251] Most of the previous research focused on learning methods to understand 3D representations based on computer graphics theory or to implement 3D modeling based on architectural theory. However, these studies except the theory about hardware and do not present it as an issue. For students to fully understand and apply VR technology, it is necessary to include a course that understands the principles and courses of HMD creating IVE in the course. For this reason, VR courses should be designed to understand the technical background and process. However, it is common for most classes to use VR as a single application or environment to harness the training potential of immersive technology [48,49]. Some classes also teach VR-related hardware or software perspectives. However, most courses are biased toward either hardware or software [50]. For this reason, existing VR-related course methods do not meet current educational and industrial requirements [51]. Therefore, we proceed with a course design that can develop a holistic understanding and application ability to face VR technology in this paper.

 

The conclusion should better highlight the findings and recommendations that build on the findings.

[Line number: 754-755, 761-763] The proposed method is structured so they could implement their desired virtual environment in software based on what they understood about the hardware-based IVE generation process. As a result, we confirmed a difference in the questionnaire results of students who took prerequisite courses and those who did not.

 

The paper's English and punctuation need to be revised..

Response: After reflecting all the corrections, we finally checked the grammar and punctuation.

Author Response File: Author Response.docx

Reviewer 3 Report

The authors took into account the recommendations/suggestions  and improved the paper.

Author Response

We send the final version that reflects the corrections of other reviewers. After reflecting all the corrections, we finally checked the grammar and punctuation.

Author Response File: Author Response.docx