Implementing Affordance-Based Design Review Method Using Virtual Reality in Architectural Design Studio

Round 1

Reviewer 1 Report

This paper presents a VR-based approach for architecture design affordance review. Several affordance review techniques were reviewed and employed. A total of 58 students were recruited to participate in the design of two projects. The results showed added benefits of the proposed method. In general the manuscript is well written. The method is based on reasonable discussions of relevant theories. Several improvement suggestions for the authors to consider:

1. Tone down the argument that previous VR review studies only relies on fuzzy and vague evaluation measures. This is not true. There is a big and still growing body of literature focusing on soliciting specific knowledge from domain experts. Many quantitative approaches, in addition to the qualitative methods, are used. The contribution of this work could be proving the benefits of VR for affordance review, which is a critical component of design cognitive process. The authors could also argue that this work added additional evidence on the role of VR for review.

2. Some of the design parameters can hardly be captured by VR, such as "noise cancelation". The authors should better justify how they selected the list of design parameters for the two studies. 

3. The presentation of results was mainly based on average and percentage values. More strict statistical analyses must be applied, such as ANOVA or nonparametric cross-group comparison depending on the features of the data. 

4. The conclusions can be improved to better connect to the main hypotheses in a clear way. The current conclusion section seems to be too simple although the authors tried to cover many lessons learned in the discussion section. 

Author Response

Responses to Reviewer 1's Comment

Point 1: Tone down the argument that previous VR review studies only relies on fuzzy and vague evaluation measures. This is not true. There is a big and still growing body of literature focusing on soliciting specific knowledge from domain experts. Many quantitative approaches, in addition to the qualitative methods, are used. The contribution of this work could be proving the benefits of VR for affordance review, which is a critical component of design cognitive process. The authors could also argue that this work added additional evidence on the role of VR for review.

Response 1: Thank you very much for your critical comment. We agree with you. So, we set the tone down by arguing that this work could add further evidence on VR utilization for an architectural design review process.

Point 2: Some of the design parameters can hardly be captured by VR, such as "noise cancelation". The authors should better justify how they selected the list of design parameters for the two studies.

Response 2: The design parameters were obtained from the studio objectives and SPCs using content analysis. One of the primary references is design objectives from the Whole Building Design Guide. Some of the parameters might be hardly captured by VR. But since the studio objectives and SPCs require it, those parameters were interpreted differently based on their perceptions during the simulation. The explanation is mentioned on lines 375-377 in the revised manuscript.

Point 3: The presentation of results was mainly based on average and percentage values. More strict statistical analyses must be applied, such as ANOVA or nonparametric cross-group comparison depending on the features of the data.

Response 3: Except for the PDS Process, other data analysis processes are done with more strict statistical analyses such as correspondent analysis followed by cluster analysis using dendrogram and paired t-test. In the revised manuscript, we present those data analysis processes more transparently as explained further on lines 645-648, 687-692, 730-738, Table 9, Table 13, and Table 16, It is also possible to perform ANOVA to compare the result between participants to seek new findings. The results from ANOVA will be discussed in the future manuscript. But for the current submission, we focus on the presented data analysis process.

Point 4: The conclusions can be improved to better connect to the main hypotheses in a clear way. The current conclusion section seems to be too simple although the authors tried to cover many lessons learned in the discussion section. 

Response 4; The conclusions section is improved by describing additional findings and more straightforward explanations that connect with the hypothesis in the revised manuscript on lines 801-811.

 

Reviewer 2 Report

Overall the paper is well written and structured. The literature is almost complete. The methodology is well discussed and structured, and it is appropriate to answer the research questions. I suggest only a few edits in the literature review.

The introduction is well written and frames the paper correctly.

The literature review needs to be expanded to include key research in the field of design review and virtual reality by Kandi, Brittle et al. 2020, Mastrolembo Ventura et al. 2020, Liu, Messner 2020, Castronovo Oprean 2017, Castronovo et al. 2013, and the foundational work by Shirattudin.

The research is well structured and explained. The sample size is adequate for publication. The methods are appropriate for the study and the analysis is sound.

Great paper!

Author Response

Responses to Reviewer 2's Comments

Point 1: The literature review needs to be expanded to include key research in the field of design review and virtual reality by Kandi, Brittle et al. 2020, Mastrolembo Ventura et al. 2020, Liu, Messner 2020, Castronovo Oprean 2017, Castronovo et al. 2013, and the foundational work by Shirattudin.

Response 1: Thank you for the suggestion. We have read all mentioned works. Those works are suitable to expand the literature review in our work. So, we have added them in the revised manuscript on lines 65, 66, 88, 162, and 264,

Reviewer 3 Report

This is a very well-organized research paper/article with scientific approach. It is a little bit hard to read/digest due to the theory of affordance (not common for architecture; was originally from user experience research) and lots of acronyms. Please spell out the entire term when first time using it. For example, “VE = virtual environment” indicated on line 407. However, the first time showing “VE” is on line 267 without further explanation. On lines 110-112, you quoted there are 3 behavioral constraints from Norman’s book, but I only can see one, physical constraint.

The research objectives and methodology (chapter 2) and case study (chapter 3) portions are very clear. Chapter 4 and 5 are way too “engineering”, especially the Affordance Identification and ASM Mapping Process. I wish to see some real feedback from the users (students) about their experience of using different VR headsets/tools/methods. In general, it is a decent research paper but not easy to read.

Author Response

Response to Reviewer 3 Comments

Point 1: This is a very well-organized research paper/article with scientific approach. It is a little bit hard to read/digest due to the theory of affordance (not common for architecture; was originally from user experience research) and lots of acronyms. Please spell out the entire term when first time using it. For example, “VE = virtual environment” indicated on line 407. However, the first time showing “VE” is on line 267 without further explanation. 

Response 1: As suggested in your review, we have revised them by spelling out the entire term when it is used for the first time. For example, the VE acronym is mentioned for the first time in line 33. So, its full term is written first in line 33, and then the acronym is used.

Point 2: On lines 110-112, you quoted there are 3 behavioral constraints from Norman’s book, but I only can see one, physical constraint.

Response 2: Thank you for the finding. In the revised manuscript, we mention all three behavioral constraints from Norman's book in lines 111-115.

Point 3: The research objectives and methodology (chapter 2) and case study (chapter 3) portions are very clear. Chapter 4 and 5 are way too “engineering”, especially the Affordance Identification and ASM Mapping Process. I wish to see some real feedback from the users (students) about their experience of using different VR headsets/tools/methods. In general, it is a decent research paper but not easy to read.

Response 3: In this study, we collected feedback solely based on the affordance they perceived using the Affordance Structure Matrix. It is a good idea to gather general feedback from the participants regarding their experience using different design review methods in future studies.