Descriptive Markers for the Cognitive Profiling of Desktop 3D Spaces

Round 1

Reviewer 1 Report

This paper compares desktop 3D and 2D environments for performing research tasks and answering survey questions on those tasks. The paper is interesting. I have some issues with the use of the term Desktop VR in the title and initial sections, while the term Desktop 3D is used in the bulk of the paper and seems more accurate.

 There are some issues with the description of the tasks users were asked to perform. Most notably, there are no images of the 2D interface and the link in the footnotes no longer works. It is very difficult for us to judge whether the design of the 2D interface has influenced the results without being able to see it. The description of the 2D interface sounds very artificial, so this is important.

The study has a small number of participants, making it difficult to judge the results. This may also be why there were so few statistically significant differences found. Looking at the completion time chart, I would have expected 2 or 3 of those to be statistically significant.

This completion time seems the most interesting finding, but again it is hard to judge this without detail of the 2D interface. I suggest the authors add these missing details.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

 

The authors Anna Sudar and Adam Csapo’s manuscript “Descriptive Markers for the Cognitive Profiling of Desktop VR Spaces” that reviewed previous studies on virtual reality immersive environments compared to 2D environments for task performance. In this study the authors compared a 2D web-based interface to an immersive 3D environment for four astronomy tasks by using eye-tracking and questionnaires. However, the description of the experiment was not adequate to understand the manuscript. I am recommending major revisions to improve the readability and description of the experiments. I have the following comments:

1. In the literature review, in section 2.2 the authors discuss several papers about improving recall. I recommend the authors also include the following reference:  Krokos, Eric, Catherine Plaisant, and Amitabh Varshney. "Virtual memory palaces: immersion aids recall." Virtual Reality 23.1 (2019): 1-15.

 

2.       Figure 2 shows one of the blocks. However, it is not clear what the task is the user conducts at this block. Similarly, the authors to not explain the tasks for the other blocks. It is difficult to understand the results presented without understanding the user tasks. I recommend the authors add a more detailed description of the user task for each of the 4 modules. In addition, it is unclear if the blocks look the same for the 2D and 3D versions. Please clarify any differences in the blocks.

 

3.       Figure 3 shows the results of the questionnaire, but the questions from the questionnaire were not provided. Therefore, the reader cannot draw any conclusions from the results. Please include the questionnaire in the manuscript or supplemental materials.

4.       Figure 9 shows the results from the user’s pupil diameter. Was the same experimental setup, ambient lighting, monitor used for both the 2D and 3D tests? Similar to comment #2, please also clarify if the same scene was shown for both the 2D and 3D testing. If the scene was brighter in 2D compared to 3D, this could also contribute to the pupil size.

5.       In section 2, some of the sentences are difficult to follow due to the number of clauses. I encourage the authors to try to simplify some of the sentences to improve the readability.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

After reviewing the manuscript, I believe that it must undergo a series of major revisions before I can assess its publication in this journal.

First, the authors must justify that the number of subjects participating in the study is sufficient to establish conclusive data. At first glance, 9 (3d) and 7(2d) seem few to me. However, possibly the type of evaluation performed does not require a larger number of volunteers, in which case, this should be commented on and justified.

Second, the results need to be further detailed. Figures that appear in the text should also be commented and clarified in the text to provide greater clarity to the readers.

Something similar should happen with the discussion section, where the most relevant data should be compared with previous studies developed by other authors.

Likewise, every scientific article should include a "conclusions" section specifying the findings and the scientific contribution of the work.

Finally, the authors talk about having implemented informed consent. Do the authors have any type of endorsement or support from any institution? If not, perhaps it should be indicated what this informed consent consisted of.

 

 

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have done a good job in responding to my comments and the new version seems suitable for publication.

Author Response

Thank you very much for helping to improve the quality of our manuscript.

Reviewer 2 Report

The authors have addressed my comments and improved both the description of the experiment and the readability of the text. I do have one additional question, was the prior-knowledge of the participants about astronomy captured in the study?

Author Response

Thank you very much for helping to improve the quality of our manuscript.

Regarding your question: indeed, there was no assessment of the subjects' prior knowledge of the field of astronomy. However, we did observe based on the videos that none of the subjects proceeded to fill out the questionnaires immediately, without first studying the accompanying learning materials. This suggests that the subjects did not directly rely on prior knowledge.

To clarify this point, we have made slight changes to the paper in two places (lines 410 and 525, which you will see highlighted in red in the re-submitted manuscript). The second place, starting on line 525, adds this issue to the list of possible improvements in a future follow-up study.

Reviewer 3 Report

I understand that the modifications made by the authors are sufficient for the publication of the article.

Author Response

Thank you very much for helping to improve the quality of our manuscript.