Examining User Perception of the Size of Multiple Objects in Virtual Reality

Round 1

Reviewer 1 Report

This article proposes the relationship between object size determination and object color in a virtual reality environment. After setting the test environment, two virtual objects (reference/adjustment) are used to collect user test data. After comparison and analysis, the experimental results are roughly confirmed the research purposes, however, too few samples are a problem.

1. There are at least 30 samples in the relevant analysis and research (Gay, 1992). There are only 10 subjects in this study. The number of samples is small. It is necessary to explain how the 10 samples in this study are representative.

2. Only a single specification HMD was used for the experiment, and the size of the virtual object for testing was not large. The result showed that the error of size discrimination was not large. The lack of the effect of different resolution HMD on size discrimination. For example, the high resolution of HTC VIVE Pro may affect the accuracy of judgment.

3. In chromatist, colors are divided into warm colors and cool colors. Objects that have different hue but have the same color are not studied. It is not clear whether color or value has an impact on size discrimination. It should be explained.

4. The detailed description of the computer equipment is superfluous. (line 242-line 247), suggest modification.

5. In Conclusion, ” This finding supports the concept of users being able to understand the size of different virtual controls that have different colors and are made of different materials.” (Line 331-line 333). There is no comparison of different materials in the experiment, such as metal, rubber, or luminous objects. Obviously, it is excessively extended. In addition, only two-color objects are compared, so it is necessary to express the reason for all colors.

All in all, although this experiment has a good focus, the sampling rate is low, and the conclusion is slightly optimistic.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper investigates whether virtual objects of different colours appear to be different sizes to the user in a VR HMD.

 

The experiment is conducted entirely with an HTC Vive Pro HMD, and the participant is presented two 182 cylinders that are approximately 500mm in front of them and 300mm below eye level. The cylinder on the left is the target (standard) and is of red colour. The cylinder on the right is the cylinder the participant is adjusting (comparison). The goal of the experiment is to determine the point of subjective equality and just-noticeable difference for users observing two cylinders that approximate buttons, dials, and levers on a control panel.

JND height in mm for height judgements: 

Height Judgement for tasks of 10mm to 90mm for Height Point of Subjective Equality

 

In the case of width judgement tasks also ranging from 10mm to 90mm, they were all less than 1.4mm or less than 2% of the target width.

 

height judgement task for just-noticeable difference were all less than 1.5mm.

 

As compared to the following paper:

“An Evaluation of Depth and Size Perception on a Spherical Fish Tank Virtual Reality Display”. IEEE Transactions on Visualization and Computer Graphics 2019.

 

“Using the spherical display, we found significantly that users had 1cm depth accuracy compared to 6.5cm accuracy using the flat display on a depth-ranking task. Likewise, their performance on a size-matching task was also significantly better with the size error of 2.3mm on the spherical display compared to 3.1mm on the flat display. “

 Therefore, in terms of originality and novelty: this paper has minor contribution.

 

Now, the question is why so similar in terms of error for width estimation?

 

The results for  height judgement task for just-noticeable difference were all less than 1.5mm and for the width judgement task less than 2.3mm.

 

Very similar results are shown both for the HTC Vive Pro HMD and Fish Tank Virtual Reality Display.  The authors are encouraged to explain why?

 

Comments for author File: Comments.docx

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

This paper explores users' perception of of objects in VR when multiple options are presented in different colors. In my opinion, the paper makes a small but necessary contribution to the field. 

My main complaints with this article are regarding presentation. It is very awkward to have the experiment setup explained in Section 2, but then talk about recruitment and demographics in Section 3 (Results). This leads to a lot of jumping back and forth. 

Also, regarding sample size, I would argue that it is better to use something more directly related to appropriate sample sizes for the field (e.g.. https://doi.org/10.1145/2858036.2858498) than the papers cited. 

The conclusion format is also something that I would like to see reconfigured. This is a personal preference, but it is easier to read if the results are directly tied to each question in format. 

The figures need to be recaptioned and made to be uniform. Figure 12, for example, should include "Just Noticeable Difference" instead of just "JND" like Table. 5. I would like to see expanded text for all of the Tables and Figures. 

Finally, I would like to see error bars included in Figures 9 and 10. 

My last quibble is that there should be error bars in 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

Clarification:

[89] please explain L and T configurations.

[182] what does approximately cover? Wouldn’t it be necessary for it to be accurate to conduct the study?

[experiment]: please provide demographical data on subjects. The group is skewed (gender), broad (age), small (size), but nowhere is it mentioned, if participants were in fact familiar with VR, had any usability issues – or established whether other factors could implicate the study (such as motion sickness). In my opinion, it would be a necessity to characterize the participants much more.

[experiment] cylinder, shape vs colour: The argument for using cylinders has escaped me while reading the article. Please provide an argument for cylinder – also if it could just as well have been another shape. With this point in mind I encourage the authors to change their title to reflect ‘cylinders’ and not ‘multiple objects’.

Rephrase:

“paper” into “article” (already in abstr. and intro.). It’s a journal and not a conference.

Comments: the related works section is adequate and the paper, in general, is well-written and clear. I find the study very relevant, but I am apprehensive in accepting it due to unclear subject data as well as a very small sample size. With respect to the latter point I encourage the authors to increase the test sample or at least indicate to the reader how to best interpret the impact by the results.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 4 Report

I am satisfied with the changes indicated by the authors.