Optical Rules to Mitigate the Parallax-Related Registration Error in See-Through Head-Mounted Displays for the Guidance of Manual Tasks

Round 1

Reviewer 1 Report

This paper discusses the use of head-mounted displays (HMDs), particularly see-through HMDs, for tasks like guiding manual surgical procedures. It addresses the challenges related to the alignment of real and virtual content in the user's view and the potential solutions to mitigate registration errors. While head-mounted displays offer many benefits for tasks like surgical procedures, there are notable drawbacks and challenges, including issues related to alignment, optics, calibration, rendering performance, and customization limitations. These drawbacks need to be carefully considered and addressed to make HMDs effective tools for specific applications.

1.       Parallax-Related Misalignment: One of the main challenges is the parallax-related misalignment between real and virtual content. This can hinder the coherent and simultaneous perception of the information presented by the HMD, potentially leading to errors in tasks like surgical procedures.

2.       Optical Distortions: The use of additional lenses to mitigate registration errors can introduce focus distortions in the user's real-world view. These distortions could compromise the correct execution of tasks, even if the spatial augmented reality (AR) registration is perfect.

3.       Calibration Requirements: To ensure accurate alignment between the user's view and the virtual information displayed on the HMD, proper eye-display calibration is required. This process can be time-consuming and may need to be repeated periodically to maintain accuracy.

4.       Rendering Performance: In tasks that involve quick movements of objects or the user's head, the rendering frame rate and rendering latency become critical. Inadequate rendering performance can introduce dynamic sources of registration error, impacting the user's experience and task performance.

Author Response

I thank you for the comments regarding the rationale and key findings of our research paper. 

Reviewer 2 Report

Authors showed additional magnifier lens in front of waveguide optics. However, authors do not show any simulated or measured results. Therefore, authors need to revise the manuscript according to the comments as below.

1. Please use abbreviated journal names in the Reference section. Please see the author guidelines of MDPI.

2. Figures 1 and 2 image quality needs to be improved because the fonts are not clearly shown due to small size.

3. Any kinds of the simulated or measured results of the optical system need to be provided.

4. There are no comparison data or Table to show competitive performance or advantages.

5. Except DOF performance, authors might need to provide other optical parameters such as optical aberrations or resolution.

Author Response

Authors showed additional magnifier lens in front of waveguide optics. However, authors do not show any simulated or measured results. Therefore, authors need to revise the manuscript according to the comments as below.

Reply: Thank you for your valuable feedback that has helped us refine our paper. In particular, we have incorporated your suggestion to emphasize the key message at the end of the introduction, which now states:

“This new paper is a roadmap within our two approaches, not previously compared, to describes and evaluates the pros and cons of both the suggested solutions.

In particular, section 2 describes how to obtain the result with HMDs that projects the virtual imaging plane directly at arm’s distance, while section 3 a sub-optimal solution based on the integration on the HMD of additional lenses to optically move the image of the observed scene to the virtual projection plane of the HMD.”

1. Please use abbreviated journal names in the Reference section. Please see the author guidelines of MDPI.

Reply: Thank you for bringing this to our attention. However, we do not understand why you state that our reference section is not compliant with the MDPI's style guidelines since we used the official MDPI latex template.

2. Figures 1 and 2 image quality needs to be improved because the fonts are not clearly shown due to small size.

Reply: Thanks for your comment. We increased the font size.

3. Any kinds of the simulated or measured results of the optical system need to be provided.

Reply: Thank you for your insightful feedback. We have incorporated your suggestion to explicitly cite our previous work, Cutolo2020 and Ferrari2022, in both Section 2 and Section 3. We stated that “as we already demonstrated in the experiments reported in Cutolo2020/Ferrari2022”.

4. There are no comparison data or Table to show competitive performance or advantages.

Reply: We introduced the table at the end of section 3 comparing the pros and cons of both approaches.

5. Except DOF performance, authors might need to provide other optical parameters such as optical aberrations or resolution.

Reply: In addition to the DOF, we also compared models of the parallax-related registration error in Equations 1 and 6/7. The possible optical aberrations have been now reported in the table.

Thank you for your comment about the display resolution. We have now introduced an additional paragraph in Section 4 to address this concern:

“Another technological point that could impact on the total registration error is the display angular resolution that spatially quantize the virtual information; this contribution is negligible in modern OST display with a number of pixels per degree close to 60, the maximum human visual acuity”

Reviewer 3 Report

The comments and suggestions for authors are listed in the file attachment. 

Comments for author File: Comments.pdf

Author Response

The authors investigated the optical rules to mitigate the parallax-related registration error in seethrough head-mounted displays(HMDs) for the guidance of manual surgical procedures. Results and discussions are interesting and helpful for understanding solutions to mitigate the parallax-related registration error of HMDs by analytical estimation. This paper can be published in Multimodal Technologies and Interaction if the following issues can be addressed

1. All abbreviations should be clarified when they appear for the first time in the abstract (i.e., OST) even though it has been mentioned in the main text.

Reply: thanks, we checked and corrected all the abbreviations.

2. What is the π symbol in Figure 1? Add the information of that physical quantity symbol in the caption of Figure 1 or the main text.

Reply: thanks, done.

3. To provide the readers with a clearer visual interpretation of Figure 2, add some proper physical parameters (e.g., f, d, etc.) in Figure 2 according to the optical rules as explained in section 3. In addition, please improve the image resolutions of Figures 1 and 2.

Reply: We have completely redone the pictures, including a third one.

4. How much is the p value in the calculation examples (as mentioned in page 4 lines 168-170, page 5 lines 171-176, and page 6 lines 219-221) using equations 1,2, 3, and 7? Add the information of that parameter value in the main text.

Reply: done.

5. I suggest paying particular attention to writing consistently of units and physical parameters writing (italic or non-italic) on page 5 lines 173-176.

Reply: thanks, done.

6. To help the readers have a more comprehensive understanding of the new research on augmented reality (AR), I suggest supplementing some latest works about AR [Photonics Research 10(1), 21-32 (2022); Laser & Photonics Reviews 16(6), 2100638 (2022); and Photonics Research Vol. 10, Issue 12, pp. 2809-2815 (2022)].

Reply: thanks for the suggested citations. We inserted Photonics Reviews 16(6), 2100638 (2022); 

Round 2

Reviewer 2 Report

Authors carefully defended the questions raised so the manuscript can be accepted as it is.

Author Response

We thank the reviewer for his/her thoughtful review and acceptance of our paper.

Reviewer 3 Report

Most of the replies satisfy me, but they should have more references involving VR in the revised manuscript. Therefore, the manuscript lacks adequate literature review.

Author Response

Thank you for the comment. You mentioned the need for additional references about VR. We understand that you meant to request additional references about AR. We have therefore updated the manuscript to reflect the correct requirement for additional references about AR:

• DOI: https://doi.org/10.37188/lam.2021.024 – A new review on the design of OST HMDs.

• DOI: https://doi.org/10.3390/jimaging8070203 – A new MPDI comprehensive review about OST-AR displays in surgery.

• DOI: 10.1016/j.media.2022.102361  – A review that highlights perceptual issues about the technology.

We believe that these references will strengthen the manuscript and provide a more comprehensive overview of the existing research on AR.

Regards