Motion Sickness in Mixed-Reality Situational Awareness System

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Despite its rich experiments, the paper lacks innovation. It primarily relies on widely adopted assessment tools, specifically the SSQ and MSQ, to evaluate user experiences and analyze the impact of various factors such as adaptation and blur on motion sickness. While these analyses hold some value, they have already been extensively explored in prior research. Regrettably, this paper does not introduce new viewpoint or innovation method.

Author Response

REVIEWER 1
Despite its rich experiments, the paper lacks innovation. It primarily relies on widely adopted assessment tools, specifically the SSQ and MSQ, to evaluate user experiences and analyze the impact of various factors such as adaptation and blur on motion sickness. While these analyses hold some value, they have already been extensively explored in prior research. Regrettably, this paper does not introduce new viewpoint or innovation method.

 

We appreciate your perceptive remarks on our text. We are grateful for the chance to clarify the value of our study, which may not have been adequately emphasized in our initial proposal.

Novel Application of Established Tools: Although we utilized well-known tools like the SSQ and MSQ, the novelty lay in their innovative implementation within the framework of MRSAS. This application is somewhat unexplored, and our study is the first to analyze motion sickness in this specific scenario. Additionally prior studies have barely tested different parameters and their effect on MS over time.

 

Our research surpasses the utilization of pre-existing technologies. We have developed a new approach for normalizing SSQ using MSQ. This methodology enables a more comprehensive comprehension of motion sickness by considering differences in individual susceptibility, which represents a notable improvement compared to conventional methods.

The research provides a thorough investigation of several parameters including adaptation, range of vision, blur, vignette, distortion, and lighting. The level of analysis conducted, particularly within the context of MRSAS, is more in depth than most of the research in this field. We examined these characteristics in both simulated and real-life environments, offering a comprehensive perspective on their influence on motion sickness.

Our research provides novel insights into the process of user adaptation to VR settings over time, which has not been extensively explored in other studies. Having this comprehension is crucial for developing MRSAS apps that are easy for users to navigate and utilize.

Our findings have practical implications for the future advancement of MRSAS, including suggestions that can enhance user experience and mitigate motion sickness. This significantly influences the design and functionality of these systems in several domains.

We strongly feel that our study makes a valuable contribution to the area by using current techniques in a unique environment, creating new approaches, and offering complete insights that are essential for the advancement of MRSAS technology. Prior to this study, we spent considerable time and effort looking for any relevant research to improve our system. Most studies we found were superficial, covered non-MRSAS applicable MS or only reported a single MS datapoint in time. We believe this elucidates the pioneering elements of our study and its importance to the area.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

In this work authors worked on enhancing the user experience within a Mixed Reality Situational Awareness System (MRSAS). They used the Simulator Sickness Questionnaire (SSQ) to access the user experience. Effect of various system parameters including Field of View (FoV) of the headset, peripheral and general blurring, camera distortions, camera white balance and users’ adaptability to VR over time is studied on the users.

This paper is not mainly dealing with the science of MR systems but targets effects of system parameters on the motion sickness of the user. Thus, this reviewer feels that the manuscript is not fit in the scope of applied sciences and authors should submit it to more relevant journal dealing with perception or psychology.

Author Response

REVIEWER 2

In this work authors worked on enhancing the user experience within a Mixed Reality Situational Awareness System (MRSAS). They used the Simulator Sickness Questionnaire (SSQ) to access the user experience. Effect of various system parameters including Field of View (FoV) of the headset, peripheral and general blurring, camera distortions, camera white balance and users’ adaptability to VR over time is studied on the users.

 

This paper is not mainly dealing with the science of MR systems but targets effects of system parameters on the motion sickness of the user. Thus, this reviewer feels that the manuscript is not fit in the scope of applied sciences and authors should submit it to more relevant journal dealing with perception or psychology.

 

 

We appreciate your perceptive remarks on the extent of our manuscript. We value your viewpoint about the focus of our study. Our paper extensively explores the impact of system parameters on motion sickness in users of MRSAS. We consider our research to be a suitable contribution to the field of applied sciences, specifically at the convergence of technology and human factors.

Our study goes beyond the scope of perception or psychology; it rigorously investigates how technological factors in MR systems impact human experiences. Applied sciences involve the practical application of scientific knowledge to provide useful answers and progress. Our work provides vital insights on how to improve MR systems to enhance user experiences. This research greatly contributes to the field of applied sciences, particularly in the areas of technology applications and human-computer interaction.

Furthermore, the journal "Applied Sciences," specifically the section on "Computing and Artificial Intelligence," covers a broad spectrum of multidisciplinary subjects. Our work enhances the variety of perspectives by offering a technology viewpoint alongside research focused on human needs and preferences. The study focuses on addressing practical obstacles in the field of MR, providing crucial empirical discoveries that are essential for the advancement of user-friendly and efficient MR systems. This is in accordance with the journal's objective of publishing interdisciplinary research with practical applications.

Our work enhances the applied sciences by connecting technology with user experience, a vital element in creating practical and creative MR applications. We believe that our manuscript contributes to the applied sciences by bridging technology with user experience, a crucial aspect in the development of practical and innovative MR applications. We hope that our response clarifies the relevance and significance of our research within the scope of the journal.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The paper introduces a method to assess the severity of motion sickness symptoms during user interactions with a virtual system. The study utilized the Simulator Sickness Questionnaire (SSQ) and Motion Susceptibility Questionnaire (MSQ), including a real-world counterpart. Various variables were analyzed, including the Field of View (FoV) of the headset, the impact of peripheral blurring, camera distortions, camera white balance, and users' adaptability to VR over time. These variables were specifically examined in the context of Vegvisir devices.

While there is a pressing need for research on VR and its physiological effects on the human body, it is essential to emphasize that the conclusions drawn from this study are exclusively applicable to VR Vegvisir systems. Extrapolating these findings to AR systems or other device types should be avoided without supporting evidence. Kindly confine the discussion and conclusions of the paper to the results obtained in this research.

 

ABSTRACT

The redaction of the following sentence requires modification: “While the experiments in this research were primarily conducted with the goal of improving the physical Vegvisir system, the results themselves are applicable for a broader array of VR/MR awareness systems and can help improve the UX of future applications.” Please adjust the content. Moreover, kindly incorporate a concise summary of the study's results and conclusions into the abstract.

 

INTRODUCTION

The contextualization of Motion Sickness, the mechanisms for its measurement, the easy way to use SSQ and MSQ in comparison to alternative methods, along with the description of the primary cause of Motion Sickness, is excellently written, concise, clear, and easily comprehensible.

It is necessary to provide a more thorough introduction to the term  “Mixed Reality Situational Awareness System (MRSAS)”, because it is mentioned in the abstract and subsequently referred to in the introduction without detailed contextualization, unlike other important related terms. 

METHODS

Given the utilization of SSQ and MSQ in this study, it is important to provide detailed information about these questionnaires.  This could involve incorporating a table or an appendix containing the questionnaire items.  As these questions were answered by the participants in this study, ensuring easy access to them is important for readers. 

Further details about the normalization method are essential. A more extensive and clearer explanation of the reasons behind and the procedures for employing this method is needed.     This explanation should be relocated in the Methods section, instead of the Results section. Please transfer and complete the information provided in lines 281-296.  It is unclear why the s parameter is required, how the equation of line 296 was obtained, and the process involved in creating the graphics presented in Figure 4.

I recommend making the available row and processed data accessible in a public repository.

RESULTS

The results, depicted through graphics, provide a visual representation of key aspects of the outcomes and are easily comprehensible.

In the line 400 you might consider that a correlation with a value of 0,53 is moderated.

DISCUSIÓN

Please review all the redactions to avoid making generalizations that cannot be established from the existing results in this study.   As Augmented Reality (AR) was not included in this study, the discussion and conclusions should not have assertions about AR.  Specialized questionnaires for AR, such as the Augmented Reality Sickness Questionnaire (ARSQ), exist. If there is a desire to include statements about AR, it is recommended to conduct a separate study using the ARSQ questionnaire.

Given that the participants in the study range in age from seventeen to twenty-eight, the conclusions can only be generalized for this specific age range.

The same limitation applies to generalizations for other Head-Mounted Display (HMD) brands.  Since the analysis primarily focused on a comprehensive evaluation of variables using the Vegvisir brand, the results are particularly applicable and valid for this specific brand.

OTHER ASPECTS

It is not customary to include the titles of papers or journals within the article´s text; referencing alone is sufficient.  Please reconsider the redaction for these instances: line 55, lines 82-83, lines 91-93, lines 101-103, lines 110-113.

The title of Section 3 is in uppercase, which differs from the formatting of the other sections. Please review and ensure consistent capitalization throughout.

You may consider condensing the information related to Figure 1. For instance, changing 'imagen a)' to 'a).

Please review all the figures, ensuring that they are referenced before their respective figure numbers.

In the line 303 please remove “the second figure” as it may cause confusion with the figure 2.

In line 400 replace “this chapter” with “this section”.

In the line 172 eliminate doble 'a'

Author Response

REVIEWER 3

The paper introduces a method to assess the severity of motion sickness symptoms during user interactions with a virtual system. The study utilized the Simulator Sickness Questionnaire (SSQ) and Motion Susceptibility Questionnaire (MSQ), including a real-world counterpart. Various variables were analyzed, including the Field of View (FoV) of the headset, the impact of peripheral blurring, camera distortions, camera white balance, and users' adaptability to VR over time. These variables were specifically examined in the context of Vegvisir devices.

 

Thank you for your extensive review of our work. We greatly appreciated every single comment as there were now obvious discrepancies we had missed in our initial submission. We have addressed your comments to the best of our ability.

 

 

While there is a pressing need for research on VR and its physiological effects on the human body, it is essential to emphasize that the conclusions drawn from this study are exclusively applicable to VR Vegvisir systems. Extrapolating these findings to AR systems or other device types should be avoided without supporting evidence. Kindly confine the discussion and conclusions of the paper to the results obtained in this research.

We have added emphasis in the article's abstract and conclusion about the direct application for the Vegvisir system as per the reviewers suggestion.

 

 

ABSTRACT

 

The redaction of the following sentence requires modification: “While the experiments in this research were primarily conducted with the goal of improving the physical Vegvisir system, the results themselves are applicable for a broader array of VR/MR awareness systems and can help improve the UX of future applications.” Please adjust the content. Moreover, kindly incorporate a concise summary of the study's results and conclusions into the abstract.

We have modified the meaning of the sentence and have added a summary of the primary results in the abstract.

 

 

INTRODUCTION

 

The contextualization of Motion Sickness, the mechanisms for its measurement, the easy way to use SSQ and MSQ in comparison to alternative methods, along with the description of the primary cause of Motion Sickness, is excellently written, concise, clear, and easily comprehensible.

 

It is necessary to provide a more thorough introduction to the term  “Mixed Reality Situational Awareness System (MRSAS)”, because it is mentioned in the abstract and subsequently referred to in the introduction without detailed contextualization, unlike other important related terms. 

Thank you for the observation, the term was not expanded nor was a concise description available on a quick query. We have added a new subsection to explain the term and its relevance. 

 

METHODS

 

Given the utilization of SSQ and MSQ in this study, it is important to provide detailed information about these questionnaires.  This could involve incorporating a table or an appendix containing the questionnaire items.  As these questions were answered by the participants in this study, ensuring easy access to them is important for readers. 

The questionnaires have been included in the submission appendix. 

 

Further details about the normalization method are essential. A more extensive and clearer explanation of the reasons behind and the procedures for employing this method is needed.     This explanation should be relocated in the Methods section, instead of the Results section. Please transfer and complete the information provided in lines 281-296.  It is unclear why the s parameter is required, how the equation of line 296 was obtained, and the process involved in creating the graphics presented in Figure 4.

We have expanded the description and rationale behind the normalization method. We have also consolidated the explanation better with the presented figures.

 

I recommend making the available row and processed data accessible in a public repository.

We are currently in the process of formatting the data so it could be properly uploaded. It was the intention to upload the data publicly during the review process so we must apologize for the delay.

 

RESULTS

 

The results, depicted through graphics, provide a visual representation of key aspects of the outcomes and are easily comprehensible.

 

In the line 400 you might consider that a correlation with a value of 0,53 is moderated.

Pearson comparison has been removed from the work due to it being applicable to primarily linear datasets. 

 

DISCUSIÓN

 

Please review all the redactions to avoid making generalizations that cannot be established from the existing results in this study.   As Augmented Reality (AR) was not included in this study, the discussion and conclusions should not have assertions about AR.  Specialized questionnaires for AR, such as the Augmented Reality Sickness Questionnaire (ARSQ), exist. If there is a desire to include statements about AR, it is recommended to conduct a separate study using the ARSQ questionnaire.

We have removed any and all generalizations that would include AR. We have also limited some generalizations of MR that might be conceived as including AR. This study only explored cases in standard VR and real time camera pass through MR.

 

Given that the participants in the study range in age from seventeen to twenty-eight, the conclusions can only be generalized for this specific age range.

We have added the emphasis of the participants' description in the abstract, conclusion and participants section. We have also included a description of the in field testing group that was left vague in the original version.

 

The same limitation applies to generalizations for other Head-Mounted Display (HMD) brands.  Since the analysis primarily focused on a comprehensive evaluation of variables using the Vegvisir brand, the results are particularly applicable and valid for this specific brand.

The emphasis of the study being primarily aimed at the Vegvisir system has been added to the abstract and conclusion. The simulation testing was conducted on the HTC Vive and not the Vegvisir brand HMD. Only the render camera setup that rendered onto the VR sphere and the VR camera FoV were adjusted to mimic the Vegvisir system. 

 

Essentially, some experiments such as the field of view test could also be applicable to other 360 degree pass through camera views in VR, as only the specific passthrough camera placements and settings were influenced by the Vegvisir system. However, we decided not to pursue this aspect of the experiments in this manuscript.

 

OTHER ASPECTS

 

It is not customary to include the titles of papers or journals within the article´s text; referencing alone is sufficient.  Please reconsider the redaction for these instances: line 55, lines 82-83, lines 91-93, lines 101-103, lines 110-113.

The references in the text have been adjusted accordingly.

 

The title of Section 3 is in uppercase, which differs from the formatting of the other sections. Please review and ensure consistent capitalization throughout.

Thank you for the observation, all chapter titles are now uppercase and not just chapters 1-3. Section and subsection titles are capitalized lower case.

 

You may consider condensing the information related to Figure 1. For instance, changing 'imagen a)' to 'a).

The information has been condensed for the offending figure.

 

Please review all the figures, ensuring that they are referenced before their respective figure numbers.

Thank you, we had neglected to reference figures 7 - 10 in text. The proper references have been added.

 

In the line 303 please remove “the second figure” as it may cause confusion with the figure 2.

The term “the second figure” has been removed from the text.

 

In line 400 replace “this chapter” with “this section”.

The term has been replaced.

 

In the line 172 eliminate doble 'a'

The double ‘a’ and other typos in the text have been fixed.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The manuscript describes research conducted to discern the effect of different VR visual parameters on motion sickness (MS). Understanding the contributors to VR MS is of value to the scientific community; however, substantial revisions are necessary for the manuscript to have adequate rigor for publication.

Abstract:

It is unclear what this sentence means: “This simulated setup was adjusted to best match the conditions found in the real system by using post-processing effects.”

Methods and quantitative results should both be presented in the abstract, including number of participants, age, and sex.

Introduction:

There are inadequate references in the introduction, for example the following statements require citations:

As an alternative method, subjective user based questionnaire scores have widely been 10 used and been proven to be reliable.

Two significant instruments employed in this context are the Simulator Sickness Questionnaire (SSQ) and the Motion Sickness Susceptibility Questionnaire (MSQ).

A large contributor to MS is latency in VR and how it breaks immersion.

Motion-to-Picture latency and its effects on MS have been widely explored and is the primary consideration for most VR applications

The following sections are background and belong in the introduction or related work:

Full 360 degree VR videos and the time consistency of stitching the scene has been shown to cause MS [26]. Several methods have already been proposed to alleviate MS in these cases but not all of them have been thoroughly researched.

Menezes et al. suggest a VR environment should be treated as a theater and not a film, this means camera animations and tricks like involuntary zooming and a lack of a reference can have a massive impact on comfort in VR [27]. A virtual body on the other hand has been shown to have a negative impact on motion sickness [28]. Serrano et al. proposed motion parallax to help with MS, but this may not counteract the MS caused by frame delay due to its added computational cost [29].

Methods:

Informed consent was obtained from participants, but it should be stated whether an ethics committee reviewed the study in accordance with the Declaration of Helsinki. If not, there are ethical concerns about the work.

There is inadequate description of the experimental procedures in section 3. For example, the manuscript should explain whether different visual parameters were changed simultaneously or sequentially, what the testing experience was for the subject, and how many subjects participated in each of the tests, including information about age range and sex. Reported MS is expected to increase over time; it is unclear from the methods as described whether confounding variables were considered or what the sequence randomization methodology was.

The manuscript should include details such as parameters for the barrel and pincushion distortions.

It is reported that the majority of infield tests were conducted as 15-minute runs; additional detail is needed on the timing of all testing, such as range and median, rather than the majority.

Test subjects were seated in either the passenger seat or diver seat; it is unclear whether this variation was controlled for or analyzed in separate sets.

The authors state that “The small sample size is considered reasonable when the research has time-intensive research procedures.” It is unclear whether the sample was powered to test a particular hypothesis; rationale should be provided for the sample size determination.

A critical limitation of the work is that participants were ages 17-28; this should be acknowledged in the conclusions or in a limitations section.

The number of test sessions participants experienced should be reported, e.g., N participants experienced 2 test sessions, and the remainder experienced 3.

A description of the “simulation side” and “infield setup” testing is needed. It is reported that a majority of test subjects were male for infield testing, but precise numbers should be given.

The following is unclear and should be described in detail in the Methods section: “Employing linear regression on each participant’s SSQ scores, we quantified the spread of the generated lines to derive a baseline score.”

Results:

Results were often presented as averages. When averages are given, including in plots, the number of subjects contributing to the averages and measures of deviation or dispersion should be included.

Results were mostly presented scaled by user MSQ. All results should be presented both scaled and unscaled, as this is a new approach and unvalidated.

The authors explored exponential and logarithmic functions to fit the data but reported that a linear function yielded the most consistent results. A measure of consistency or goodness of fit should be reported for each of the functions explored. Similarly, for all curve fitting plots, sample size (n) and a goodness of fit measure should be included.

Two edge cases were excluded due to their rapid and severe SSQ score increase. These cases were attributed to “erroneous tests, showcasing SSQ scores surpassing 100, and resultant impairment.” The authors should clarify what about the tests was erroneous and whether the subjects experienced impairment.

The Figure 6 legend includes a statement that time in minutes is on the x-axis; however, the axis labels should be added to all axes.

When the term “significant” is used to describe data or a test, it should be accompanied by a statistical rationale for that determination. For example, a difference in negative correlation was found between the slopes for “real world” and “simulated” data. A t-test coefficient was provided with a p-value of 0.698. The difference in slopes was described as significant, but a p-value of 0.698 is not generally understood to convey statistical significance. Similarly, the authors report at another point that “motion sickness score was reduced significantly” without providing a measure of statistical significance.

The authors’ description of SSQ/MSQ data over time suggests a non-monotonic curve; recommend trying a fit with a polynomial.

Pearson coefficient reporting is erroneous; the relationship between two variables is generally considered strong only when their r value is larger than 0.7. Additionally, Pearson correlation is only applicable when the relationship between the variables is linear; if a polynomial fit is more appropriate, reporting Pearson correlation would not be relevant.

The text states, “The third experiment was conducted with a much longer wait period”; information should be given about the wait period (“more than a week” per the figure legend), including summary statistics such as range and mediate.

In addition to all figures, Table 1 should also include sample size for all summary statistics.

References:

Confirm the following reference is correct: Benz, T.M.; Riedl, B.; Chuang, L.L. Projection displays induce less simulator sickness than head-mounted displays in a real 598 vehicle driving simulator. In Proceedings of the Proceedings of the 11^th International Conference on Automotive User Interfaces 599 and Interactive Vehicular Applications, 2019, pp. 379–387.

Comments on the Quality of English Language

Grammatical revisions are required throughout the manuscript; for example, the plural and possessive forms are sometimes confused, and some sentences are difficult to understand.

Author Response

REVIEWER 4

The manuscript describes research conducted to discern the effect of different VR visual parameters on motion sickness (MS). Understanding the contributors to VR MS is of value to the scientific community; however, substantial revisions are necessary for the manuscript to have adequate rigor for publication.

 

Thank you for your time and effort in reviewing our work. We greatly appreciated all your comments and have addressed all of them to the best of our ability. There were many shortcomings that we missed in our initial submission, and we are very grateful in your expertise in pointing these out to us.

 

 

Abstract:

 

It is unclear what this sentence means: “This simulated setup was adjusted to best match the conditions found in the real system by using post-processing effects.”

The offending section has been expanded to better explain what was adjusted in the simulated setup.

 

Methods and quantitative results should both be presented in the abstract, including number of participants, age, and sex.

Information about the participants have been added in the text, abstract and conclusion.

 

Introduction:

 

There are inadequate references in the introduction, for example the following statements require citations:

 

As an alternative method, subjective user based questionnaire scores have widely been 10 used and been proven to be reliable.

 

Two significant instruments employed in this context are the Simulator Sickness Questionnaire (SSQ) and the Motion Sickness Susceptibility Questionnaire (MSQ).

 

A large contributor to MS is latency in VR and how it breaks immersion.

 

Motion-to-Picture latency and its effects on MS have been widely explored and is the primary consideration for most VR applications

Citations have been added to the unreferenced statements.

 

The following sections are background and belong in the introduction or related work:

 

Full 360 degree VR videos and the time consistency of stitching the scene has been shown to cause MS [26]. Several methods have already been proposed to alleviate MS in these cases but not all of them have been thoroughly researched.

 

Menezes et al. suggest a VR environment should be treated as a theater and not a film, this means camera animations and tricks like involuntary zooming and a lack of a reference can have a massive impact on comfort in VR [27]. A virtual body on the other hand has been shown to have a negative impact on motion sickness [28]. Serrano et al. proposed motion parallax to help with MS, but this may not counteract the MS caused by frame delay due to its added computational cost [29].

Thank you, the two paragraphs have been moved to the related works section.

 

Methods:

 

Informed consent was obtained from participants, but it should be stated whether an ethics committee reviewed the study in accordance with the Declaration of Helsinki. If not, there are ethical concerns about the work.

Informed consent was acquired from the participants prior to any testing. The participants were informed about the risk of motion sickness prior to touching the VR HMD. A disclaimer about this consent was also added to the Participants’ section.

 

There is inadequate description of the experimental procedures in section 3. For example, the manuscript should explain whether different visual parameters were changed simultaneously or sequentially, what the testing experience was for the subject, and how many subjects participated in each of the tests, including information about age range and sex. Reported MS is expected to increase over time; it is unclear from the methods as described whether confounding variables were considered or what the sequence randomization methodology was.

The text has been altered to include a better description of the experimental procedure and randomization. All experiments now also have the exact number of tests. Gender and age range for individual tests could not be added. Confounding variables were not explored in this study, all experiments were conducted as standalone and presumed as independent.

 

The manuscript should include details such as parameters for the barrel and pincushion distortions.

The distortion strengths that were tested on have been measured and added to the work.

 

It is reported that the majority of infield tests were conducted as 15-minute runs; additional detail is needed on the timing of all testing, such as range and median, rather than the majority.

A thorough description of the in field test as well as the other tests have been added to the results section. 

 

Test subjects were seated in either the passenger seat or diver seat; it is unclear whether this variation was controlled for or analyzed in separate sets.

This variation was regrettably not controlled for. 

 

The authors state that “The small sample size is considered reasonable when the research has time-intensive research procedures.” It is unclear whether the sample was powered to test a particular hypothesis; rationale should be provided for the sample size determination.

The statement has been updated to include the rationale behind the sample size determination.

 

A critical limitation of the work is that participants were ages 17-28; this should be acknowledged in the conclusions or in a limitations section.

We have added the emphasis of the participants' description in the abstract, conclusion and participants section. We have also included a description of the in field testing group that was left vague in the text.

 

The number of test sessions participants experienced should be reported, e.g., N participants experienced 2 test sessions, and the remainder experienced 3.

The overlap amount for the repeated tests have been added as well as a general overview of how many tests all participants took part in.

 

A description of the “simulation side” and “infield setup” testing is needed. It is reported that a majority of test subjects were male for infield testing, but precise numbers should be given.

A more precise description of the two setups as well as the participant groups have been added to the text.

The following is unclear and should be described in detail in the Methods section: “Employing linear regression on each participant’s SSQ scores, we quantified the spread of the generated lines to derive a baseline score.”

A better description of the MSQ based normalization method has been added. The section was not moved into the Methods section as the calculations and concept relied entirely on the initial results.

 

Results:

 

Results were often presented as averages. When averages are given, including in plots, the number of subjects contributing to the averages and measures of deviation or dispersion should be included.

The number of tests per experimental setup have been added to the text.

 

Results were mostly presented scaled by user MSQ. All results should be presented both scaled and unscaled, as this is a new approach and unvalidated.

Figures with no unscaled counterparts have been changed to either include them or only have the raw median values.

 

The authors explored exponential and logarithmic functions to fit the data but reported that a linear function yielded the most consistent results. A measure of consistency or goodness of fit should be reported for each of the functions explored. Similarly, for all curve fitting plots, sample size (n) and a goodness of fit measure should be included.

The function fitting section has been updated with data that was previously not included in the text. Sample size, test lengths and RMSE as a fit measure have been added.

 

Two edge cases were excluded due to their rapid and severe SSQ score increase. These cases were attributed to “erroneous tests, showcasing SSQ scores surpassing 100, and resultant impairment.” The authors should clarify what about the tests was erroneous and whether the subjects experienced impairment.

The specific cause of the erroneous tests and the reason for their exclusion from the dataset have been added to the section.

 

The Figure 6 legend includes a statement that time in minutes is on the x-axis; however, the axis labels should be added to all axes.

All figures have been updated to contain the x label of minutes.

 

When the term “significant” is used to describe data or a test, it should be accompanied by a statistical rationale for that determination. For example, a difference in negative correlation was found between the slopes for “real world” and “simulated” data. A t-test coefficient was provided with a p-value of 0.698. The difference in slopes was described as significant, but a p-value of 0.698 is not generally understood to convey statistical significance. Similarly, the authors report at another point that “motion sickness score was reduced significantly” without providing a measure of statistical significance.

The section of text has been reworded in order to not convey an unintended meaning.

 

The authors’ description of SSQ/MSQ data over time suggests a non-monotonic curve; recommend trying a fit with a polynomial.

A polynomial as well as an exponential curve which were tested prior have been added as comparative fit examples. 

 

Pearson coefficient reporting is erroneous; the relationship between two variables is generally considered strong only when their r value is larger than 0.7. Additionally, Pearson correlation is only applicable when the relationship between the variables is linear; if a polynomial fit is more appropriate, reporting Pearson correlation would not be relevant.

Pearson correlation has been removed from the text.

 

The text states, “The third experiment was conducted with a much longer wait period”; information should be given about the wait period (“more than a week” per the figure legend), including summary statistics such as range and mediate.

A range for the wait period has been included. We do not see a relevance for more detailed summary statistics for that wait period.

 

In addition to all figures, Table 1 should also include sample size for all summary statistics.

All tables now have the sample sizes included in the table descriptions.

 

References:

 

Confirm the following reference is correct: Benz, T.M.; Riedl, B.; Chuang, L.L. Projection displays induce less simulator sickness than head-mounted displays in a real 598 vehicle driving simulator. In Proceedings of the Proceedings of the 11th International Conference on Automotive User Interfaces 599 and Interactive Vehicular Applications, 2019, pp. 379–387.

The reference is correct. The description of their work as well as the comparative SSQ datapoint were verified again. The comparative datapoint was taken based on this statement: 

“On the overall score for simulator sickness, there is “strong evidence” (BF0+ = 12.95; meanI n−V ehicleDispl ay = 12.47, sdI n−V ehicleDispl ay = 12.03; meanH MD−Displ ay = 37.19, sdH MD−Displ ay = 32.57)”

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

In general, the paper could be written better and more concise.

(1)   The content is a bit scattered and the key points are not prominent. Can you consider removing the lens distortion or white balance or blur or vignette (one or more)? The influence of these factors is very small, even negligible.

(2)   Please provide instructions on how to analyze the field-of-view (FoV) using the HTC VIVE Pro, along with physical schematics of the hardware, and how the FoV of the cameras, virtual cameras, and head-mounted display work with this setup.

Author Response

Reviewer 1

In general, the paper could be written better and more concise.

 

Thank you for your time and effort in reviewing our submission. We have addressed your comments to the best of our ability.

 

 

 

(1)   The content is a bit scattered and the key points are not prominent. Can you consider removing the lens distortion or white balance or blur or vignette (one or more)? The influence of these factors is very small, even negligible.

 

The influence of these factors being small or negligible is part of what this work is reporting. The lack of a noticeable or discernable difference both by subject reviews as well as the gathered numerical test values indicates that these distortions, defects, and effects do not impact MS based on our findings.

 

 

 

(2)   Please provide instructions on how to analyze the field-of-view (FoV) using the HTC VIVE Pro, along with physical schematics of the hardware, and how the FoV of the cameras, virtual cameras, and head-mounted display work with this setup.

 

We have added a flowchart describing the flow of the image feed through the system under chapter 3.2. This should better describe the general setup of the system. The manually recorded FoV for the HTC VIVE Pro was recorded using both Test HMD tool by know2001 as well as by placing the controllers at the edge of the user’s vision and measuring the angle manually. The value 81 deg is a mean of multiple separate measurements.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Following are my comments on this work:

 

1. Units at many places are missing e.g. FoV 49-54 (unit degree missing), age etc.

2. English grammar need to be improved at various places.

3. At few places references are missing e.g. line 30,47 etc.

4. If authors abbreviated some words like field of view (FoV), head-mounted display (HMD), mixed-reality (MR) then after that abbreviation only should be used.

5. Line 110, paper title may be removed.

6. Line 120, journal title may be removed.

7. Line 191, use of square bracket create confusion with references and unit is also missing.

8. Line 280 typo error "form"

9. It will be good if Figure 2 is divided into named sub-figures (a,b,e,etc.) and described in more detail.

10. line 327, "field setup was" should be "field tests were"

11. Line 329, "when" may be deleted

12. line 330, unit "year" is missing.

13. Please mention waht the dots represent in Fig. 6.

14. Please elaborate meaning and implications of of "T-tests"

15. Please also include the "legend" for the red line used in figures 7-10.

16. Line 554 replace "means" with "mean"

 

Comments on the Quality of English Language

Need improvement

Author Response

Reviewer 2

 

Thank you for your time and effort in reviewing our submission. We have addressed your comments to the best of our ability.

 

 

Following are my comments on this work:

 

1. Units at many places are missing e.g. FoV 49-54 (unit degree missing), age etc.

 

Several occurrences in text where the units were not specifically mentioned have been changed to include them.

 

 

 

2. English grammar need to be improved at various places.

 

We have gone over the text and more erroneous sentences.

 

 

 

3. At few places references are missing e.g. line 30,47 etc.

 

References have been moved from the end of relevant sections and sentences to specific reference points within the text.

 

 

 

4. If authors abbreviated some words like field of view (FoV), head-mounted display (HMD), mixed-reality (MR) then after that abbreviation only should be used.

 

Unabbreviated versions of FoV and HMD have been replaced. MR only appears in the abbreviated form in a single section in the related works section. It is not used in the article itself.

 

 

 

5. Line 110, paper title may be removed.

 

The remarked title was highlighted but mistakenly marked as not strikethrough in the resubmitted text. Thank you for bringing it to our attention.

 

 

 

6. Line 120, journal title may be removed.

 

The journal title has been removed.

 

 

 

7. Line 191, use of square bracket create confusion with references and unit is also missing.

 

The square brackets have been replaced with curly brackets as those do not appear elsewhere in the text.

 

 

 

8. Line 280 typo error "form"

 

The typo has been edited.

 

 

 

9. It will be good if Figure 2 is divided into named sub-figures (a,b,e,etc.) and described in more detail.

 

The 4 main images in the figure are now labeled, the accompanying text has been expanded to describe the purpose of the image better.

 

 

 

10. line 327, "field setup was" should be "field tests were"

 

The line in question has been changed.

 

 

 

11. Line 329, "when" may be deleted

 

The unnecessary “when” has been removed.

 

 

 

12. line 330, unit "year" is missing.

 

Several occurrences in text where the units were not specifically mentioned have been changed to include them.

 

 

 

13. Please mention waht the dots represent in Fig. 6.

 

A description of the dots has been added to the figure.

 

 

 

14. Please elaborate meaning and implications of of "T-tests"

 

ANCOVA and T-test were used to only evaluate any quantitative similarity between the two datasets independent from any visual similarity. This has been additionally emphasized in the text.

 

 

 

15. Please also include the "legend" for the red line used in figures 7-10.

 

The legend in figures 7-10 has been updated to include the red reference line, additionally the description of the line from figure 7 has been duplicated to figures 8-10.

 

 

 

16. Line 554 replace "means" with "mean"

 

The typo has been edited.

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The information about subjects at the end of the abstract should be moved to the methods section of the abstract.

Other previous review comments have been adequately addressed, with the exception of review of the experiments by an ethics committee.

Comments on the Quality of English Language

Acceptable

Author Response

Reviewer 4

The information about subjects at the end of the abstract should be moved to the methods section of the abstract.

 

The subject information was moved to the start of the methods section.

 

 

 

Other previous review comments have been adequately addressed, with the exception of review of the experiments by an ethics committee.

 

Thank you again for your time and valued input. In the previous review the question was whether the study is in accordance with the Declaration of Helsinki, which applies to medical research. This research does not fall under this category. However, similarly to the ethical requirements of the declaration, the subjects were informed about possible discomfort, were able to stop the experiment at any time and no personal data nor identifiable medical data was gathered. To our understanding this research does not require any further review by an ethics committee.

 

 

Author Response File: Author Response.pdf