Less Is More: Higher-Skilled Sim Racers Allocate Significantly Less Attention to the Track Relative to the Display Features than Lower-Skilled Sim Racers

Round 1

Reviewer 1 Report

The abstract is unusual in that it is more of a narrative summary than a study abstract per se.  Many journals require structured abstracts in order to force authors to provide essential details in the abstract.  For example, there is no mention of the number of subjects, how many visits, randomization, statistical tests or significance, etc

The abstract was also a bit confusing.  The authors used gaze behavior to study visual attention.  Then they say that, basically, high and low skilled racers have, mostly,  the same gaze behavior.  They then write, however, that low and skilled players allocate their attention differently (i.e., but you just wrote that mostly they were the same)??  This was clear when reading the entire manuscript but less so in the abstract.

One issue is the somewhat linear line you draw between gaze behavior and attention.  What this study really showed is that gaze behavior differs between high skilled and low skilled sim players.  Allocation of attention etc is an inference based on differences in eye tracking.  It is very possible, for example, that high skilled players learn to attend to items that are not in their direct line of sight.  In other words, they are paying the same attention to the track they simply do not have to look directly at it any longer.  When possible, I would stick more closely to what you actually measured as opposed to the inference you are drawing from those measures.

Author Response

Please see the attachement for responses to all 3 reviewers. thanks 

Author Response File: Author Response.pdf

Reviewer 2 Report

This is an interesting paper that aims to examine how attentional allocation in a simulated racing environment varies with the level of expertise.

Using an eye tracker, four fixation-based measures are recorded as proxies for attentional allocation. Racers are classified by their lap times and the top vs bottom quarters of the population are compared with and without normalization with respect to lap times.

Overall, two measures differed significantly among the two populations: Total fixation duration (normalized or unnormalized) and track/hud fixation ratio.  On the other hand, fixation count and the average fixation duration were not significantly different.

Interpretation of these findings is ambiguous:

Hypothesis 1 has three inequalities to be tested. Only one of these is confirmed based on statistical significance.

Hypothesis 2 also has 3 inequalities to be tested. Of these two do not show any statistically significant difference and the one that shows statistical significance is in the opposite direction.

Hypothesis 3 is supported by the data. However, the rationale of this hypothesis is not clear: Why do the authors assume that HUD display carries irrelevant information (lines 86 and 87)?

Given the above considerations, I think that the results of the study are ambiguous in terms of attentional allocation and the data do not justify authors’ conclusions.

Part of the problem might be that the eye-movement parameters measured here may not be informative in and of themselves.

For example, smooth pursuit (which also requires attention) is not taken into account. I am not an expert in track-racing but at the very least the manuscript should state why.  

In terms of performance measure, only track time is considered and a more in depth analysis of performance and its correlation with eye movements are lacking: For example, what makes track times faster? Straight line speed, racing line, how turns are negotiated, etc. and how do eye movements correlate with these behaviors. For example, an expert may use the HUD during straight line acceleration, fixate on a breaking point landmark, or the apex in approaching turns, etc. Not considering these **task-specific** details and lumping all measures into track vs HUD may be a reason behind inconsistent results.

I am not sure the extent of data stored and available to the authors for further analysis: For example, do they have smooth pursuit data, do they have track/racing-line data to break down the analysis to a level which is much more specific to the task.

I recommend an extensive re-analysis using as much data as possible to correlate better the variables of interest to the specifics of the task.

Author Response

Please see the attachement for responses to all 3 reviewers. thanks 

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors presented evidence for different attention allocation during a sim racing task in low and high expert participants. They measured gaze behaviors in general and normalized for the fasted lap duration, and they found larger fixation duration for low skilled participants as an index of expertise and attention allocation. The study presents its novelty, and I can see the long term implication, especially in the field of gaming which is now significantly expanding. However, the paper could benefit from additional and important analyses on other parameters of gaze behavior. 

o   Figure1: Panel B is really not informative and could maybe be incorporated to panel A. I would like to see one heat map (or the averaged heat maps) of gaze behavior, which would give already information about what participants very looking at.

o   There are many metrics of oculomotor behavior that the authors ignored. Some of these metrics could also provide insight on the ability to with attention and not only on where and for how long attention was allocated somewhere. I assume one limitation is the refresh rate of the Tobii system (so saccades and smooth pursuits are off limits); however together with the number of fixations and average fixation duration , the authors could calculate average inter-fixation distance (Euclidean distance between subsequent fixation points measured in degrees of visual angle, akin to saccade amplitude), inter-observer consistency (the similarity between the heatmap created from fixations of a given participant and the heatmap created from fixations of all remaining participants, determined by using a correlation between these heatmaps; Lyu et al., 2020), the probability of blinking, first-saccade latency (measured in milliseconds), and heatmap entropy (a measure of dispersion of all fixations registered on an image with values ranging from zero to one; Gameiro, Kaspar, Ko ̈nig, Nordholt, & Ko ̈nig, 2017). 

o   I would like to see some single subject’s data. Instead of simple bar graphs I suggest to plot box-plots which would be more informative of the gaze distribution. 

Author Response

Please see the attachment for responses to all 3 reviewers. thanks 

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors responded well to the comments and suggestions from the previous round of review and improved the manuscript significantly. As highlighted in the previous review, although there are some limitations  of the study (limited task-specific info, limited eye tracking data), I think given that the authors used all the info/data available from their experimental design and that this study is one of the first studies in this area, it meets the standards for publication.