Pilot Sleep Behavior across Time during Ultra-Long-Range Flights

Round 1

Reviewer 1 Report

The authors responded to the points I raised.

Within the intrinsic limitations of this study (mostly, recognized by the authors), it seems now acceptable for publication

Reviewer 2 Report

This is an interesting manuscript describing the commercial pilot sleep behavior during ultra-long-range operations and the accuracy of Zulu watch measures of sleep duration and sleep timing estimation in operations compared against sleep diary. The research met its objectives, and the methodology and results are clearly presented and well written.

All comments raised during the previous round of review were satisfactorily addressed.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Thank you for inviting me to review this manuscript on hot topic

Sleep Depth and Patterns of In-Flight Sleep Behavior Across 2

Time During Global Pandemic Conditions The manuscript is easy to read and all data and conclusions presenting huge interest with possible impact on clinical practice.

 

I have several proposals:

1. Please use logic and traditional structure of the manuscript aftyer material and methods, results and after results discussions

Author Response

Thank you very much for your kind and concise review. The reviewer is correct that traditionally, articles are organized as Introduction, Methods and Materials, Results, then Discussion. However, the format for this article follows the journal-provided template and instructions for authors (Clocks & Sleep | Instructions for Authors (mdpi.com)). After verifying the format with the editor, we have retained the journal-specific logic and structure rather than the traditional structure suggested by the reviewer. Thank you again.

Reviewer 2 Report

Thank you for this nice paper on the commercial pilot sleep.

I appreciated reading this article. It is well constructed, globally well written and I appreciated the different figures used to illustrate the results.

I just have few comments.

1) I would avoid to giving results in the figure caption : 

• lines 144-145
• lines 166-138

These comments might be used to develop a little bit more the result part.

2) Regarding the results of Zulu Watch vs diary measurements, there were around more than an hour of difference; is these difference comparabale to other studies validating actigraphy vs sleep diaries ?

3) Regarding time duration of sleep duration, it seems that the pilots are highly sleep deprived during the missions. It might be interesting to have sleep duration per 24 hours and to have the number of sleep episodes by 24 h to have a simple descritption of pilots sleep during these different missions and realize in which extend they are sleep deprived.

4) Did you evaluate the somnolence across the missions ?

5) It would be very interesting in a further study to compare in wich extend the sleep during mission is different from the sleep at home ? And to measure if the pilot get sleep rebounds when they get back to home the next days.

4) It would be usefull to have more details about pilot caracteristics : age, BMI, associated treatments ? Dieseases ? Sleep complaints ? Sleep diseases ?  Habitual sleep duration when they are at home ? A table or details in material and methods.

My last question is about the activity data. You wrote that the activity data were collected in 2-min epochs in this study and I thought I've seen that for the validation of Zulu Watch, the records were in 30 sec-epochs. I guess you changed the frequency sample to have longer records but do you think that it might have an impact on the sleep algorythm ?

I highly appreciated the objectivity of the discussion particularly about the limits of the Zulu Watch.

 

Thank you by adavance for your answer

 

 

Author Response

The authors greatly appreciate the reviewer’s in-depth and constructive comments. This is obviously a field of great interest to the reviewer and their expertise is evident. We have tried to address the comments as completely as possible. With regards to Comment 1, we have removed the text describing results have been removed from the figure captions. For Comment 2, regarding the difference between self-report and actigraphy, we have added a brief explanation of the discrepancies found between sleep diary and actigraphy with additional citations and a discussion of how previous studies may differ from the current population in the Discussion section on lines 246-248. The overestimation by sleep diary found in these analyses are comparable to previous studies. For Comment 3, regarding the time duration of sleep, we have added the average sleep duration and number of sleep events per 24 hours to the Results section (lines 121-121), with additional context added to the Method section (lines 350-354) and Discussion section (lines 218-236). For Comments, 4-6, these data were not collected by the airline. These analyses are limited by secondary use of data. The authors realize that this limitation had not been fully expressed in the initial draft. Additional reference to study limitations related to the secondary data use study design have been added to the introduction (lines 77-79), discussion (lines 258-264) and methods (283-285) sections. Additionally, Zulu watch measures of sleep and sleep diary were analyzed using the fatigue prediction software SAFTE-FAST to evaluate fatigue risk and will be discussed in an independent manuscript (under submission). Finally, for the last comment, in our previous validation of the Zulu watch against laboratory polysomnography (PSG), the Zulu watch collected data in 2-minute bins, but PSG and Actiwatch 2 data were collected in 30-second bins. The PSG and Actiwatch 2 actigraphy data were re-binned into 2-minute intervals to compare against the Zulu watch. Neither the frequency sampling or sleep determination algorithm for the Zulu watch have been changed. However, a longer sampling rating is related to lower sensitivity and accuracy, which was discussed in our previous paper, “Validation of Zulu Watch against Polysomnography and Actigraphy for On-Wrist Sleep-Wake Determination and Sleep-Depth Estimation”. We hope that these responses satisfy the reviewer’s questions and comments. Thank you again.

Reviewer 3 Report

This descriptive study reporst data on twenty pilots flying across five COVID-19 humanitarian missions (between Brazil and China). These pilots wore Zulu sleep-tracking device (the watch) and completed sleep diaries.

As main findings, the authors reports some significant correlations between sleep diaries and measures collected by sleep tracker

 

This is a difficult to define study. Essentially, it is a descriptive study on the performance of this sleep tracking device in real-world operations.

 

Some mandatory changes:

1. Mostly, global pandemic conditions have no specific relation with the study. Pandemic justified the humanatarian missions and the cofinement of pilots in China. Nothing more. Reference to pandemic should be removed from the title, abstract and in the main text, with the notably exception of procedure (i.e., describing the missions)
2. The logic of providing some information on validity of a new sleep tracker by comparing its measures against those provided sleep diaries has little sense. If sleep diariy provides a reliable method to estimate some characteristics of sleep, there is no need of this (and others) sleep tracker. If they actually are not reliable and valid, consequently can't be used as a criterion to contribute to the validation of a new sleep tracker
3. Although significant, the correlations are not so high. This should be discussed (i.e., in terms of percentage of explained variance), since they point to a quite low validity
4. The so called "validation in laboratory" consist of a preliminary finding with a very small sample (eigh subjects, if I remember). Also this should be mentioned
5. Remove sleep depth form the title. The tracker is not accurate in estimating it also in the preliminary study in laboratory (i.e., using PSG)

Author Response

The authors thank the reviewer for their insightful and detailed comments. It is true that this is a difficult to define article, reporting findings from previously-collected data obtained during unique flight conditions. We have taken great efforts to try to address the reviewer’s comments throughout. Firstly, the title has been changed to reflect Comments 1 and 5. The revised title is “Pilot Sleep Behavior Across Time During Ultra-Long-Range Flights”. Secondly, reference to the pandemic has been modified throughout the manuscript as well. The authors disagree that the pandemic conditions had no relation to the study however. Not only did these flights exist solely for the purpose of obtaining pandemic relief supplies, but the conditions of the flight were otherwise unique for commercial aviation. Pilot fatigue and sleep patterns during these flights cannot be considered equivalent to ULR flights under normal conditions. This has been highlighted in the discussion as well on lines 197-210. In Comment 2, the reviewer brings up an important point and limitation to validation testing outside the laboratory environment. Very few, if any, sleep trackers have been validated in a field environment despite the fact that it is mainly in a real-world environment where it makes sense to use an actigraph or sleep tracker instead of polysomnography.  Additional discussion regarding this limitation, and the authors future plans to mitigate or resolve this limitation, has been added to the Discussion section on lines 258-259. In response to Comment 3, the Discussion section had been revised to address the strength of the correlations on lines 239-244. The authors feel that due to the initial limitations in data analysis, a detailed discussion on explained variance would not contribute to the overall significance of the manuscript. Regarding Comment 4, we are sorry that the reviewer feels that our previous validation test was insufficient or preliminary. These data have been peer-reviewed and published using between 2-3 nights of data from each participant, for a total of 22 nights of data. The authors have discussed the limitations of the in-laboratory validation testing in our previous paper, “Validation of Zulu Watch against Polysomnography and Actigraphy for On-Wrist Sleep-Wake Determination and Sleep-Depth Estimation”, and have made attempts to discuss the limited ability to test validity in a field environment in this publication. We hope that these revisions have sufficiently addressed the reviewer’s concerns. Thank you again.

Reviewer 4 Report

Below is a review comment that hopes can help you to improve your paper.

- In line 48, "... has never a ..." it's missing the verb form: "... has never been a ..."
- In line 61, please replace "," with semi-column.
- In line 61-62, "home-base" should be "home-based"
- In line 114, "re-corded" should be "recorded"
- In line 249, "in" should be a verb: "is"
- In line 278, the verb form of "were" should be "was" 
- In line 326, "For the purpose of these analyses" should be "The purpose of these analyses"
- In line 333, "time data" should be "time-data"
- In line 337, verb form "were" should be "was"

In addition, 
1. From line 58-60, suggest describe it in a list instead of in one sentence.

2. All figure captions should be in the figure-caption style of MDPI format. Please re-check the format figure and figure caption.

3. For figure 1, Please short the figure caption and move the detailed information or explanation to the body text better than in the figure caption.
 
4. In line 131, 156, and 182, These lines are unnecessary.

5. For figure 3, Please check the "( )" symbols mistake.

6. From line 271-272, suggest describe it in a list instead of in the continuing text.

7. Please check the reference format.

Author Response

Thank you to the reviewer for their concise and helpful comments. The requested corrections have been made throughout the manuscript and are marked with tracked changes with a few exceptions.  “Homebase” in this manuscript refers to the common aviation term for a pilot’s normal start and end airport, and so, line 61-62 were not changed to “home-based” however.  Additionally, “For the purposes of these analyses” was not changed to “The purpose of these analyses” as that would change the meaning of the sentence, which was to define the term “sleep duration” within the context of the analyses.  The figure format and reference format has been updated to use MDPI’s preferred formats as well. We hope that these changes satisfy the reviewer’s requests. Thank you again for your review.

Round 2

Reviewer 1 Report

accept as it

Reviewer 2 Report

Thank you for your answers.

You clarified all points.

Despite some methodological weakeness (mostely linked to the secondary use aspects of the data), that's interesting results.

Have a nice day.

Reviewer 3 Report

First of all, I find that is not acceptable that a reviewer can't work on a revised version in which changes have not been highlighted.

In the next round of revision, please highlight changes in the revised manuscript.

I have to repeat some requests of the first round or revision:

1. Remove sleep depth form the abstract and throughout the main text. The tracker does not estimatet or measures it. Strictly wording, PSG does not measure "sleep depth. Within the methods to operationalize "sleep depth", we have the quantification of slow-wave activity (SWA), responsiveness of acoustic stimulations and so on. Estimates provided by the Zulu does NOT measure or estimate "sleep depth". It only provides some estimates of few sleep measures. Without removing this basic and inappropriate term, this article is not acceptable
2. I was not able to find any comment on the weakness of providing some information on validity of a new sleep tracker by comparing its measures against those provided by sleep diaries.
3. Similarly, I was not able to find a comment on the relatively weak correlations, in terms of percentage of explained variance