SLEEPexpert+: Blending Internet-Based Cognitive Behavioral Therapy for Insomnia with In-Person Psychotherapy—A Feasibility Study in Routine Care

Round 1

Reviewer 1 Report

This manuscript presents a summary of a feasibility trial to deliver blended clinician-led face-to-face CBT for Insomnia with a digital CBT-I platform (SLEEPexpert+). The authors use a mixed methods approach, utilizing qualitative feedback from clinicians and a case-series approach for quantitative outcomes from the small number of participants who had complete data at the end of the data collection period. The authors highlight treatment being "intensified" although this is not operationally defined and testable with the current study design. Recommend operationally defining the term or taking the opportunity of this feasibility study to inform the operational definition to be tested in future research.

Other recommendations to consider:

2.3. Stimulus control is not explicitly mentioned as a module in SLEEPexpert+. Is it built into the bedtime restriction module? If not, why no inclusion of an evidence-based component of CBTI?

2.5. Provide more details regarding increased suicide risk. Were exclusions due to SI plus intent and plan, etc.? Was SI OK as long as not intent, plan was present?

3.1.3 should read "Experiences regarding SLEEPexpert+" as 3.1.2 is "Expectations..."

One of the key benefits of blending CBTI with SLEEPexpert+ is the potential for "outsourcing" of features like PMR/relaxation training. It is also an opportunity for consistency in exposure/delivery/instruction for those in the blended program. Outsourcing of psychoeducation may also be a key feature considering the high usage among participants (Figure 1). 

3.1.4. The comment re: homework highlights the importance that clinicians need to explain the importance of "homework" or work in general that occurs outside of the appointment that is necessary for treatment gains/recovery.

3.1.5. Please expand on the statement "...an acceptance-based approach could positively influence the effectiveness of SLEEPexpert+." Is this referring to the newer treatment protocols of ACT for Insomnia? Or is this more focused on acceptance-based approaches for integrating the technology?

3.2.1. Figure 1. The modules for bedtime restriction (13.4%) is quite low but may be the most important. Thoughts on how to promote greater use of the evidence-based modules (i.e., bedtime restriction, relaxation, cog restructuring)?

3.2.2. The case series is a useful presentation of the quantitative, pre-post treatment data. Consider adding more structure to each case series: number of f2f sessions, total duration of treatment (weeks), use of SLEEPexpert+ (hours/weeks/#modules), and the measures collected at pre and post-treatment. The information is there for many of the cases presented but consistency across all would be helpful. 

Even though only 6 participants completed treatment, can you comment on who was a super user of SLEEPexpert+ and their therapeutic journey, even they did not complete treatment at the end of the study period. This would be valuable information as only 1 of the case series used SLEEPexpert+ very much (Figure 2, Case 1).

4. Discussion (paragraph 1) As mentioned above, the statement "...treatment was intensified..." is not very clear. Is this simply by using SLEEPexpert+ treatment is intensified - the participant is doing "more" treatment than if only in CBTI alone? Intensification seems to imply a value add to treatment outcomes above and beyond what is achieved in CBTI alone - more intense treatment results in better outcomes. 

In paragraph 2, the authors note blended CBTI can increase availability and intensity of CBTI in routine care. This is not supported by the current study findings. While intensification may be true, it may only occur for those who use SLEEPexpert+ a sufficient amount, which is yet to be determined (e.g., a dose response effect?). Further, the time spent on the specific modules may be a factor. For example, is reviewing and spending time on the education module as beneficial as time spent on the bedtime restriction/relaxation/cognitive restructuring modules?

The authors note that the bedtime restriction module was not used as often. Could this be due to bedtime restriction always being a component of f2f, so it was perhaps less relevant to review again on SLEEPexpert+?

Agree about adding non-CBTI relevant modules, such as depression, anxiety, chronic pain. This may be a variation of intensification that is important for the many comorbid cases seen for CBTI. 

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The study investigated the feasibility of a blended CBT-I treatment in routine care at a sleep clinic. The study is in general well-presented and the treatment approach seems very reasonable and appropriate. The conclusions that can be drawn from the study are relatively limited, mainly because of the small sample size. However, as a first step before an RCT the results are still interesting. See below for some specific comments:

 

-       I think the study would benefit from a more detailed description of the semi-structured interview. I would suggesting adding a list of the questions in the supplement. It would also help to get a description of how the themes were chosen in the analysis step 

-       Please also add a description (or reference) for the type of sleep diary used

-       I understand from the description that the study was planned as clinical development rather than a research project and that it thereby did not need ethical approval. Is that right? What about consent for the therapists and patients? Please clarify

-       It is a bit unclear to me in terms of what co-morbidites patients could have. It says in the description that they were excluded if having another disorder that needed treatment. However, case 1 is described as having a moderate episode of depression. Would not that require treatment? Some of the other cases also seem to have had anxiety disorders. I guess they would also require treatment at some point. In line, what about other types of psychiatric medication: Where any of the cases using any antidepressants? Please include such details

-       I think the paper would benefit from some description/ illustrations of the actual treatment program: What types of rules were used for the different modules (like sleep restriction)? Which passages were presented as text and videos respectively? What did it look like? Since user experience is a key part here, it would be helpful to see some “screenshots” or examples 

-       The authors write that “Program usage data indicate that the treatment was intensified through the blended treatment approach…” – what data is supporting that statements? Since there is no control I think it is not possible to say that?

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

While this is a worthwhile study and I am genuinely excited to see where this research goes, the study would benefit from more specific information overall. The aims are unclear.. what were the authors trying to accomplish both with respect to both qualitative and quantitative arms? The study methodology is unclear as well. It would be helpful if the authors

o   Expanded upon the components of the app, inclusion criteria (what were the cutoffs for insomnia), the inclusion of a demographics table, data tables with sleep diary data, why the study only included men, where were the subjects recruited from, training of the therapists (were they trained in CBT-I), more information on why anxiety, depression, & wellness were included (there is minimal information about this in the introduction), why the DSM V was used (was this a secondary analysis), etc.

o   Provided more information  on the qualitative piece of the study (who coded for themes? Was there an arbiter? Was software used?).

o   Broke up the manuscript into “Study 1” and “Study 2”, where one study focuses on the qualitative piece, and one focuses on the quantitative piece. This would help with the organization of the paper. The authors could also consider breaking this up into two papers where one is focused on the quantitative outcomes and one is focused on the qualitative outcomes.

o   The discussion section should focus more on the findings of the paper. The limitations section should be expanded and include more information, specifically why the sample size was so small.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

Dear Authors,

Thank you for this interesting and relevant study. The study is well written, however, I suggest some modifications to further improve the paper.

Introduction:

Lines 40-41-  “However, only a fraction of patients 40 will receive CBT-I because insufficiently trained professionals are available to meet population needs [9].”

Point of clarification: Do you mean the available professionals are insufficiently trained? Or there are insufficient number of trained professionals?

 

Methods:

The study design was not described ( case study? Experimental?). Kindly add one section on study design and how this was implemented

Also, I suggest that instead of feasibility, Pilot study may be used, considering the very limited number of therapist and patient participants.

Lines 143-144:  (December 2022 to April 2022)- kindly check the duration dates

Participants – You have very limited number of participants (Therapists and patients)

I suggest that the two groups of participants be described in separate paragraphs with the study aim for each group further explained.

Measures- How were the questionnaires administered? Any difficulty? What is the reliability of the sleep diaries?

I suggest that the interview protocol with the interview questions to the therapist be more specified.

Have you identified any intervening variables and how were these controlled?

 

Results:

I suggest that a summary table for the qualitative data be created which will include the theme, categories, and codes to have a complete picture at one glance.

Where are the results gathered from the questionnaires? I suggest the demographic profile and the results from the four questionnaires (Insomnia, anxiety, depression, well-being) with the interpretations be presented.

Since there are only 6 patient participants, a tabular summary of the results categorized according to the variables measured in the study should be created for better readers’ appraisal.

Example of categories:  before and after level of clinical insomnia anxiety, depression, well-being, SE, f2f sessions, comorbidities, etc.

Categorizing the results will also help you identify the statistical analysis done  which can be included under the data analysis instead of stating that the quantitative analysis are presented as case studies.

The result should be further substantiated

 Discussion: It would be helpful to describe  further the components of the SLEEPExpert+ and how these and its usage contributed to the outcomes.

Major flaw of the study:

The study has very limited number of participants and the results may not be reliable. I suggest that the study be expanded to include more therapists and patient participants.

 

Thank you and good luck.

 

Few grammar issues were noted.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

Thank you for your attention to the reviewers comments! While this research is novel and would be a contribution to the literature, there remains questions about the study's study design and methodology. Specifically, if the  aims of study still feel underdeveloped and unclear.

Additionally, while it is understood that this is a blended intervention, the manuscript would greatly benefit from breaking it up into two sections or papers (i.e., qualitative and quantitative). The authors are encouraged to review qualitative study methodology (e.g., grounded theory). Standard qualitative assessment and evaluation procedures usually require 3 reviewers who code and arbitrate for specific themes. If the author's are going to present qualitative data, it would be helpful if they followed a more standardized format.

Author Response

Thank you for your time and comment.
Indeed, our study is one of the first on a blended treatment for insomnia patients in routine care. Against this background, we deliberately conducted a feasibility study based on a small sample size but rich information based on both qualitative and quantitative data. We ask for your understanding that we do not want to divide the manuscript into two studies or two papers.
Regarding the qualitative methodology: As we write in the manuscript, we used inductive analysis, according to Birks and Mills (2010). This is an approach often associated with grounded theory methodology. We could describe this method in more detail and write more about specific aspects such as data collection, data coding, constant comparison, category formation, theoretical saturation, etc. However, we feel that this would be exaggerated regarding the few data or interviews we have categorized. The paper reports on first experiences with blended treatments and is not intended to be a qualitative method paper. Furthermore, indeed, we do not evaluate or report interrater reliabilities on the codings. However, we indicate we used consensus ratings (first author and last author) to assess and interpret the categories, which is also an accepted approach.
Thanks again.
 

 

 

Reviewer 4 Report

Dear Authors,

Thank you for your revised paper.

I would like you to review how to write a feasibility paper and revise your manuscript accordingly including missing components of a feasibility study.

Thank you.

 

Minor English editing needed.

Author Response

Thank you for your comment.
From our point of view, feasibility papers are pretty heterogeneous, depending on the context. We could indeed have assessed more data that could have been relevant, but we did not, and thus, we cannot revise our paper accordingly. 
Your feedback has prompted us to reflect on future research endeavors. Once again, thank you for your valuable input.