Effectiveness of Digital Interventions for Deficit-Oriented and Asset-Oriented Psychological Outcomes in the Workplace: A Systematic Review and Narrative Synthesis

Round 1

Reviewer 1 Report

This systematic review with narrative synthesis considers the effectiveness of digital workplace interventions aimed at improving mental health. As an additional aim of the study, the relationship between the theory base and the effectiveness of interventions was also considered. The systematic review methods applied are of high quality, including the registration and publication of an a priori protocol, searching multiple databases, and assessing the risk of bias in the included studies. The paper is well-written with only minor grammatical mistakes. One error that is found throughout the paper is the use of the singular verb-tense “was” with a plural subject. This is a problem that can surely be easily remedied. Nevertheless, there are a few suggestions that I have that could improve the quality of the review and make it easier to understand for readers.

 

Introduction

1.       Line 33: What does the abbreviation bm behind the amounts of money mean? I am not familiar with this abbreviation.

2.       Could you include a phrasing of the review aims and/or inclusion criteria in relation to the Population (P), Intervention (I), Comparison (C), Outcome (O), and Study Design (S) categories, also known as the PICOS-scheme? This would help readers to understand which types of studies were included and excluded.

Methods

3.       The AMSTAR Checklist for systematic reviews asks if a review team justified the use of a language restriction. In this review, only papers published in English were included. Can this be justified?

4.       Line 159: The protocol is cited as Popay et al. here. It was not initially clear to me that this was the study protocol. The authors should consider re-mentioning the study protocol in this sentence.

5.       Nvivo and the JBI extraction form should be cited.

Results

6.       The reasons for excluding papers are listed in the paper, but it is not possible to determine which papers were excluded for which reason.

7.       Lines 167-176: The way numbers are reported here is confusing. Large numbers are missing thousands-separators (is this a style recommendation from the journal), and one number >10 is written out, while another number <10 is reported as a digit.

8.       Figure 2: I do not find this diagram adds to the paper. I think the narrative synthesis process is described already in lines 205-210. On its own, I do not understand this diagram. Either it needs to be constructed differently or omitted.

9.       Section 3.4.1: The reporting of psychological well-being measures is important but confusing. It ends up reading like a long list of instruments without much further meaning. Would it be possible just to list one or two of the most frequently used measurements in the text and summarize all of the measures in a table?

10.   Lines 248-250: Studies measuring blood pressure and neurocognitive efficiency are listed in the section summarizing outcomes related to mental health concerns. Is this because the studies measured these outcomes in addition to using a depression or anxiety screening tool? Otherwise, I would not see blood pressure or neurocognitive efficiency as measures of mental health but possibly more as physiological indicators of stress. Could you explain why these outcomes are categorized as they are?

11.   Figure 3: The use of an evidence map is interesting but hard to follow. I have to search and do some counting to determine if there were more studies with evidence or no evidence for a certain type of intervention/theory. Could the mapping first branch off into interventions and then place the no evidence list next to the evidence list?

12.   Table 1: The formatting of this table is confusing. Do the headers for “Some effects” and “Multiple effects” also pertain to the 2 columns labeled “Theory mechanisms “? Why is there not a “Theory mechanisms” column next to “No effects”?

Discussion

13.   The authors cite two meta-analyses in the discussion but state that it was not possible to conduct one here. Could this be discussed in more detail?

14.   Line 603: Here it is stated that only two digital meditation/mindfulness interventions were effective in reducing anxiety. Could the authors include the total number of meditation/mindfulness interventions included, such as “two of ##”?

 

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

I have completed my review of the manuscript entitled "Effectiveness of Digital Interventions for Deficit-oriented and Asset-oriented Psychological Outcomes in the Workplace: A Systematic Review and Narrative Synthesis". This study investigated the evidence on the effectiveness of digital psychological interventions in the workplace. The subject matter of this work is laudable. However, the paper has some concerns as following:

 Method

The research methodology sound and relevant to the field.

P3 lines 143-144, What was the kappa statistics of two reviewers? Please indicate in the results section.

 

Results

Figure 1 Articles (Abstracts & Full texts) meeting the inclusion criteria (N= 45) seems to be wrong. I seems to be N=44 (=301-257).

In addition, Articles identified through references lists of other systematic reviews: (N=6).

The direction of the arrow is reversed.

 

Discussion

Any discussion of the results of systematic reviews should focus on the strengths and limitations of the evidence in the original studies used in the review.

The authors should state that there may be a positive publication bias. Indeed, most studies identified reported positive findings.

Author Response

Please see attachment

Author Response File: Author Response.docx

Reviewer 3 Report

I read “Effectiveness of Digital Interventions for Deficit-oriented and Asset-oriented Psychological Outcomes in the Workplace: A Systematic Review and Narrative Synthesis” with great interest and I think it is an important contribution to the field. I think that this article was very well written and informative. I have a few comments and suggestions below.

I think the introduction could be somewhat improved. I understand that the paper builds on a previously published protocol, but I think the authors could make it easier for the readers to follow the arguments. The first sentence, “outcomes associated with poor wellbeing outcomes at work…” is confusing from the beginning. Is there some mistake in the sentence or are the authors describing outcomes that are associated with other outcomes? The first paragraph describes the problem, but there needs to be some kind of transition leading to the next paragraph. The introduction should introduce and build up the research questions and the motivation for investigating these questions. Key concepts should be defined, like deficit-oriented and asset oriented. The reader should understand what needs to be known and why. Finally, a lot of text is devoted to issues in the pandemic, while all articles refer to the pre-pandemic period.

 I apologize if some comments are too detailed, but maybe it helps in the editing and proofing process. In the inclusion criteria section, here is written where it should be there. There is also lone parentheses in the end of the first paragraph of this section. Terms should be used consistently. This is a small issue, but on page four, the heading says, “data collection process” and the description says “data extraction process”. In the study characteristics section, it may be useful to define what the author’s specifically mean by “quasi-experimental” and “pilot studies”.

At the beginning of page 7, it should say “one cluster” instead of “once cluster”. Is the main difference between outcome cluster A and C that one covers negative mental health and the other covers positive mental health? Maybe this should be explained? I am not sure that it is totally necessary to mention every scale that was used for every measure, but I could be wrong. Would it be possible to group the scales a little more broadly in terms of what symptoms they measure? I am a bit confused that absenteeism and presenteeism are both classified as “withdrawal behavior”. I guess they are quite opposite problems. Absenteeism should include absences from work such as sick leave while presenteeism should include individuals coming to work even though they are ill. Or have I misunderstood something?

Some of the text in the boxes in figure 3 is cut off.

At the end of page 16, you might consider a different heading structure. If I understand correctly, A, B, and C refer to the primary outcomes, and then D is describing the types of interventions, not the secondary outcome which is labeled as D earlier in the text.

The risk of bias figure has no title and appears quite distorted.

I would suggest that the discussion section begin with a brief summary of what was found. The authors refer to recent meta-analyses covering similar subject matter (23 and 42 in the text). This comes back to my comments about the introduction, but I think the reader needs to know what has already been done on the topic and what this systematic review is contributing to the knowledge gap. What is this systematic review adding in comparison to the recent meta-analyses?

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

The authors have done a good job addressing the issues raised by the reviewers. The paper needs to be edited for typos before publication. 

Author Response

Thank you for your feedback. We can confirm that we have completed proof-reading the manuscripts and identified and corrected typos.