Study of Impact of COVID-19 on Mental Health and Wellbeing of Staff Working in a Forensic Mental Health Service

Round 1

Reviewer 1 Report

This is a nicely executed study, well written and logical. Limitations of the research are acknowledged. My main suggestion is that the findings be discussed more thoroughly, with citations of relevant literature. The advantages and disadvantages of working from home might be elaborated. For example, the increased hours of work in these workers has often been reported anecdotally but needs to be confirmed empirically. Similarly, the stressors entailed in facing patients directly merits interpretation in terms of particular emotions, e.g. fear of contracting covid, patient hostility, anger at inadequate supplies. Other studies should be described and integrated with the current results.  

Author Response

Reviewer 1

This is a nicely executed study, well written and logical. Limitations of the research are acknowledged.

My main suggestion is that the findings be discussed more thoroughly, with citations of relevant literature. The advantages and disadvantages of working from home might be elaborated. For example, the increased hours of work in these workers has often been reported anecdotally but needs to be confirmed empirically.

We have added the following reference which supports perspectives of increased worker hours:

17. Waizenegger L, McKenna B, Cai W, Bendz T. An affordance perspective of team collaboration and enforced working from home during COVID-19. Eur J Inf Syst 2020, 4: 429-42.
18. Liberati E, Richards N, Parker J, Willars J, Scott D, Boydell N, et al. Remote care for mental health: qualitative study with service users, carers and staff during the Covid-19 pandemic. BMJ Open 2021; 11. Available from: http://dx.doi.org/10.1136/bmjopen-2021-049210

Similarly, the stressors entailed in facing patients directly merits interpretation in terms of particular emotions, e.g. fear of contracting covid, patient hostility, anger at inadequate supplies. Other studies should be described and integrated with the current results.

Included studies supporting this are included:

7. Muller AE, Hafstad EV, Himmels J, Smedslund G, Flottorp S, Stensland S, et al. The mental health impact of the Covid-19 pandemic on healthcare workers, and interventions to help them: A rapid systematic review. Psychiatry Res 2020; 293. Available from: https://doi.org/10.1016/j.psychres.2020.113441
8. Riguzzi M, Gashi S. Lessons from the First Wave of COVID-19: Work-Related Consequences, Clinical Knowledge, Emotional Distress, and Safety-Conscious Behavior in Healthcare Workers in Switzerland. Front Psychol 2021;

Reviewer 2 Report

Dear authors,

This research paper describes the actual topic – Service Evaluation: Study of impact of COVID-19 on mental health and wellbeing of staff working in a forensic mental health service. In their article authors seek to explore the impact of Covid-19 on the mental health and wellbeing and trust support of Tees Esk and Wear Valleys (TEWV) NHS forensic staff using an online survey. As well, authors notice, that data were drawn from a cross sectional representative sample of 246 TEWV forensic staff. Staff working at home and on the front line were both affected by depression, stress and anxiety.

And I would like to share with authors some doubts and remarks too: it seems important to notice, that it would be needed to concentrate on the abstract of the article, as more detailed description could make better article readability. As well it would be needed to concentrate on the conclusions of the study. Thus, it would be needed to include to the concluding ideas more future oriented theoretical implications, thus accessing deeper concluding insights.

Author Response

The reviewer. 2 Reviewer 2

Dear authors,

This research paper describes the actual topic – Service Evaluation: Study of impact of COVID-19 on mental health and wellbeing of staff working in a forensic mental health service. In their article authors seek to explore the impact of Covid-19 on the mental health and wellbeing and trust support of Tees Esk and Wear Valleys (TEWV) NHS forensic staff using an online survey. As well, authors notice, that data were drawn from a cross sectional representative sample of 246 TEWV forensic staff. Staff working at home and on the front line were both affected by depression, stress and anxiety.

And I would like to share with authors some doubts and remarks too: it seems important to notice, that it would be needed to concentrate on the abstract of the article, as more detailed description could make better article readability.

Thank you for your comments – we have amended the abstract and provided some additional information but within the prescribed word limit.  The revised text is below:

“This study explored the impact of Covid-19 on the mental health and wellbeing and trust support of Tees Esk and Wear Valleys (TEWV) NHS forensic staff using an online google survey during the second wave of the pandemic. Survey respondents were a voluntary cross-sectional sample of 246 TEWV staff working in the forensic directorate staff; this included males (n=60, 24.5%); with the majority of staff aged between 36-50 years (n=99, 40.2%) and 50 years or older (n=80, 32.5%).  The results showed that staff working at home and on the front line were both affected by depression, stress and anxiety. Those most at risk were younger staff members.  We concluded that the mental health and well-being of staff working should be a priority.  It is important to consider targeted support that should be aimed at younger staff members to provide an open culture enabling for those who want support to have readily available signposted resources.  Staff working in different settings may have experienced a different impact of COVID-19 on their mental health and wellbeing, and whilst some interventions might be successfully applied across the service, it would be beneficial to understand the unique needs of staff working in specific settings.”

As well it would be needed to concentrate on the conclusions of the study. Thus, it would be needed to include to the concluding ideas more future oriented theoretical implications, thus accessing deeper concluding insights.

We have added some new text to the conclusions of study:

“Staff working in different settings may have experienced a different impact of Covid-19 on their mental health and wellbeing and that, and whilst some interventions might be successfully applied across the service, it would be beneficial to understand the unique needs of staff working in specific settings.  We also know that the institutional responses in hospitals (patient face-to-face contact continued) differed from that in other settings (face-to-face often replaced by virtual contact).  “

Reviewer 3 Report

First of all, congratulations to the authors for their work.
Here are my comments on the article:

In my opinion, the title is unclear simply by referring to an impact study, without the need to specify "Evaluation Service".
The summary lacks the inclusion of the scale for measuring stress, anxiety and depression, as well as the statistical analysis used.
The introduction is very sparse. The study is hardly contextualised. There is no discussion of the problem under study, nor are important terms such as anxiety, depression or well-being defined. Hardly any data is given about the clinics where the survey participants work beyond the conditions of the patients they attend. This section needs to be properly elaborated.
The methodology does not specify the type of sampling. If it is non-random, as it seems to be, it cannot be claimed that the sample is representative of the population.
The survey appears to be ad hoc, and not validated, except by a review by a senior management team. Is this the case? Why not use a validated stress, anxiety, depression and quality of life survey?
It is not part of the procedure to protect patients from infection during the study period. This could in any case be included in the introduction.
The procedure of approaching the participants is said to be online via a Google Forms survey. What is the meaning of neighbourhood meetings etc.?
In the results, if no statistical significance is found, it should not be included as a relevant result. This is the case for insomnia among staff working in front of patients or online. The same is true for home-based and non-home-based staff and the level of anxiety and depression, among others. It is accepted that in research studies both positive and negative results in relationships are equally relevant, but it does not make sense to highlight the absence of statistically significant differences if a table with all the data is already included. It is preferable to only highlight significant relationships in the text.
The name of the table should be at the beginning of the table and not at the end.
The first two sentences of the discussion should be part of the introduction.
The second and third paragraphs of the discussion do not relate to the findings of the study. This should be done or it would not make sense to include them.
The limitations present strengths of the study. This is not adequate.
Perhaps the discussion is the best elaborated section of the whole study.
The conclusions are also accurate.
The bibliographical references are adequate and up to date.

I hope that my contributions will help you to improve the article.
The reviewer.

Author Response

Reviewer 3

First of all, congratulations to the authors for their work. Here are my comments on the article: In my opinion, the title is unclear simply by referring to an impact study, without the need to specify "Evaluation Service".

Thank you for your comments – we will changed the title of the article and removed the term ‘evaluation service’.  The new title reads:

“Study of impact of COVID-19 on mental health and wellbeing of staff working in a forensic mental health service.”

The introduction is very sparse. The study is hardly contextualised. There is no discussion of the problem under study, nor are important terms such as anxiety, depression or well-being defined.

We have added some more context to the introduction including a paragraph on the problem understudy and some definitions as you suggest for anxiety and depression – the additional paragraphs are below:

“Previous literature evidences the substantial psychosocial impact of the pandemic on the healthcare workforce (5). The annual NHS staff survey reported 44% of all staff being ill from work-related stress, a significant increase on previous year. Approximately one third of NHS staff working on COVID-19 wards in the last year, reported feeling unwell due to work-related stress (6).  More specifically, this finding was shown in a comparable mental healthcare sector study which demonstrated moderate levels of burnout and mild to moderate levels of anxiety and depression in staff working in learning disability service in Ireland during the pandemic (7). A systematic review found similar results worldwide in Italy (8), Spain (9), Norway (10) and Switzerland (11)”.

“Depression is an affective state characterised by low mood or reduced interest in activities which may be associated with low energy levels, poor concentration and disturbed sleep. Anxiety symptoms include excessive worry, impaired concentration, restlessness and increased heart rate. (Ref: American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi-org.ezproxy.frederick.edu/10.1176/appi.books.9780890425596) Stress symptoms may include mental or physical tension, feelings of pressure and negative emotions (Ref: Baqutayan, S. M. S. (2015). Stress and coping mechanisms: A historical overview. Mediterranean Journal of Social Sciences, 6(2 S1), 479-479.)”

Hardly any data is given about the clinics where the survey participants work beyond the conditions of the patients they attend. This section needs to be properly elaborated.

Thank you for your comment – we have added the following text

“Staff worked in a range of setting including in a secure mental health hospital with 187 beds for adults with a mental disorder, learning disability or autism and non-secure wards with 11 beds for adults with learning disability. Some participants worked in community teams where healthcare was provided in outpatient clinics, in the homes of patients, care/nursing homes, in police stations and in the Courts. Other participants worked as part of a mental health in-reach team located in each of the region's eleven prisons.”

The methodology does not specify the type of sampling.

No sampling strategy was employed other than sending via email the questionnaire to every staff member that worked within the Forensic Directorate.

If it is non-random, as it seems to be, it cannot be claimed that the sample is representative of the population.

We agree with the reviewers comments we have therefore removed this word ‘representative’ from the text it now reads:

“Data were provided by a cross sectional group of staff (n=246) who worked in within the Forensic Directorate of Tees, Esk and Wear Valleys NHS Foundation Trust (TEWV).”

The survey appears to be ad hoc, and not validated, except by a review by a senior management team. Is this the case?

The survey was developed as an evaluation of the forensic service by a Consultant Psychiatrist with support from the academic lead author (AP).  It was devised to assess the needs of staff combined with the interests of the Trust during this time-period.  You are correct in your assumption that it is not validated and that it was reviewed by the senior management team and other colleagues (AA).

Why not use a validated stress, anxiety, depression and quality of life survey?

The questionnaire was an evaluation of service and impact on staff well-being in a group of staff who care for people with mental health problems and are well aware of their own mental health.  We did not therefore want to use validated measures of stress, anxiety and depression and quality of life – the survey is limited in this nature and therefore only provides evidence based on the self-report of the staff themselves and how they were feeling with regards to their own concerns about their own perceptions of their own mental health.  We have added this limitation to a section in the discussion now reading:

“This study has some limitations that should be considered during the interpretation of the results. The first is that the data collected was self-reported and is therefore subjective, however the purpose of this study was to gauge the wellbeing of staff rather than diagnose any potential illnesses. Staff working in this environment are trained to support, diagnose, and manage mental health and therefore arguably have a good insight into their own mental health.”

It is not part of the procedure to protect patients from infection during the study period. This could in any case be included in the introduction. The procedure of approaching the participants is said to be online via a Google Forms survey. What is the meaning of neighbourhood meetings etc.?

Thank you for your comment – we cannot locate the term ‘neighbourhood meeting’ in the manuscript.  We have however elaborated on the procedure and promotion of what was done in approaching participants by including the following text:

“Promotional channels included: staff meetings, research champions (members of staff who encouraged uptake of the survey), forensic directorate management meetings, emails sent to team managers, and the weekly forensic directorate staff briefing.”

In the results, if no statistical significance is found, it should not be included as a relevant result. This is the case for insomnia among staff working in front of patients or online. The same is true for homebased and non-home-based staff and the level of anxiety and depression, among others. It is accepted that in research studies both positive and negative results in relationships are equally relevant, but it does not make sense to highlight the absence of statistically significant differences if a table with all the data is already included. It is preferable to only highlight significant relationships in the text.

Thank you for your comment – we hope that the reviewer can consider us leaving in the non-significant results in the text; when these are removed it looks as if everything is significant and unless the reader were to study the tables in the details the non-significant results would not be highlighted at all.  To keep the essence of unbiased reported we would therefore favour keeping in the text that relates also to the non-significant results as these are just as important to highlight in our findings as the significant results.

The name of the table should be at the beginning of the table and not at the end.

We have moved the title of the tables to the beginning.

The first two sentences of the discussion should be part of the introduction.

We have moved the first two sentences from the discussion into the introduction section as advised.

The second and third paragraphs of the discussion do not relate to the findings of the study. This should be done or it would not make sense to include them.

Thank you for your comment we have re-written these paragraphs to more closely align the research findings with the wider literature.  The revised text is written below:

“Similar to our study findings other research has shown that several factors contributed to the adverse effect on wellbeing in mental healthcare staff . These included, risk of infection, staffing pressures, longer working hours, moral injury, increased responsibility, patient frustration, feelings of uncertainty, and feelings of guilt among others (12).  Our research also concurred with the same major finding from this paper reporting similar increases in self-reported anxiety, stress, depression, and sleep difficulties in both those working as usual and those working from home. Working from home has some acknowledged benefits (e.g., greater flexibility), there were also several negative aspects noted.  Other research found that the average length of a workday has increased by 48.5 minutes, and the number of meetings per person had increased 12.9% since working from home was introduced (13). Furthermore, home working was found to increase screen time resulting in feelings of fatigue, headaches, and increased eye strain (14). The term ‘zoom fatigue’ has even been coined describing the emotional and physical drain of constant video conferencing (15). There have also been reports of back-to-back virtual meetings as no travel time is required (16) and the expectation of constant connectivity (17).  The necessary rapid introduction of remote working also meant that some staff had to learn to use new technologies too quickly and/or without sufficient training and support (18); this will have an impact on how people work in the future and new policies are required to support staff regardless of whether they work at home or continue with some element of hybrid working following the pandemic.

Not all staff may have had adequate physical space and required facilities to work from home effectively, such as good internet connection or a desk. Indeed, the results from this survey showed dissatisfaction with IT systems as one of the main issues for staff. One study in mental health staff found that some experienced digital connectivity issues when communicating with vulnerable service users, which impacted the ability to build a trusting relationship (19).  Although virtual health care has been available for several years, it had not been widely adopted by health care providers. The pandemic prompted urgent implementation of “tele mental health” as a delivery option in order to reduce transmission of COVID-19 (20). It is important to note the digitization of mental health care has some benefits including improved time efficiency, more flexibility in working hours, reduced waiting times and reduction in non-attendance (21), these benefits have been demonstrated globally across mental health services (22).”

The limitations present strengths of the study. This is not adequate.

Below is the paragraph summarising the limitations of the study:

“This study has some limitations that should be considered during the interpretation of the results. The first is that the data collected was self-reported and is therefore subjective, however the purpose of this study was to gauge the wellbeing of staff rather than diagnose any potential illnesses. Staff working in this environment are trained to support, diagnose, and manage mental health and therefore arguably have a good insight into their own mental health. It should also be acknowledged that the study had a relatively small sample size and was cross-sectional in nature so cannot show change over time; nor can we claim that the sample was truly representative of all staff members within the Trust.  Additionally, the survey was online meaning that it is prone to self-selection bias, all of which limit the generalizability of findings. It also could have been beneficial if there were items included asking what was done well or positives of the new ways of working. Finally, there were no qualitative data collected, elaboration on some of the answers may have helped to provide wider contextual findings.  The survey did not focus on the patient needs and their experiences of what it was like to be living in a secure hospital facility whilst experiencing the pandemic.”

 Perhaps the discussion is the best elaborated section of the whole study. The conclusions are also accurate.

The bibliographical references are adequate and up to date. I hope that my contributions will help you to improve the article.

Round 2

Reviewer 3 Report

Dear authors,

I think the article has improved considerably. Congratulations.

In spite of that the sampling technique if it exists, it is non-random of convenience.

They still do not sufficiently justify the use of an ad hoc survey.

The results that do not reach statistical significance should be noted in the results but not those that do not, which would be developed in the discussion.

I hope that my contributions will help you to improve your article.

The reviewer.

Author Response

Dear Reviewer/Editors

Thank you for your two final comments below. I have responded to these in italics so you can see our feedback.  We have made these changes on the manuscript in track changes.  I hope this is satisfactory.

Reviewer feedback

They still do not sufficiently justify the use of an ad hoc survey.

Response

We have added the following to the manuscript "The survey did not use standardized measures of depression, anxiety, stress or alcohol intake; instead self-report items were used.  This was deemed more appropriate in a group of NHS staff working within the mental health directorate and with whom are very familiar of their own mental health and well-being. Additionally, the survey was not aimed at diagnosing staff with their own mental health condition but was interested in staff self-perception of their own well-being. "

Reviewer feedback

The results that do not reach statistical significance should be noted in the results but not those that do not, which would be developed in the discussion.

Response

We have removed the non-significant results from the results section and added some text to the discussion.  See track changes on the manuscript.

We look forward to hearing your outcome.

Best wishes

Amanda