**Mental Health Challenges during the COVID-19 Pandemic**

Editor

**Alfonso Troisi**

MDPI • Basel • Beijing • Wuhan • Barcelona • Belgrade • Manchester • Tokyo • Cluj • Tianjin

*Editor* Alfonso Troisi Department of Systems Medicine University of Rome Tor Vergata Rome Italy

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Special Issue published online in the open access journal *Journal of Clinical Medicine* (ISSN 2077-0383) (available at: www.mdpi.com/journal/jcm/special issues/Mental Health Pandemic).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-7195-9 (Hbk) ISBN 978-3-0365-7194-2 (PDF)**

Cover image courtesy of Maria Claudia Blanco

A depressed woman wearing a mask. Still from short film: "Au bord du delire" directed by Maria ´ Claudia Blanco, La Femis Production, FR, 2022 ´

© 2023 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications.

The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND.

## **Contents**


Fear of COVID-19 among Healthcare Workers: The Role of Neuroticism and Fearful Attachment Reprinted from: *J. Clin. Med.* **2021**, *10*, 4358, doi:10.3390/jcm10194358 ................ **113**


## **About the Editor**

#### **Alfonso Troisi**

Alfonso Troisi is a Professor of Psychiatry at the International Medical School of the University of Rome Tor Vergata. In addition, he is director of the outpatient treatment program for eating disorders at the Psychiatric Clinic of the Medical Center of the University of Rome Tor Vergata.

Alfonso Troisi has published over 200 papers and book chapters. He co-authored the book Darwinian Psychiatry (Oxford University Press, 1998) and authored the books The Painted Mind: Behavioral Science Reflected in Great Paintings (Oxford University Press, 2017) and Bariatric Psychology and Psychiatry (Springer, 2020). His Hirsch index is 40 based on the Scopus database and 55 based on the Google Scholar database. His name was included in the list of the World's Top Scientists published in October 2020 by the University of Stanford (PLoS Biol 18(10): e3000918). Among the 159,684 scientists in all disciplines, Alfonso Troisi's rank was 51,167. Among the limited number of psychiatrists included in the list (3,519), Alfonso Troisi's rank was 1,070.

The breadth of his current research covers: the analysis of nonverbal behavior in psychiatric patients, the relationship between adult attachment style and psychopathology, the role of social cognition deficits in psychoses, the impact of adverse early experiences on adult mental health and the psychiatric aspects of obesity. Professor Troisi was instrumental in promoting the development of the Darwinian approach for the study of mental disorders and in showing the clinical utility of concepts and methods derived from evolutionary disciplines including human ethology, behavioral ecology and evolutionary psychology. In line with his prolonged activity in this interdisciplinary endeavor, he has been nominated as a member of the Task Force for Evolutionary Psychiatry of the World Federation of Societies of Biological Psychiatry (WFSBP) and the Section on Evolutionary Psychiatry of the World Psychiatric Association (WPA).

## **Preface to "Mental Health Challenges during the COVID-19 Pandemic"**

The papers included in this reprint were originally published in the Special Issue of the Journal of Clinical Medicine entitled, "Mental Health Challenges during the COVID-19 Pandemic". The Special Issue was launched in early 2021 and closed in late 2022, covering the most critical phase of the COVID-19 pandemic. The aim of the Special Issue was to provide readers with updated research findings on the psychological and psychiatric consequences of the pandemic. Review papers, clinical studies and theoretical models were equally considered for publication. The response from potential contributors went beyond my expectations, with there being many submissions and, ultimately, 17 papers accepted for publication.

The wide scope of the reprint reflects the complexity of the impact of the COVID-19 pandemic on mental health. The focus of the studies ranges from the psychological consequences of social distancing and the closure of schools to the neurological consequences of the viral infection, and from the stressful impact of an increased workload on frontline health workers to the unmet needs of special populations such as pregnant women, veterans and patients with neurodevelopmental disorders. Thanks to its richness in content and worldwide perspectives (contributors were from Canada, Croatia, Greece, India, Israel, Italy, Pakistan, the Philippines, Poland, Spain, and the USA), this reprint is a valuable source of information to researchers and clinicians interested in the mental health implications of the COVID-19 pandemic.

Finally, I wish to thank Phoebe Zhang, Assistant Editor at the MDPI Branch Office, for her excellent editorial support.

> **Alfonso Troisi** *Editor*

## *Editorial* **Mental Health Challenges during the COVID-19 Pandemic**

**Alfonso Troisi**

International Medical School, University of Rome Tor Vergata, via Montpellier 1, 00133 Rome, Italy; alfonso.troisi@uniroma2.it

The impact of the COVID-19 pandemic on mental health has unveiled the complexity of the relationship between psychiatry and the rest of medicine, as clearly shown by the collection of studies published in this Special Issue entitled "Mental Health Challenges during the COVID-19 Pandemic".

Mental health and well-being depend on the combination of many individual variables, including genetic background, physiological homeostasis, child rearing experiences, socioeconomic conditions, lifestyle, and interpersonal relationships. Although these variables are partly interconnected, they belong to distinct levels of analysis and are the object of study of different disciplines. It is uncommon to observe a single medical condition causing adverse effects on mental health through its concurrent impact on many of the variables listed above. Yet, this is exactly what has been happening with the COVID-19 pandemic.

The papers by Muñoz-Fernández et al. [1] and Kavvadas et al. [2] show that home confinement and alteration of daily activities were associated with increased levels of negative affectivity in Spanish adolescents and Greek university students, respectively. These findings are in line with growing evidence showing the importance of rewarding interpersonal relationships for establishing and maintaining optimal levels of psychological well-being [3]. This explains why preventive measures that proved to be effective in terms of controlling the spread of the virus led to other problems, particularly relating to the mental health of younger people.

Reviewing research and clinical data on the neurological effects of the SARS-CoV-2 infection, the papers by Zia et al. [4] and Ali Awan et al. [5] focused on the other extreme of the continuum that extends from the social to the organic. COVID-19 may be associated with a variety of neurologic complications, and several plausible mechanisms exist to account for these observations. As the current understanding of COVID-19 continues to evolve, a synthesis of the literature on the neurological impact of this novel virus may help inform clinical management and highlight potentially important avenues of investigation. Interestingly, the pathogenesis of neurological damage can involve indirect mechanisms, as shown by Rajagopalan et al. [6], who found alterations in fetal brainstem structure associated with increased maternal perception of pandemic-related stress in pregnant women.

Pregnant women belong to those special populations that have been responding to the challenge of the COVID-19 pandemic in peculiar ways, as shown by the review paper by Mazurchiewicz et al. [7]. Other special populations were investigated by Letica-Crepulja et al. [8], who analyzed symptom levels and coping strategies during the COVID-19 pandemic among treatment-seeking veterans with pre-existing post-traumatic disorder (PTSD), and by Marino et al. [9], who assessed the efficacy of a web-based remote training program in the management of behavioral disorders of children with autism spectrum disorders. In effect, one of the few positive aspects imposed by the COVID-19 pandemic is that telehealth has been rapidly deployed to help meet critical mental health needs.

Not only individual but also group variables may modulate the impact of the COVID-19 on mental health. Cultural differences should not be neglected when developing public health strategies to mitigate the adverse effects of stress, social isolation, fear, and uncertainty. Emodi-Perlam et al. [10] conducted a cross-sectional online survey and found major

**Citation:** Troisi, A. Mental Health Challenges during the COVID-19 Pandemic. *J. Clin. Med.* **2023**, *12*, 1213. https://doi.org/10.3390/ jcm12031213

Received: 6 January 2023 Accepted: 30 January 2023 Published: 3 February 2023

**Copyright:** © 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

differences in personal worries about physical health, finances, and relations with relatives and friends of participants living in Canada, Israel, and Poland. These findings are consistent with one of the basic postulates of contemporary psychotraumatology: the individual response to stressors and traumatic events depends in part on the cultural context in which the person lives and the order and priority of ideological values [11].

The paper by Gesi et al. [12] reports on the number and characteristics of subjects accessing the emergency rooms for suicidal behavior in three Emergency Departments in Lombardy (Italy) before (2019) and during the first wave (2020) of the COVID-19 pandemic. The proportion of subjects accessing the Emergency Department for suicidality was significantly higher in 2020 than in 2019. Interestingly, during the pandemic, a greater proportion of subjects did not show any mental disorders and were psychotropic drug-free.

Five of the 16 papers published in this Special Issue focused on the mental health problems of healthcare workers (HCWs) working in COVID-19 healthcare facilities. HCWs face a high risk of contracting a potentially severe viral infection, as shown by mortality statistics. At the same time, compared to the general population, they have a better knowledge of infection risk factors and are consistently adopting preventive measures because of their professional duties. Thus, they are the ideal sample to study the interaction between emotional and rational psychological factors in modulating individual levels of fear of infection and its impact on mental health. Olaya et al. [13] carried out a systematic review and meta-analysis of the prevalence of depression among HCWs during the first wave of the COVID-19 pandemic. They found that almost half of the frontline HCWs showed increased levels of depression. The papers by Perego et al. [14] and Soto-Cámara et al. [15] identified several variables that act as risk factors for the development of depression and anxiety in HCWs (i.e., female gender, less work experience, lower levels of perceived social support, living with minors). Troisi et al. [16] found that personality was a significant predictor of fear of infection in HCWs working in a COVID-19 university hospital. Those participants who reported a more intense fear of infection had higher levels of neuroticism and fearful attachment. Considering that excessive fear of infection can put at risk HCWs' psychological well-being and occupational efficiency, these findings can be useful to identify vulnerable subgroups and to implement selective programs of prevention based on counseling and psychological support. In their paper, Llorente-Alonso et al. [17] discuss the utility of selective programs such as job crafting and psychological empowerment to reduce the emotional distress of HCWs fighting the COVID-19 pandemic.

The burden of taking care of COVID-19 patients falls not only on HCWs, but also on their family caregivers. Apostol-Nicodemus et al. [18] carried out a cohort prospective study to assess the psychosocial impact of the pandemic on Philippine families of adult COVID-19 patients in isolation. They found that 43.2% of the caregivers had anxiety symptoms and 16.2% had depressive symptoms two weeks after the discharge of their relatives with a COVID-19 infection.

Before the COVID-19 pandemic, it was difficult to imagine that the worldwide spread of a respiratory infectious disease could have so many implications for mental health. The papers published in this Special Issue help us to understand why this has been happening. By definition, psychiatry is an interdisciplinary field extending from the investigation of neural correlates to the analysis of social dynamics [19]. The challenge of facing a global infectious threat has confirmed the importance of psychiatry for the rest of medicine and the necessity of considering the relationship between physical and mental health in terms of bidirectional pathways.

In this regard, one specific aspect is worth discussing. The COVID-19 pandemic has revealed how primitive emotional reactions to the risk of being infected by a potentially severe contagious disease can impact prevention and treatment programs.

There is a substantial difference between the fear of infection and the fear of the degenerative diseases that rank at the top of the morbidity statistics in affluent countries. Cancer, Alzheimer's disease, heart disease, stroke, and diabetes are evolutionary novelties because their etiology and pathogenesis depend largely on risk factors and life habits that

are typical of modern environments (e.g., extended longevity, high-calorie diet, sedentary lifestyle, obesity, smoking, drinking alcohol, pollution, etc.). Our ancestors living in the natural environment were not exposed to these risk factors and, therefore, they had an infinitesimal likelihood of getting cancer or developing senile dementia. Meanwhile, they had a very high likelihood of dying from an infection. From an evolutionary perspective, this means that infectious diseases have exerted strong selective pressures on human psychology and behavior.

Selection pressures have reinforced our defenses against infections by causing the evolution of a behavioral immune system that is separate from, and complementary to, the physiological immune system. The behavioral immune system includes a set of proactive mechanisms that inhibit contact with pathogens in the first place. These mechanisms offer a sort of psychological and behavioral prophylaxis against infection [20,21]. Like the physiological immune system, the behavioral immune system includes both detection and response mechanisms. When an external cue connoting infection risk (e.g., seeing another person with symptoms of infectious disease) is detected, it triggers a cascade of emotional and behavioral responses that minimize the infection risk (e.g., through social avoidance of people who appear to pose an infection risk). Fear of infection and pathogen disgust sensitivity are the two psychological mechanisms serving the adaptive function of the behavioral immune system [22].

Fear and disgust are deeply rooted in our emotional brain, and their activation can interfere with the implementation of public health strategies based on rational decisions. For example, one study reported a correlation between higher pathogen disgust sensitivity and negative attitudes toward COVID-19 vaccination [23]. A possible explanation for the negative impact of high pathogen disgust sensitivity on vaccination adherence is that vaccines are administered in ways that in and by themselves are cues to contamination, such as puncturing the skin or the inhalation or ingestion of a foreign substance [24].

One should consider that vaccination is an evolutionary novelty not directly linked with the cues that activate the behavioral immune system. Accordingly, the intention to vaccinate is a deliberate, conscious choice which might be only partially related to individual differences in germ aversion. In effect, when studies have focused on preventive measures other than vaccination, the functional utility of the behavioral immune system for combating the COVID-19 pandemic has emerged clearly. Shook et al. [25] found that germ aversion correlated with the frequency of preventive health behaviors such as social distancing, avoiding touching one's face, wearing a facemask, hand washing and disinfecting objects. Cox et al. [26] reported that heightened disgust proneness before the pandemic resulted in an increased use of protective behaviors during the pandemic. Makhanova and Shepherd [27] found that germ aversion was negatively associated with the number of face-to-face interactions and positively associated with anxiety about social proximity.

The study of the behavioral immune system is a paradigmatic model for understanding the complex relationship between psychiatry and the rest of medicine. For example, there is evidence that, when social distancing results in social isolation, the functionality of the physiological immune system is reduced [28]. By contrast, the activity of the physiological immune system is enhanced by visual exposure to symptoms of infectious disease in others [29]. In conclusion, a lesson we are learning from the COVID-19 pandemic is that public health strategies should routinely include psychiatry and allied disciplines within the theoretical framework developed for optimizing prevention and treatment programs.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Article* **Public Concerns during the COVID-19 Lockdown: A Multicultural Cross-Sectional Study among Internet Survey Respondents in Three Countries**

**Alona Emodi-Perlman 1,† , Ilana Eli 1,† , Nir Uziel 1, Joanna Smardz <sup>2</sup> , Anahat Khehra <sup>3</sup> , Efrat Gilon <sup>1</sup> , Gniewko Wieckiewicz <sup>4</sup> , Liran Levin <sup>3</sup> and Mieszko Wieckiewicz 2,\***


**Abstract:** (1) Background: this study aimed to evaluate the worries, anxiety, and depression in the public during the initial coronavirus disease 2019 (COVID-19) pandemic lockdown in three culturally different groups of internet survey respondents: Middle Eastern (Israel), European (Poland), and North American (Canada). (2) Methods: a cross-sectional online survey was conducted in the mentioned countries during the lockdown periods. The survey included a demographic questionnaire, a questionnaire on personal concerns, and the Patient Health Questionnaire-4 (PHQ-4). A total of 2207 people successfully completed the survey. (3) Results: Polish respondents were the most concerned about being infected. Canadian respondents worried the most about their finances, relations with relatives and friends, and both physical and mental health. Polish respondents worried the least about their physical health, and Israeli respondents worried the least about their mental health and relations with relatives and friends. Canadian respondents obtained the highest score in the PHQ-4, while the scores of Israeli respondents were the lowest. (4) Conclusions: various factors should be considered while formulating appropriate solutions in emergency circumstances such as a pandemic. Understanding these factors will aid in the development of strategies to mitigate the adverse effects of stress, social isolation, and uncertainty on the well-being and mental health of culturally different societies.

**Keywords:** COVID-19; SARS-CoV-2; coronavirus pandemic; anxiety; depression; mental health

#### **1. Introduction**

Coronavirus disease 2019 (COVID-19) is a novel severe respiratory syndrome caused by a new betacoronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1–5]. The COVID-19 pandemic has caught the world by surprise. Within a relatively short time, most countries were affected and responded with partial-to-total lockdowns to curb disease spread [6]. The daily lives of people were greatly affected in every aspect, including restrictions in socializing, working (up to full quarantine), and/or planning for the future.

Mid-March 2020, the rate of contraction and the rate of deaths by COVID-19 constantly rose. On 15 March 2020, the number of daily confirmed cases in Israel was 2 per million people. Within one month (on 15 April 2020), the number rose to 45.32 per million (relative

**Citation:** Emodi-Perlman, A.; Eli, I.; Uziel, N.; Smardz, J.; Khehra, A.; Gilon, E.; Wieckiewicz, G.; Levin, L.; Wieckiewicz, M. Public Concerns during the COVID-19 Lockdown: A Multicultural Cross-Sectional Study among Internet Survey Respondents in Three Countries. *J. Clin. Med.* **2021**, *10*, 1577. https://doi.org/10.3390/ jcm10081577

Academic Editor: Mauro Giovanni Carta

Received: 4 February 2021 Accepted: 2 April 2021 Published: 8 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

change of +2169%). The total number of confirmed deaths from COVID-19 during that period (15 March till 15 April) rose from 1 to 139 (relative change of +13,800%) [7]. On 19 March 2020, the Israeli government declared an almost complete lockdown. All schools, kindergartens, and universities were shut down, and schooling was continued (partially) through the internet. Leaving home to a distance greater than 100 m was prohibited except in the cases of emergency, shopping for basic products, or work in vital posts (specifically defined by the government). Most adults were either put on a no-pay leave or were instructed to work from home. Personal contact with non-cohabitating family members and/or with friends was prohibited, even during traditional religious family gatherings such as Passover.

The situation in Poland on 15 March 2020 was that of 0.41 per million COVID-19 confirmed cases. Within one month, the number rose to 8.97 per million (relative change of +2100%). The total number of confirmed deaths from COVID-19 during that period rose from 3 to 286 (relative change of +9433%) [7]. An almost complete lockdown was implemented in the country in mid-March, with regulations similar to those in Israel except for no limitation on the distance from home.

In Canada, the number of daily confirmed cases on 15 March 2020 was 0.71 per million people. Within one month (on 15 April 2020), the number rose to 34.32 per million (relative change of +4725%). During that period of time, the total number of confirmed death cases due to COVID-19 rose from 1 to 1008 (relative change of +100,700%) [7]. In the Province of Alberta, a state of public health emergency was declared on 17 March 2020. Schools, shops, arenas, restaurants, places of worship, recreation centers, bars, etc. were closed. All the other provinces across Canada declared a similar state of public health emergency.

Undoubtedly, all communities experienced feelings of separation, apprehension, stress, anxiety, and even depression. Uncertainty regarding the length of the situation and its final consequences only added to the anguish.

A recent review on the psychological impact of quarantine reported that it had substantial negative psychological effects on people, contributing to posttraumatic stress symptoms, confusion, and anger. Stressors included longer durations of quarantine, infection fears, frustration, boredom, inadequate supplies, insufficient information, financial loss, and stigma [8].

A number of studies have been published about the emotional aspects of the COVID-19 pandemic. For instance, Germani et al. described the impact of this pandemic on young Italian adults [9]. In a study conducted in Denmark, the researchers reported that the psychological well-being of subjects was negatively affected by the situation [10]. In a study performed in China, more than half of the participants showed a significant psychological disturbance due to the pandemic, while in a study from the United States, nearly half of the participants were found to be anxious [11,12]. In a study conducted in India, Varshney et al. reported that the factors predicting higher psychological impact among the general public were younger age, female sex, and presence of known physical comorbidity [13]. The infection might also have a putative tropism toward the central nervous system (CNS) that may explain some of the symptoms seen in clinical practice, with possible late neuropsychiatric manifestations [14]. Clearly, the pandemic took its toll on societies worldwide. The effect may vary among cultures and societies. A previous study showed that the COVID-19 pandemic caused significant adverse effects on the psychoemotional status of both Israeli and Polish populations, resulting in the intensification of their orofacial pain. However, the two populations varied in their reaction to the stress. For example, the odds of occurrence of orofacial pain symptoms among Polish subjects were on average over 3 times higher than that among Israeli subjects [15]. Understanding the factors that cause worries, anxiety, and depression among different communities will enable us to develop response strategies to mitigate the adverse effects of stress, social isolation, and uncertainty on well-being, and physical and mental health in culturally different societies.

The present study aimed to evaluate the worries, anxiety, and depression in the public during the initial pandemic lockdown in three culturally different groups of internet survey respondents: Middle Eastern (Israel), European (Poland), and North American (Canada).

#### **2. Materials and Methods**

A cross-sectional online survey was conducted using anonymous questionnaires. The final questionnaire was compiled from a tool commonly used with regard to anxiety and depression (Patient Health Questionnaie-4 (PHQ-4), as detailed below), and specific questions referring to demographics and concerns specific to the COVID-19. The latter were agreed upon and tested for content validity by a group of subject matter experts (SMEs). The group consisted of four researchers (A.E.-P., I.E., N.U., and E.G.) who work at the Tel Aviv University and have vast academic experience in population studies. Each SMEs proposed questions for the study, and following discussions, the final questions were agreed upon. The questionnaire was compiled in Hebrew and translated to Polish by the Polish group. One of the Israeli researchers (I.E.), who is native in the Polish language, verified the Polish translation by retranslating it to Hebrew and vice versa. The Israeli group translated the questionnaire to English. One of the members of the Canadian group (L.L.), who is native in Hebrew, verified the English translation as above.

SurveyGizmo (www.surveygizmo.com (accessed on 19 April 2020)) was used to collect data. The survey was anonymous. Therefore, due to the need to avoid respondents' identification, and according to the EU regulations, quality control measures could not be practiced.

In Israel, the survey was posted in Hebrew, which assured that only Hebrew-speaking Israeli responders participated. In Poland, the survey was posted in Polish, enabling the participation of only Polish-speaking participants. In Canada, the survey was posted in English, with most of the responders coming from the province of Alberta.

The surveys were posted at least 4 weeks after the implementation of the first lockdown in each of the countries while the pandemic was still progressing at a rapid pace (see below).

In Israel, the survey was posted on 16 April 2020. Since the beginning of the pandemic, the relative change in cumulative confirmed COVID-19 cases per million in Israel at that time was +1,294,700% [7]. The questionnaire was distributed through social media (WhatsApp groups and Facebook groups).

In Canada, collection of data started on 13 May 2020. The relative change in cumulative confirmed cases per million in Canada at that time was +7,356,700% [7]. The questionnaire was distributed through social media (Facebook, Instagram, Reddit, and WhatsApp groups).

In Poland, collection of data started on 29 April 2020. The relative change in the number of cumulative confirmed cases per million in Poland at that time was +1,263,900% [7]. The questionnaire was posted on Reddit (r/Polska subreddit).

All responses were obtained anonymously in all three countries.

The study was conducted in full accordance with the World Medical Association Declaration of Helsinki. In Israel, the Ethics Committee of the Tel Aviv University approved all the study procedures (ID: 0001332-1). In Poland, the study was approved by the Bioethical Committee of the Wroclaw Medical University (ID: KB-302/2020). In Canada, the Research Ethics Board at the University of Alberta approved the study (ID: Pro00100768). Informed consent was obtained from all the subjects as required.

#### *2.1. Instruments*

The following data were collected through questionnaires:

2.1.1. Demographic and General Information

This included the following:

a. Consent to participate in the study


#### 2.1.2. Personal Concerns Regarding the COVID-19 Pandemic

Subjects were requested to indicate the following:


#### 2.1.3. Information on Anxiety and Depression

Subjects responded to the Patient Health Questionnaire-4 (PHQ-4), a brief screening tool used for assessing anxiety and depression. The questionnaire is a four-item inventory and is rated on a 4-point Likert-type scale. It allows for a very brief and accurate measurement of depression and anxiety. The reliability and validity of PHQ-4 have been repeatedly established [17,18]. It has been incorporated as a part of various diagnostic protocols and has been translated to numerous languages, including Hebrew and Polish [19].

The total score of the PHQ-4 ranges from 0 to 12: the higher the score, the higher the chances of the presence of anxiety and depression. The recommended PHQ-4 cutoff scores are as follows: 0–2, normal; 3–5, mild; 6–8, moderate; and 9–12, severe [20]. In the present study, a total PHQ-4 score was used for comparisons among countries, genders, and age groups.

The questionnaire also enables separate assessments of anxiety and depression.

A score of ≥3 for the first two questions suggests anxiety. A score of ≥3 for the two last questions suggests depression. In the present study, scores of the first two questions (anxiety) and of the last two questions (depression) were used for comparisons among countries, gender, and age groups.

The questionnaire referred to the last 2 weeks, namely, to the lockdown period.

#### *2.2. Statistical Analysis*

Data were analyzed using the SPSS software.

A logistic regression analysis was carried out to evaluate subjects' feelings of being at risk of contamination by the virus. Variables that were entered into the equation were country, gender, age, and the interactions among them (age × country, country × gender, age × gender, and age × country × gender).

Four 3-way analyses of variance (ANOVA) were carried out to analyze subjects' worries regarding finances, physical health, mental health, and relations with others. The analyses evaluated the effects of country (×3), gender (×2), age (×2), and possible interactions among them.

Three additional 3-way ANOVA were carried out to analyze subjects' total PHQ-4 score as well as the separate scores of anxiety and of depression. The analyses evaluated the effects of country (×3), gender (×2), age (×2), and possible interactions among them.

#### **3. Results**

In Israel, a total of 867 subjects responded to the questionnaire during the study period, out of whom 80.7% (*n* = 699) completed the questionnaire in full.

In Poland, a total of 1096 subjects responded to the questionnaire during the study period, out of whom 99.63% completed the questionnaire in full (*n* = 1092).

In Canada, a total of 548 subjects responded to the questionnaire during the study period, out of whom 75.9% completed the questionnaire in full (*n* = 416). Among the responders, 93% were Canadians, mostly from the province of Alberta (78.4%).

#### *3.1. Demographics*

The distributions of gender and age of the study participants are presented in Table 1.


**Table 1.** Gender and age distributions.

Significant differences were observed among countries with respect to gender (*p* < 0.000) and age (*p* < 0.0000).

In all three countries, there was a higher proportion of female responders than males, with the number of female responders being the highest in Canada (83%) compared to Poland (58%) and Israel (66%).

In general, the Polish and Canadian participants were younger (76% and 83% in the "young adult" group, respectively) compared to Israeli participants, in which only 30% of participants were in this age group. Due to the small number of respondents in the "older" group (>56 years) in Poland and Canada (2.8% and 3.4%, respectively), this group was collapsed with the "adult" age group and only two age groups were analyzed: the "younger group" (18–35 years) and the "older group" (>35 years).

#### *3.2. Personal Concerns Regarding the COVID-19 Pandemic*

#### 3.2.1. Feeling at Risk of Contamination

Significant differences were found among the respondents with regard to concerns of being at risk of contamination by the virus with respect to country, gender, and age group. Poles felt most at risk of contamination (46.8% positive response) compared to Canadians (21.9% positive response) and Israelis (20.7% positive response).

Logistic regression (with the variables country; gender; age group; and the interactions age × country, country × gender, age × gender, and age × country × gender) showed that, compared to Israelis, the odds of Poles feeling at risk of contamination were over three times as much (odds ratio 3.2, 95% CI 2.37–4.32, *p* < 0.000). In the case of Canadians, the odds of feeling at risk of contamination were almost twice that compared to the Israelis (odds ratio 1.91, 95% CI 1.09–3.35, *p* < 0.05).

Older responders felt in general more at risk than the younger responders (31.4% versus 19.9% positive response, odds ratio 1.87, 95% CI 1.49–2.34, *p* < 0.000).

Women felt in general more at risk than men (35.8% versus 30.7% positive response). The odds of Polish women feeling at risk of contamination was 1.86 compared to Israeli women (95% CI 1.44–2.38, *p* < 0.000).

#### 3.2.2. Worry about Finances

Three-way ANOVA that compared subjects' levels of worry about finances showed the main effects of country (F(2,2173) = 8.53, *p* < 0.000; Israel < Poland < Canada), gender (F(1,2173) = 4.74, *p* < 0.05; male < female), and age (F(1,2173) = 6.33, *p* < 0.05; younger < older).

Significant interactions among the variables country, age, and gender were also observed (F(2,2173) = 6.59, *p* < 0.005). Parameter estimates of the 3rd-order interactions were as follows: (i) country (IL) × gender (F) × age (young)—B = 1.12, 95% CI 0.33–1.90, *p* < 0.005; (ii) country (PL) × gender (F) × age (young)—B = 1.41, 95% CI 0.64–2.17, *p* < 0.000 (*IL* = *Israel*, *PL* = *Poland*, *F* = *female*).

Generally, Poles worried the least (M = 2.38) while Canadians worried the most (M = 2.84). Females worried more than males, and younger subjects worried more than the older age groups. The group that worried the least about finances was Polish males (both age groups, M = 2.27), while the group that worried the most was Canadian younger males (M = 2.33).

#### 3.2.3. Worry about Physical Health

Three-way ANOVA that compared subjects' worries about their physical health showed the main effects of country (F(2,2167) = 8.38, *p* < 0.000; Poland < Israel < Canada) and gender (F(1,2167) = 4.44, *p* < 0.05; male < female). The effect of age was borderline (F(1,2167) = 3.739, *p* = 0.05).

A significant interaction between gender and age was also observed (F(2,2167) = 9.18, *p* < 0.005). Parameter estimates of the 2nd-order interaction were B = −0.721, *p* = 0.02, 95% CI lower bound −1.317, upper bound −0.124.

Generally, Poles worried the least (M = 2.26) while Canadians worried the most (M = 2.61) about their physical health. Females worried more than males. The group that worried the most about their physical health were Canadian older males (M = 2.91), while Israeli adult males showed the least concern (2.10).

#### 3.2.4. Worry about Mental Health

Three-way ANOVA that compared subjects' worries about their mental health showed the main effects of country (F(2,2174) = 44.62, *p* < 0.000; Israel < Poland < Canada), gender (F(1,2174) = 33.61, *p* < 0.000; male < female), and age (F(1,2174) = 15.75, *p* < 0.000; older < younger). No interactions were detected among variables.

Generally, Canadians showed the highest concern about their mental health (M = 3.14) while Israelis showed the lowest concern regarding this aspect of the pandemic (M = 1.97). Females worried more than males and younger subjects, worried more than older subjects.

#### 3.2.5. Worry regarding Relations with Meaningful Figures

Three-way ANOVA that compared subjects' worries about their relationships with meaningful figures in their lives showed the main effects of country (F(2,2172) = 6.75, *p* < 0.001; Israel < Poland < Canada), gender (F(1,2172) = 11.73, *p* < 0.001; male < female), and age (F(1,2172) = 24.48, *p* < 0.000; older < younger). A significant interaction was observed between gender and age (F(2,2172) = 8.05, *p* < 0.000). Parameter estimates of the 2nd-order interaction were B = −0.478, 95% CI lower bound −0.869, CI upper bound −0.086, *p* < 0.05.

Generally, the group that worried the most about their relations with relatives and friends was the Canadian subjects (M = 2.69) while Israelis was the group that worried the least (M = 1.94). Females worried more than males, while younger subjects worried more than older subjects. In the three countries, older females showed higher concern regarding relations with relatives and friends than their male counterparts (M = 2.10 versus 1.96).

#### *3.3. Anxiety and Depression*

#### 3.3.1. Total PHQ-4 Score

Three-way ANOVA of the total PHQ-4 score showed the main effects of country (F(2,2174) = 27.103, *p* < 0.000; Israel < Poland < Canada), gender (F(1,2174) = 38.85, *p* < 0.000; male < female), and age (F(1,2174) = 16.59, *p* < 0.000; older < younger). No interactions were observed among the variables.

All three countries scored within the "mild" category of the total PHQ-4 score. In general, Israeli respondents showed the lowest PHQ-4 scores (M = 2.91) and Canadian respondents showed the highest scores (M = 4.94). Females showed higher scores than males (M = 4.37 vs. 3.27), and younger subjects showed higher scores than the older age group (M = 4.43 vs. 3.24).

#### 3.3.2. Anxiety (According to PHQ-4)

Three-way ANOVA analysis of the separate anxiety scores showed a basically similar pattern for the main effects of country (F(2,2173) = 16.418, *p* < 0.000; Israel < Poland < Canada), gender (F(1,2173) = 46.181, *p* < 0.000; male < female), and age (F(1,2173) = 10.843, *p* < 0.000; older < younger). No interactions were observed among variables.

None of the countries reached a definite score defined of "anxiety" (>3). Nevertheless, Israelis were the least anxious group (M = 1.46) while Canadians were the most anxious one (M = 2.65). Females were more anxious than males (M = 2.22 vs. 1.42), and younger subjects were more anxious than the older age group (M = 2.14 vs. 1.61).

#### 3.3.3. Depression (According to PHQ-4)

A similar pattern was also revealed for the separate assessment of depression. Three-way ANOVA of the separate depression scores showed the main effects of country (F(2,2174) = 49.00, *p* < 0.000; Israel < Poland < Canada), gender (F(1,2174) = 29.94, *p* < 0.000; male < female), and age (F(1,2174) = 18.70, *p* < 0.000; older < younger). No interactions were observed among variables.

None of the countries reached a score definition of "depression" (>3). Nevertheless, Israelis were the least depressed group (M = 1.14) while Canadians were the most depressed one (M = 2.31). Females were more depressed than males (M = 2.01 vs. 1.54), and younger subjects were more depressed than the older age group (M = 2.12 vs. 1.40).

#### **4. Discussion**

In late December 2019, a new unfamiliar public health threat, the COVID-19 pandemic, began to spread around the world. With almost complete uncertainty concerning the method of virus spread and the modes of treatment, insufficient availability of local health services, and lack of a vaccine or efficient drugs for treatment, most countries adopted the policies of social distancing and partial-to-total lockdown. The new situation burdened people not only with immediate severe health threats but also with economic uncertainty and social isolation, causing further potential deleterious effects on their mental and physical health.

From recent studies indicating the direct and indirect influences of the pandemic, it is apparent that subjects around the world might react differently to the new stressful situation. In the present study, subjects from different regions—Middle Eastern, European, and North American—were compared in terms of their personal worries and emotional reactions. A summary of the main effects of the study is presented in Table 2.



\* Variables as defined in the text; \*\* country: IL—Israel, PL—Poland, CA—Canada; PHQ-4—Patient Health Questionnaire-4.

The study showed that subjects' reactions to the situation varied significantly among the three countries. Except for worry about physical health, Canadians appeared to be the most worried. Israelis appeared to be consistently less worried than the Canadians, while the Poles showed mixed behavior.

Israelis are used to changing their routine way of life within a short period as they frequently face sudden emergencies. The country is under constant security threats from the outside (at the borders) and the inside (terror attacks). Every few years, an emergency arises requiring the citizens to react quickly (i.e., stay in shelters). In general, the armed forces are highly trusted to contain the situation and to bring back normalcy. During the initial stage of the COVID-19 pandemic, the country reacted quickly. Borders were closed, and the army was mobilized to help. Although the political system underwent continuous changes (despite several rounds of indecisive elections, there was great difficulty in creating a stable government), the pandemic situation was generally under control.

Unlike Israel, Canada has not faced any major emergencies for many decades. Although the country was affected by severe acute respiratory syndrome (SARS) in 2003, which led to partial quarantine, the effects were evident mainly in Ontario [21]. Based on the SARS experience, one could assume that Canadians were more prepared for the COVID-19 pandemic and, therefore, less worried about its different aspects compared to other countries such as Poland, which had no such experience. Interestingly, the results were found to be quite the opposite. The study showed that Canadians worried more than the two other groups regarding their finances, physical health, mental health, and relations with relatives and friends.

Apparently, the unexpected threat rocked the lives of the Canadian people to a greater extent than the other two countries. The results were in line with a previous publication regarding the impact of COVID-19 in Canada, which showed that 87% of the population were concerned about the impact on vulnerable people, 21% about their own health, 36% about family stress from confinement, and 34% about maintaining social ties [22].

During the initial stage of the pandemic in Poland, a public opinion study showed that most of Polish citizens were concerned about their own health and about the health of their relatives, especially those of old age [23]. A previously published survey also showed that up to 57% of Polish citizens had concerns about their finances due to the COVID-19 pandemic [24]. The present study confirmed these results to some extent as Polish participants showed the second highest concern about mental health and relations with relatives and friends.

Among the three studied countries, one of the most prominent differences was observed for the subjects' perceptions of being at risk of virus contamination. The odds of Poles and Canadian to worry about this issue were two to three fold higher than those of the Israelis. This may be explained by the advanced and generally good public health services available in Israel. In the country, all citizens have governmental health insurance and are entitled to all the necessary health services with no additional costs (besides a mandatory monthly fee). Hospitals are required to maintain high medical standards, and medical personnel are well-trained. Up until now, Israel has been efficiently carrying out vaccination processes against COVID-19, with high percentages of the adult population being already successfully vaccinated.

Although healthcare in Poland does not differ significantly from that in Israel and medical costs are reimbursed to some extent, access to some specialist procedures is limited. Due to the extended wait time for their implementation, there is distrust among Poles about their national healthcare system.

Apparently, the differences among countries with regard to subjects' worries about the situation are also reflected in their emotional status (anxiety and depression).

In a recent Canadian survey series, performed during the COVID-19 pandemic only half of the Canadians reported excellent and very good mental health [25]. In Poland, the ranges of depressive, anxiety, and stress symptoms were around 50% during the COVID-19 outbreak [26]. In Israel, more than one-third of the responders reported stress and anxiety due to the crisis [27].

The impact of COVID-19 lockdowns on the mental health of people suffering from psychiatric disorders may be even more substantial and may lead to an increase in general psychopathology, anxiety, fear, and stress related to the quarantine. Tele-psychiatry was implemented in some countries to address population concerns and worries. Acceptance of telemedicine by decision makers might allow for quick response in times where face-to-face visits are not accessible [28,29]. As the pandemic still prevails around the globe, such an approach should be seriously considered.

The differences between genders, in levels of worries, anxiety, and depression, are not surprising. As a global phenomenon, women seem to be hardest hit by unemployment due to the pandemic [30–33]. Regretfully, information regarding domestic and professional variables such as family responsibilities, education level, employment type, or annual income was not collected in the present study. According to the United Nation policy brief, the pandemic amplified and heightened all preexisting inequalities (such as gender inequality) and exposed vulnerabilities in social, political, and economic systems, which in turn amplify the impacts of the pandemic [34].

The effect of age on the personal concerns and emotional aspects of subjects reported in the present study was inconsistent. Findlay et al. showed that Canadian youth had a higher risk for poor mental health both before and during the COVID-19 pandemic, while Okruszek et al. showed that young Polish adults were more concerned about the collapse of healthcare than any other issue [35–37]. This confirms that, in the days of health threats and financial uncertainty, anxiety, depression, and worries about mental health and social relations affect young adults worldwide. It is noteworthy that the group of responders over the age of 56 years in the present study was especially small in both Canada and Poland and therefore could not be properly analyzed. Possibly, some of the subjects at that age are not active in the social media used to distribute the questionnaire.

The results showed complex interactions among some of the study variables. Men and women belonging to different age groups reacted differently in each of the examined countries. These interactions suggest that the reactions of different subgroups (gender/age) are affected by subtle factors that are specific for each country. Further research will be needed to better analyze these differences.

Personal concerns evoked by the pandemic not only adversely affect the emotional status of subjects but also can also take a toll on physiological phenomena [37–39]. A previous study showed that personal worries, depression, and anxiety during the COVID-19 pandemic can act as predictors of symptoms that cause aggravation of chronic pain, the effect of which varies between communities [15]. Researchers, clinicians, and politicians are in need of literature that can help them understand the impact of this crisis on people's emotional and mental health, which can be long lasting.

Study limitations: although an effort was made to perform the survey at similar times as far as the pandemic progression is concerned, the slightly later timing of gathering data in Canada might have biased the results in this country. Moreover, the study was carried out as an anonymous internet survey with no actual ability to control the participants. As a result, the study groups differ as far as gender and age are concerned and are not necessarily representative of the country's populations. As pointed above, there was a under representation of the older age group (>56), which might have shown a different pattern of behavior (such as being more worried about contamination and physical health but possibly less worried about financial issues than the younger groups).

#### **5. Conclusions**

Better understanding the factors that cause worries, anxiety, and depression among different communities will enable us to develop response strategies to mitigate the adverse effects of stress, social isolation, and uncertainty on well-being, and physical and mental health in culturally different societies.

**Author Contributions:** Study conception and design, A.E.-P. and I.E.; data collection, A.E.-P., I.E., G.W., N.U., A.K., L.L., and E.G.; data analysis and interpretation, A.E.-P. and I.E.; article drafting, A.E.-P., I.E., A.K., L.L., M.W., and J.S.; critical revision of the article, A.E.-P., I.E., and M.W. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and was approved by the following: in Israel, the Ethics Committee of the Tel Aviv University approved all the study procedures (ID: 0001332-1); in Poland, the Bioethical Committee of the Wroclaw Medical University approved the procedures (ID: KB-302/2020); in Canada, the Research Ethics Board at the University of Alberta approved the study (ID: Pro00100768).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on reasonable request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Noelia Muñoz-Fernández <sup>1</sup> and Ana Rodríguez-Meirinhos 2,\***


**Abstract:** The global outbreak of COVID-19 has brought changes in adolescents' daily routines, restrictions to in-person interactions, and serious concerns about the situation. The purpose of this study was to explore COVID-19-related concerns, daily routines, and online peer activities during the confinement period according to sex and age groups. Additionally, the relationship of these factors and optimism along with adolescents' frustration was examined. Participants included 1246 Spanish students aged 16–25 years old (M = 19.57; SD = 2.53; 70.8% girls). The results indicated that the top concern was their studies. COVID-19-related concerns, daily routines, and online peer activities varied by sex and age. Findings also revealed moderate to high levels of frustration, which were associated with adolescents' main concerns, online peer activities, maintaining routines, and optimism. The results are discussed in light of their implications in designing support programs and resources to reduce the psychological impact of COVID-19 on adolescent mental health.

**Keywords:** COVID-19; adolescents; concerns; activities; frustration

**Citation:** Muñoz-Fernández, N.; Rodríguez-Meirinhos, A. Adolescents' Concerns, Routines, Peer Activities, Frustration, and Optimism in the Time of COVID-19 Confinement in Spain. *J. Clin. Med.* **2021**, *10*, 798. https://doi.org/10.3390/jcm10040798

Academic Editor: Erik Roj Larsen

Received: 25 January 2021 Accepted: 12 February 2021 Published: 16 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

#### **1. Introduction**

The novel coronavirus disease (COVID-19) has become a major public health concern and was declared a pandemic by the World Health Organization in March 2020. Since the first cases were confirmed, the epidemic has rapidly spread worldwide. To control the outbreak, the Spanish government, similar to the governments of other countries all over the world, ordered a nationwide restrictive confinement for three months, starting from mid-March. During this period, a high restriction of mobility was imposed. Citizens were not allowed to leave their house except for essential reasons (e.g., buying supplies in the supermarket or pharmacy, taking care of vulnerable people, or going to work if teleworking was not possible). Schools were closed and other educational, social, cultural, artistic, sporting, or similar activities were canceled.

Although adolescents and young people are at lower risk of critical COVID-19 symptoms [1], strict confinement orders entailed important changes in daily routines and social interactions. With schools closed, youths had to adapt quickly to new remote learning environments, while uncertainties about their studies and their near academic future emerged [2]. These difficulties have been particularly marked among higher education students, who were preparing for university entrance exams, taking semester assessments, or doing practical training that could not adequately be replaced by online instruction [3]. Other extracurricular (e.g., educational and sports) and out-of-home leisure activities (e.g., hanging out with friends and dating), which provide valuable resources for socialization [4] were canceled, leaving adolescents with limited opportunities for face-to-face social contact. Although online social networks and instant messaging apps may have compensated for these shortages [5], there is emerging evidence that COVID-19 confinement has increased the risk of social isolation and loneliness among youths [6]. However, changes in interpersonal relationships go beyond friendships, also affecting family interactions. Family dynamics have had to change according to the stay-at-home mandates, forcing families

to spend all their time together [7]. As a consequence, adolescents may have experienced restrictions in their personal space, while parents have faced an increase in daily stressors (including demands of caregiving and parenting, teleworking, home-schooling, threat of contagion, or financial insecurity, among others).

Exposure to COVID-19 challenges is having a substantial cost on psychological wellbeing [8,9]. To date, a growing number of studies among children and youths have reported high rates of anxiety and stress, along with difficulties with concentrating and worrying [10–12]. Although research is still progressing, it seems that from the very beginning, concerns about the consequences of COVID-19 have been particularly salient among young people. In early reports at the first stage of the outbreak in China, college students acknowledged being fearful of what was happening [13]. As this epidemic spread worldwide, this finding has been confirmed by more recent studies with adolescents and university students. These studies have found moderate levels of concern about COVID-19 [14], which were significantly related with increased anxiety and depression [15–20]. However, these worries may not be experienced by all young people in the same way. Indeed, studies indicate that levels of anxiety and fear about COVID-19 increase with age [14] and are higher among females [12,17,18,21]. Given these findings, research aimed at identifying the particular concerns that young people are experiencing would help to define better how they are coping with this crisis. To date, research has focused more on analyzing the overall levels of fear of COVID-19, e.g., [14,17,22], rather than describing adolescents' reasons for these concerns. Among the few available sources of evidence, studies indicate that the main worries of adolescents revolve around the issues that currently cause more uncertainty: the health of those more vulnerable to COVID-19 and the economic situation [23,24]. Although less studied, school-related concerns may have also become magnified [12,25], especially when considering that the closure of schools during the pandemic is a global issue [2] and that education is of central importance for adolescents' future.

Another important, yet less studied, consequence of the confinement may be frustration [26]. As shown by previous research, feelings of frustration emerge in situations in which people feel pressured to comply with rules which are perceived as a threat to their freedom [27]. Although adolescents may have different motivations to adhere to imposed measures, the confinement situation has certainly involved a number of restrictions on individual choice and decision-making (i.e., limitations on non-essential movements, prohibition of gathering with friends, and the obligation of wearing a face mask), which may have resulted in elevated levels of frustration. However, in the context of this pandemic, individual experiences of feeling thwarted may have been not only a common but a harmful consequence of the lockdown. Accordingly, ample research has well established that the psychological costs of frustration are related to higher levels of stress, depression, or anxiety, among others (see [28] for a review).

While there is no doubt that COVID-19 is having negative consequences in different areas of young people's lives, research investigating the factors that help adolescents handle this stressful experience is very valuable [29]. Over the last months, social and health agencies have offered guidelines to support adolescents. However, empirical evidence on the factors that mitigate the risk of psychological distress is still lacking. Among these, experts have outlined the importance of establishing stay-at-home routines and doing a variety of activities (e.g., school-work, hobbies, and exercising) to reduce the psychological stress of the confinement [30], yet few studies have examined the role that these activities play in adolescent psychological functioning. On an interpersonal level, interactions with peers may also have an important influence on the way that adolescents experience this health crisis. In adolescence, peer relationships become especially salient as they contribute not only to satisfying the needs of intimacy and companionship but also to navigate the challenges of this developmental stage [31]. During the lockdown, youths stayed connected with their friends and classmates although face-to-face interactions moved to an online setting [32]. Technology and social media may have indeed helped to compensate for the lack of in-person interactions. Nonetheless, little is known about the specific online

activities that youths have been engaging in with their peers, and the role of these activities in adolescent psychological outcomes during the lockdown. Finally, some authors have drawn attention to optimism as a factor that may favor better adaptive outcomes under challenging situations [31]. As such, recent studies demonstrated that keeping a more optimistic view of the situation was related to lower rates of anxiety and depressive symptoms during the COVID-19 pandemic [32,33].

The present study focused on the psychological impact of the COVID-19 lockdown in order to offer insights into the factors related to adolescents' feelings of frustration. Specifically, the first aim was to describe adolescents' main concerns about the impact of COVID-19. It was expected that school-related concerns [12,25], concerns about the health of those more vulnerable to COVID-19, and concerns about the economic situation [23,24] would be the most salient worries among young people. In addition, it was hypothesized that COVID-19-related concerns would be higher among females [12,17,18,21] and participants in higher age groups [14]. Second, this study explored the activities that, on a daily basis or with their peers, adolescents have been engaging in during the COVID-19 lockdown. Participants were expected to engage in several stay-at-home routines [30]. Besides, considering that social media by its nature may compensate for a lack of face-to-face social interactions [32], we hypothesized that youths may have been using technology to feel supported, loved, or cared for by their friends during the confinement period. Third, given that frustration experiences have significant costs on wellbeing and diminished functioning, this study also examined adolescents' feelings of frustration during the confinement period and their link with COVID-19-related concerns, daily routines, online activities with peers, and optimism. Additionally, sex and age differences were examined. Adolescents were expected to display moderate to high levels of frustration [26]. As literature addressing the role of adolescents' concerns and routines on frustration is still scant, the association between these variables was addressed in an exploratory fashion.

#### **2. Materials and Methods**

#### *2.1. Participants and Procedure*

The study sample consisted of 1246 students (70.8% girls) from Spain. The participant age range was 16–25 years (M = 19.57; SD = 2.53). Following Steinberg [34], participants in the 16 to 18 age group were considered middle adolescents, meanwhile, participants in the 19 to 25 age group were considered late adolescents. In this study, 42.3% (*n* = 527) were middle adolescents and 57.7% (*n* = 719) were late adolescents. The distribution of participants according to sex was similar between the younger (16–18 years old; 70.4% girls) and the older (19–25 years old; 71.07% girls) age groups, χ2(1) = 0.07, *p* > 0.05. In terms of geographic distribution, most participants (88.9%) came from the southern area of Spain (Andalucía), although students from all other regions of Spain were also represented in the sample. All respondents were students enrolled in compulsory secondary education or professional training (17.9%), post-secondary education (24%), or university (58.1%). During the confinement, participants were living in two-parent families (86.4%), singleparent families (7%), with other relatives (1.5%), with roommates and/or their partners (4.8%), or alone (0.3%).

Among the study sample, few participants (0.1%) and their relatives (3.1%) were diagnosed with COVID-19. Additionally, although testing for COVID-19 was not performed, other participants also indicated that they (7.1%) or their relatives (8.2%) had COVID-19-like symptoms. In terms of the socio-economic impact of the health crisis, more than half of the sample (52.4%) reported a significant reduction in their family's income as a result of COVID-19.

Data were collected through an online survey using the Qualtrics software platform during the fifth–sixth week of the state of emergency in Spain (from 17 April to 1 May 2020). Participants were recruited through snowball sampling. The authors first distributed the online questionnaire through colleagues and potentially eligible participants who met the inclusion criteria: students between 16–25 years old, of Spanish nationality, and who

were living in Spain. Then, initial participants were asked to send the questionnaire to other potential informants that met the inclusion criteria. The survey link was also shared through social media (Facebook, Instagram, Twitter, and WhatsApp), in the newsletter of the authors' university, and with school institutions and local youth organizations. Of the 1582 students who actively consented to participate, 336 respondents were excluded from the study sample. Reasons for exclusion included the following: participants did not meet the inclusion criteria; the questionnaire was blank, or respondents only answered demographic questions; and the time spent to complete the questionnaire was less than 7 min, which is significantly faster (10th percentile) than the average (11 min).

The approval to conduct this study was obtained from the Ethics Committee of the Universidad Loyola Andalucía. Participation was voluntary and anonymity was guaranteed. Active informed consent was obtained prior to participation.

#### *2.2. Measures*

COVID-19-related concerns: An ad hoc five-item questionnaire was used to measure concerns about the impact of COVID-19 during the confinement. With the question "how much have you been worrying about ... ?" participants indicated their concerns related to: their own health (i.e., "getting ill with COVID-19"); the health of others (i.e., "my relatives getting ill with COVID-19"); their family financial strain, currently (i.e., "your family financial situation in this moment") and in the future (i.e., "your family financial situation in few months"); and their education (i.e., "this situation could negatively affect your studies"). This measure was created considering the main results of previous studies about youth concerns [12,23]. Participants reported their level of concern with each statement on a scale ranging from 1 (nothing) to 5 (a lot). Cronbach's alpha for this study was 0.73.

Daily routines during the confinement: To assess routines during the confinement, participants were asked to indicate the frequency with which they engaged in a set of five activities on a daily basis. These activities included the following: "maintaining a routine (e.g., getting up, eating ... at the same time)"; "doing physical or sports activities"; "doing intellectual activities (e.g., studying and reading)"; "doing leisure activities (e.g., playing games, watching a series, or listening to music)"; and "doing creative activities (e.g., writing and handcrafting)". A pilot study with nine adolescents was conducted to explore if this pool of items covered all the possible daily routines during the confinement. Participants agreed that no other daily routine should be included, so this measure was tested in the full sample. Items were answered on a frequency scale ranging from 1 (never) to 5 (many times).

Online peer activities during the confinement: Online activities that participants engaged in during the confinement to feel supported, loved, or cared for by their friends were measured with an ad hoc seven-item questionnaire. These activities included: "sharing personal pictures or videos of activities I do at home"; "sharing funny memes or videos"; "messaging on WhatsApp, Telegram, or others"; "making calls or video calls"; "playing online"; "doing challenges"; and "doing activities simultaneously with friends (e.g., watching series, doing homework, and playing sports)". Items were chosen based on a pilot study with nine participants who were asked about the online activities that they were doing during confinement. Items were answered on a frequency scale ranging from 1 (never) to 5 (many times). Cronbach's alpha for this study was 0.67.

Feelings of frustration: The general sense of frustration during the confinement was captured using a single item created for this study. Participants were asked "in the past two weeks, to what extent did you feel frustrated?" [35]. Response options were on a Likert scale from 1 (nothing) to 5 (very much).

Optimism: Dispositional optimism, deemed as the general expectation that good things will happen, was measured with the three-item optimism subscale of the Comprehensive Inventory of Thriving [36]. Respondents rated items (e.g., "I have a positive outlook on life") on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). Cronbach's alpha for this study was 0.85.

#### *2.3. Plan of Analysis*

First, to examine adolescents' concerns about the impact of COVID-19, their daily routines during the COVID-19 lockdown, and the online peer activities used to help youths feel supported during the confinement, descriptive statistics were computed. Besides, three different two-way multivariate analyses of variance (MANOVAs) were conducted to examine mean differences in adolescents' concerns about COVID-19, their daily routines, and online peer activities based on sex, age groups, and the interaction between sex and age. Second, multiple regression analysis was calculated to examine the association of adolescents' concerns, their daily routines, online peer activities, and optimism with adolescents' experiences of frustration. We entered independent variables in the model using the stepwise method. Variables were organized in three blocks: sex and age groups were entered in Block 1; COVID-19-related concerns, online peer activities, and optimism were entered as predictors in Block 2; and finally, the five daily routines were entered in Block 3. The stepwise method used was iterative. The order in which predictors were entered into the model was based on a statistical criterion [37]. Thus, the number of models was not dependent on the number of blocks, but rather on the number of predictors that were significantly associated with the dependent variable. This method began by introducing the independent variable of Block 1 with the highest simple correlation with the outcome. If this predictor significantly improved the percentage of variance explained by the model, it was retained and another predictor was considered. The second predictor included was the next independent variable within the block that had the largest semipartial correlation with the outcome or, in other words, the predictor that explained the largest part of the remaining variance in the model. If no other variable was identified, it moved on to the next block. The analysis concluded when no more variables from any of the three blocks could make a significant contribution to the predictive power of the model. Each time a variable was introduced, all the statistics of the model were recalculated, resulting in a new model. Collinearity was assessed by calculating tolerance and Variance Inflation Factors (VIF) for each independent variable introduced in the model. The partial eta square (*ηp*2) and the coefficient *R*<sup>2</sup> were used as measures of effect size.

#### **3. Results**

#### *3.1. Concerns about COVID-19 among Youths by Sex and Age*

As shown in Table 1, among the top concerns were that a relative could get infected with COVID-19 and that the pandemic could impact studies. In contrast, participants were least concerned about their own health.


**Table 1.** Descriptive and MANOVA statistics for adolescents' concerns by sex and age groups.

\* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* ≤ 0.001. Abbreviations: MANOVA, Multivariate Analysis of Variance; *M*, Mean; *F*, F-Snedecor; *SD*, Standard Deviation; *ηp*2, partial eta square.

> Results from MANOVA with the five concerns about COVID-19 as dependent variables and sex and age groups as independent variables revealed significant multivariate effects for both sex, Wilks' <sup>λ</sup> = 0.94; *<sup>F</sup>*(5, 1063) = 14.13, *<sup>p</sup>* ≤ 0.001, *<sup>η</sup>p*<sup>2</sup> = 0.06, and age, Wilks' λ = 0.99; *F*(5, 1063) = 2.56, *p* = 0.026, *ηp*<sup>2</sup> = 0.01. Subsequent univariate ANOVAs on

adolescents' concerns (see Table 1) indicated that girls showed significantly higher levels of concern for all issues than boys. Regarding age differences, the results also indicated that younger adolescents were significantly more worried about their studies and the health of their relatives than their older counterparts. Finally, the multivariate interaction between sex and age was not significant, Wilks' λ = 0.99; *F*(5, 1063) = 0.77, *p* > 0.05, *ηp*<sup>2</sup> = 0.00.

#### *3.2. Daily Routines of Youths during COVID-19 Confinement by Sex and Age*

Regarding routines during the confinement, Table 2 displays descriptive statistics and mean comparisons based on participants' sex and age. In general, the results indicated that the most frequent activities were intellectual (e.g., studying and reading) and leisure (e.g., playing games, watching a series, and listening to music) activities. In contrast, the least frequent were creative activities (e.g., writing and handcrafting).

**Table 2.** Descriptive and MANOVA statistics for daily routines by sex and age groups.


\* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* ≤ 0.001. Abbreviations: MANOVA, Multivariate Analysis of Variance; *M*, Mean; *F*, F-Snedecor; *SD*, Standard Deviation; *ηp*2, partial eta square.

> Next, a MANOVA, including sex and age as fixed factors and daily routines as dependent variables, yielded significant multivariate effects of sex, Wilks' λ = 0.92; *<sup>F</sup>*(5, 1060) = 17.86, *<sup>p</sup>* ≤ 0.001, *<sup>η</sup>p*<sup>2</sup> = 0.08, and age, Wilks' <sup>λ</sup> = 0.99; *<sup>F</sup>*(5, 1060) = 3.14, *<sup>p</sup>* = 0.008, *ηp*<sup>2</sup> = 0.02, on adolescents' daily routines during COVID-19 confinement. Next, univariate ANOVAs (see Table 2) indicated that girls and late adolescents were more likely to maintain daily routines during the confinement. Girls were more engaged in intellectual, creative, and sports activities on a daily basis. Similarly, younger adolescents reported doing leisure activities with greater frequency than older adolescents. No multivariate interaction effect between sex and age was found, Wilks' λ = 0.99; *F*(5, 1060) = 1.03, *p* > 0.05, partial *η*<sup>2</sup> = 0.00.

#### *3.3. Online Peer Activities Used to Help Youths Feel Supported by Friends during COVID-19 Confinement by Sex and Age*

Table 3 provides an overview of descriptive statistics for online peer activities during the confinement by adolescent sex and age. As shown, participants engaged in online activities to maintain relationships with friends quite regularly. Among the most frequent were activities aimed at maintaining communication (i.e., through messaging on WhatsApp, Telegram, or others, or making calls or video calls) and having companionship (i.e., doing leisure activities simultaneously with friends).

A MANOVA with sex and age as independent variables and online peer activities as dependent variables provided evidence of multivariate effects for both sex, Wilks' λ = 0.82; *<sup>F</sup>*(7, 1189) = 36.99, *<sup>p</sup>* ≤ 0.001, *<sup>η</sup>p*<sup>2</sup> = 0.18, and age, Wilks' <sup>λ</sup> = 0.97; *<sup>F</sup>*(7, 1189) = 5.25, *<sup>p</sup>* ≤ 0.001, *ηp*<sup>2</sup> = 0.03. As shown in Table 3, separate univariate ANOVAs on the outcome variables revealed that girls used the internet more extensively than boys to maintain relationships with their friends. While boys played online games with peers more often than girls, girls shared more pictures or videos of themselves doing activities at home, used WhatsApp or Telegram more often to message their friends, did more challenges, and got involved in more simultaneous activities with their friends using the internet. Similarly, when

comparing online activities across age groups, the results showed that the younger group of adolescents made more calls or video calls with their friends, were more engaged in challenges, and did more activities simultaneously with their peers. Finally, a multivariate interaction effect between sex and age was observed, Wilks' λ = 0.98; *F*(7, 1189) = 3.04, *p* = 0.004, *ηp*<sup>2</sup> = 0.02. Subsequent univariate analyses indicated that the interaction between sex and age was significant for the activities of sharing videos or memes, *F*(1, 1195) = 6.16, *p* = 0.013, *ηp*<sup>2</sup> = 0.01, making calls or video calls, *F*(1, 1195) = 4.50, *p* = 0.034, *ηp*<sup>2</sup> = 0.01, and playing online games, *F*(1, 1195) = 7.26, *p* = 0.007, *ηp*<sup>2</sup> = 0.01; with boys in the youngest age group doing these activities more frequently than boys in the older age group.

**Table 3.** Descriptive and MANOVA statistics for online peer activities by sex and age groups.


\* *<sup>p</sup>* < 0.05, \*\*\* *<sup>p</sup>* ≤ 0.001. Abbreviations: MANOVA, Multivariate Analysis of Variance; *<sup>M</sup>*, Mean; *<sup>F</sup>*, F-Snedecor; *SD*, Standard Deviation; *<sup>η</sup>p*2, partial eta square.

#### *3.4. Feelings of Frustration during the Confinement: The Role of Adolescents' Sex, Age, Concerns, Daily Routines, Online Peer Activities, and Optimism*

The multiple regression (see Table 4) revealed that sex, optimism, COVID-19-related concerns, online peer activities, maintaining daily routines, and leisure activities contributed significantly to the regression model, *F*(6, 1052) = 31.80, *p* ≤ 0.001. Together the six independent variables accounted for 15.4% of the variance in frustration. Being female, experiencing more concerns about the impacts of COVID-19, and doing online peer activities more frequently were positively related to frustration. In contrast, higher levels of optimism, maintaining daily routines, and doing more leisure activities were negatively associated with frustration.

Regarding collinearity, tolerance values ranged from 0.88 to 0.92 and VIF values ranged from 1.08 to 1.13. These values indicate that the variables introduced in the model were not highly correlated and there was no collinearity among the independent variables.

As a follow up, correlations between adolescents' frustration and specific aspects of COVID-19-related concerns and online peer activities were computed. In relation to COVID-19-related concerns, frustration was positively associated with concerns about their relatives getting COVID-19 (*r* = 0.08, *p* = 0.010), their family financial strain, currently (*r* = 0.11, *p* ≤ 0.001), and in the future (*r* = 0.13, *p* ≤ 0.001) as well as about their own education (*r* = 0.20, *p* ≤ 0.001). Regarding online peer activities, positive, although small correlations, were found between frustration and sharing personal pictures or videos (*r* = 0.07, *p* = 0.017), messaging friends on WhatsApp, Telegram, or others (*r* = 0.06, *p* = 0.038), and making calls or video calls (*r* = 0.08, *p* = 0.013).


**Table 4.** Results of the multiple linear regression on adolescents' frustration.

*<sup>N</sup>* = 1059; \* *<sup>p</sup>* < 0.05, \*\* *<sup>p</sup>* < 0.01, \*\*\* *<sup>p</sup>* ≤ 0.001. Abbreviations: *<sup>B</sup>*, Unstandardized beta coefficient; *<sup>t</sup>*, t-value; <sup>Δ</sup>*R*2, increment in the fraction of the variation in frustration accounted for the independent variables of each model.

#### **4. Discussion**

This paper aimed to provide insight into the concerns that young people experienced during the COVID-19 confinement in Spain as well as to explore daily and online activities. Most previous studies have analyzed the psychological impact of COVID-19 in the general or university population, but there are still few studies focused on adolescents. Besides, considering that the progress of the pandemic is still uncertain, this study has also described the degree of frustration experienced by adolescents and tried to identify factors associated with it.

The first aim of this study was to explore adolescents' concerns about COVID-19. According to the hypotheses of this study, findings evidenced that adolescents were the most worried about the risk of their relatives getting sick and the least worried about their own risk of infection. Previous studies have also shown this result [14]. From a developmental perspective, the lesser degree of concern that adolescents showed about their own health may be explained from the hypothesis of the "personal fable". Previous studies suggest that a characteristic of adolescent thinking is their propensity to regard themselves as invulnerable. That is, they think that problems and difficulties are not going to happen to them. This cognitive feature is associated with greater involvement in risky behaviors [38]. In times of pandemic, breaking the social distancing guidelines or not following health and safety measures may be considered as new risk behaviors. As such, if adolescents are less concerned about their own health, they may become more involved in irresponsible behaviors that put everyone's health at risk [39,40]. For this reason, future studies may want to analyze the effectiveness of prevention campaigns aimed at fostering social responsibility. Perhaps personalizing the consequences of adolescents' risk behaviors on the health of their relatives (e.g., grandparents) may be a more effective way to achieve greater adherence to prevention measures. Beyond health concerns, the findings of this

study also indicated that, consistent with research on university students [12], adolescents were very concerned about their studies. A review by Sahu [41] describes that some young people have had to adapt to online education with limited resources at home (e.g., teenagers have had to share computer equipment with different family members who need to telework and study). They have also faced an increase in the amount of schoolwork because of continuous evaluation in addition to uncertainty and pressure of highly monitored final evaluations. All of these factors may have been a source of stress for young people. Thus, these results were in line with the hypothesis of this study and underline that it is also crucial to take care of our education. Additionally, the results have contributed to understanding the role of sex and age on adolescents' experienced concerns. Regardless of the issue, girls were found to be more worried than boys. For age, although in general there were no differences, younger people were more concerned about the health of others and their studies. These results are in line with previous studies with young people and adults, which have reported increased levels of increased worry, fear, anxiety, and depression among girls [14,17,32]. Moreover, contrary to our expectations, the younger groups of participants were more worried than older adolescents. A previous study among Italian adolescents between 13 to 20 years old [14] has shown that older adolescents were moderately more concerned than younger adolescents. However, the study results are similar to previous studies with university and adult samples, which found higher levels of COVID-19 fear and concerns among younger participants [19,24]. Thus, the results highlight the relevance of understanding that the COVID-19 health crisis is not affecting all people equally, so it is necessary to redouble efforts to the most vulnerable.

The second aim of this study was to provide insights into the activities that young people have been participating in during the confinement on a daily basis and with their peers. Findings suggested that adolescents have spent time on intellectual and leisure activities. In contrast, creative and physical activities were undertaken less frequently. In view of these findings, it is important to implement resources aimed at reducing sedentary behaviors and sports activities should be promoted. In this regard, previous theoretical studies have warned that the possible increase in sedentary behavior together with the excessive time spent on technology may have affected the sleep patterns of children and adolescents, with consequent risk to their development and health [42]. Regarding online activities with peers, this study has also yielded interesting results. Before the pandemic, the analysis of the use of new technologies in peer relations was undertaken in the knowledge that young people tend to alternate face-to-face interactions with online interactions. However, the pandemic situation and the lack of in-person contact has provided a unique opportunity to analyze the contribution of the online world to interpersonal relationships. According to the expectations, the results of this study indicated that young people very often used new technologies to feel supported by their friends. The most frequent online activities were conversations via instant messaging applications and the use of new technologies to do activities simultaneously with peers. In contrast, challenges and online games were less frequently undertaken. Girls used the internet more often than boys, and younger people tried to see their friends more, even if it was through the screen, using challenges, or doing the same activities simultaneously with peers. All these interactions, which are related to spending time together, are traditionally known as companionship or intimacy [43]. In-person, higher levels of companionship are associated with better psychological and relational adjustment [44]. However, the role of these relationships on adolescents' wellbeing is less known when interactions are 100% online.

The third aim of this study was to examine the factors that related to adolescents' experiences of frustration during the lockdown. Although frustration itself is not an indicator of mental health, prior research has shown that it is a significant predictor of adolescents' psychological problems [28]. Findings revealed that optimism, followed by sex, COVID-19-related worries, online peer activities, daily routines, and leisure activities were significantly related to higher levels of frustration. Concerning daily routines, the results showed that keeping daily routines and doing leisure activities were related to

lower levels of frustration. Possibly, routines during the confinement have been important to foster a sense of normality, providing structure in the uncertain. In further detail, subsequent analyses indicated that adolescents' more salient concerns about COVID-19 were associated with higher levels of frustration. Specifically, the findings showed that youths who were more worried about their education, the financial situation of their family, or the health of their relatives also reported more frustration. Another important finding concerns the online activities used to keep youths connected with their peers during the confinement. Although correlations were modest, results indicate that youths who spent more time sharing personal pictures or videos, messaging friends, and making calls or video calls experienced greater levels of frustration. This finding might seem counterintuitive because previous literature has shown that intimacy and companionship with peers are related to better psychological adjustment [44]. However, our results may be far from what is expected because the interaction with peers, due to the confinement, has been restricted to the online context. A previous study found that the balance between online and offline communication matters [45]. For example, among university student couples, closeness increased when online and offline communication was balanced (i.e., couples communicated by both means). In contrast, intimacy and closeness decreased when communication was only online. Thus, these results suggested that the online context was useful for socialization, especially when face-to-face interactions were restricted, but that the internet alone could not compensate or replace face-to-face interaction. Finally, it is important to note that higher levels of optimism were significantly related to lower rates of frustration. This result is consistent with research on the stress buffering effect of optimism [46]. Literature on the psychological impact of COVID-19 has also evidenced that optimism mediates the relationship between COVID-19-related stress and psychological problems, and is associated with lower levels of depression or anxiety, among others [47,48]. Perhaps adolescents with a more optimistic perspective reappraised the situation in a more positive way [49]. Thus, although further studies with adolescents are needed, the results seem to indicate that keeping an optimistic perspective is important in mitigating the psychological impacts of COVID-19.

#### *Limitations*

Despite the strengths of this study, some limitations should be outlined. First, the sample was mostly composed of young people from the southern area of Spain. Although the aim of the study was not to make comparisons between different areas of the country, future studies could benefit from a stratified sampling. Second, adolescents' frustration was assessed through a single-item questionnaire. The main reason was that online surveys should be short to facilitate participation. However, future studies could evaluate this construct through a validated multi-item instrument. Third, the cross-sectional design of this study did not allow for an analysis of the directionality of the relationships between the variables. Future longitudinal studies could explore the temporal association between the factors associated with adolescents' frustration. Finally, given that data were collected during the strict confinement period in Spain, this study assumed that participants were not having any actual face-to-face contact with their peers and that all their social routines were online. However, as they were not explicitly asked about the extent to which they followed the strict isolation measures imposed, it remains possible that a few individuals in the study had had actual contact with their friends.

#### **5. Conclusions**

The pandemic has led to moderate-to-high levels of frustration among adolescents. The results of this study have allowed us to delve deeper into the role that concerns, daily activities, online interactions with friends, and optimism play in the degree of frustration that young people have experienced. First, matters related to family health, family financial situation, and education were found to be associated with more frustration. Among these, the academic concern was the most relevant variable because of the magnitude

of the association with adolescents' frustration. In this sense, it is necessary to develop an educational response that involves the different stakeholders (politicians, teachers, families, and students) to ensure that we take care of both students' physical and mental health. To this end, it is necessary to rethink the changes that should be implemented in order to prepare schools for a new COVID-19 outbreak. Second, findings have allowed us to make conclusions about the importance of adolescents maintaining a daily routine, and participating in physical, leisure, and creative activities. Special attention should be paid to creative activities because, despite being negatively related to frustration, they were the least frequent activity undertaken during confinement. Third, another relevant conclusion of this study was that optimism turned out to be the variable that showed a stronger negative association with adolescents' frustration. Thus, it seems that being able to reinterpret the situation positively represents a precious resource for adolescents in facing this health crisis. In summary, this work contributes to understanding the emotional impact that the COVID-19 crisis has on Spanish adolescents, exploring not only the factors that are related to more significant psychological effects but also to some variables that are associated with greater resilience.

**Author Contributions:** Conceptualization, N.M.-F. and A.R.-M.; Methodology, N.M.-F. and A.R.-M.; Formal analysis, N.M.-F. and A.R.-M.; Investigation, N.M.-F. and A.R.-M.; Resources, N.M.-F. and A.R.-M.; Data curation, N.M.-F. and A.R.-M.; Writing—original draft preparation, N.M.-F. and A.R.- M.; Writing—review and editing, N.M.-F. and A.R.-M.; Project Administration, N.M.-F. and A.R.-M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of Universidad Loyola Andalucía (16 April 2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Stress, Anxiety and Depression Prevalence among Greek University Students during COVID-19 Pandemic: A Two-Year Survey**

**Dimitrios Kavvadas 1,2,\* , Asimoula Kavvada 1,3, Sofia Karachrysafi 1,2 , Vasileios Papaliagkas <sup>3</sup> , Stavros Cheristanidis 1,4, Maria Chatzidimitriou <sup>3</sup> and Theodora Papamitsou 1,2**


**Abstract:** Background: The negative effect of COVID-19 pandemic on college students' mental health is well-demonstrated. The aim of this study is to assess the impact of the pandemic on the students of Aristotle University of Thessaloniki (Northern Greece), in terms of stress, anxiety, and depression, and to analyze the probable correlation of various social and phycological factors. Methods: The survey was conducted in the form of a questionnaire, which was first distributed in November 2020 and then re-launched in November 2021. The evaluation was carried out through the DASS21 screening tool. Associations regarding participants' characteristics and the three variables (stress, anxiety, and depression) were investigated with Pearson's chi-squared (X2) test. Results: The first-year results (November 2020) revealed severe prevalence of stress, anxiety, and depression (37.4%, 27.2% and 47% respectively). The second-year results (November 2021) revealed a significant augmentation in all three variables, mainly for the extreme severe scales (47.3%, 41.1% and 55% respectively). Participants who were receiving psychiatric treatment exhibited higher levels of stress, anxiety, and depression, especially during the second year of the pandemic (*p*-Value < 0.00001). Female students' mental health was at higher risk, as elevated prevalence of negative symptoms was observed (*p*-Value < 0.00001). Conclusions: The community of Aristotle University of Thessaloniki has been greatly affected during the last 2 years. The inherent risks of the confinement measures on students' well-being and mental health are undeniable. Recurrent annual psychological evaluation in universities and colleges is strongly advised.

**Keywords:** mental health; college; students; DASS21; pandemic; gender

#### **1. Introduction**

An unknown etiology outbreak of pneumonia turned into the pandemic of COVID-19 (Coronavirus disease-19) disease and spread rapidly throughout the planet [1,2]. The mortality was unexpectedly high, leading countries to take drastic measures [3].

Several studies focused on the correlation among demographic characteristics and psychological deterioration during the pandemic [4–6]. Remarkable efforts have been made on the evaluation of psychological and mental effects on university students. Evidence suggest an increase in stress, anxiety, and depression levels in several countries. In China, according to Ma et al., the probability of developing acute stress, depression, and anxiety disorder was 34.9%, 21.1%, and 11.0% respectively, based on a sample of 746,217 students

**Citation:** Kavvadas, D.; Kavvada, A.; Karachrysafi, S.; Papaliagkas, V.; Cheristanidis, S.; Chatzidimitriou, M.; Papamitsou, T. Stress, Anxiety and Depression Prevalence among Greek University Students during COVID-19 Pandemic: A Two-Year Survey. *J. Clin. Med.* **2022**, *11*, 4263. https://doi.org/10.3390/ jcm11154263

Academic Editor: Alfonso Troisi

Received: 18 June 2022 Accepted: 20 July 2022 Published: 22 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

during November 2020. In the United States, in a similar study conducted during 2020, 195 students were evaluated and 71% of the sample revealed increased stress and anxiety during the pandemic, in comparison to the pre-COVID period [7–9]. Post-traumatic stress disorder, accompanied by sleep disorders, has also been observed. These findings confirm that intense and constant concern can lead to a severe reduction of the academic performance and to psychological health distress. For instance, according to Son et al., over 90% of the sample of university students expressed concerns for their relatives' wellbeing and 89% had difficulties to concentrate. Almost 86% reported sleep disorders and deterioration of their social life. Moreover, 82% observed decreased performance in their academic courses. Similar reports were recorded by Tang et al., during the first year of the pandemic, on 2485 students from six Chinese universities [7,10]. An aggravating factor in the above is the loss of their jobs or the alteration to their employment routine [11]. Several studies suggest that younger age, poor health, extensive exposure to computer screens, and the constant fear for SARS-CoV-2 infection, pose significant risk factors for the students' mental health during the pandemic. There is a worldwide indication that female gender should be included in the risk factors. Chang et al. observed that female students were more likely to suffer from anxiety and depression during the COVID-19 period [9]. Browning et al. collected data from seven United States Universities during the first months of the pandemic. Their findings suggest that female gender is hampered by the already deteriorating mental health of American students during the pandemic [12]. Similar observations were made in a survey conducted in France, including 69,054 students during the first lockdown period of 2020 [13]. Other risk factors are financial hardship, distance learning, and uncertainty about the academic future due to the ongoing preventive measures and quarantine [14].

A comparative study revealed severe stress management imbalance during the pandemic, compared to 2018 [15]. Similarly to other studies, it was noticed that female students and students with disabilities and different sexual orientation respondents faced significant difficulties and their mental health was aggravated during the pandemic [15,16].

Studies were also carried out in Greece during the first year of the pandemic, presenting similar results. Kaparounaki et al. conducted a study on student mental health during the first months of the pandemic in Greece (April 2020). Among the 1000 participating students, increased stress and depression levels (42.5% and 74.3% respectively) were observed [17]. Patsali et al. collected data from April to May 2020. A total of 1104 female Greek students and 431 male Greek students participated in the study. The results showed that more than 65% of the respondents reported a significant increase in stress due to quarantine. Severe depression and feelings of despair were also reported. Female gender appeared as a potential risk factor of depression [18]. Moreover, a study on 1060 students (mostly females) from several Greek universities during the first months of the pandemic [19] was performed through the Depression Anxiety Stress Scale (DASS-21). Moderate to extreme severe depression was observed in 35.6% of the students, while 31.8% and 19.7% reported moderate to extreme severe stress and anxiety, respectively. The prevalence of negative emotions was particularly high compared to similar rates on Chinese students, but similar to Mediterranean countries like Italy and Spain [19].

The research hypothesis expected increased levels of stress, anxiety, and depression in the second distribution of the survey. In addition, it was assumed that demographic and individual characteristics of the participants (such as gender, cohabitation status, vaccination status, psychological or psychiatric treatment) were significantly correlated with the different levels of stress, anxiety, and depression. The hypothesis was derived from recent global data with evidence of continuous deterioration of university students' mental health during the pandemic [7–10,14].

#### **2. Materials and Methods**

#### *2.1. Study Design*

The study was conducted through an anonymous questionnaire, which assessed the psychological, social data of the participants and included the Depression Anxiety Stress Scale (DASS21). The survey was launched twice. The questionnaire was the same for both distributions.

The aim of this research was to assess the general mental health of the Aristotle University of Thessaloniki (AUTh) students during COVID-19 "peak" periods, in Greece. For this purpose, DASS21 screening test was used in order to evaluate stress, anxiety, and depression levels. Furthermore, the impact of physical, social, and phycological factors and their probable correlation to the students' current mental state were analyzed. According to the literature, a further gender analysis was considered to be essential.

#### *2.2. Population and Samples*

The study sample included students (BSc, MSc, PhD) of the university. The research was conducted at two time intervals. The first period of the survey took place in November 2020 (first major peak of the pandemic) and included 2322 participants, while the second was in November 2021 (second peak) with 3160 participations. Based on the official registration of the AUTh, as of November 2021 there were 51,577 active students [20]. Therefore, the collected students' samples represent approximately 5–6% of the students' population (2110 and 2916 for the first and the second year respectively) (Figure 1). The rest participants were excluded as they represented the academic and administrative staff of the university. It should be noted that more than 1500 respondents' entries were also excluded from the surveys due to incomplete completion of the questionnaire (Figure 1). The percentage of female active students was approximately 54% and of the male active students was 46%.

**Figure 1.** Participants' flow chart with inclusion and exclusion criteria.

#### *2.3. Questionnaire Platform and Approvals*

The hosting platform was the LimeSurvey AUTH (Aristotle University of Thessaloniki) under the supervision of the certified questionnaires authority of Aristotle University). Due to the legislation on personal data protection (GDPR), both the AUTh bioethics committee

(Bioethics Committee No. 1.254/20-10-20) as well as the AUTh Data Protection office, granted permission. The LimeSurvey AUTH platform gathered the responses and access was granted only to the head professor of the project (author T.P.) via a personalized link, which created a secure authorized profile.

#### *2.4. Distribution and Content of the Questionnaire*

The questionnaire was available from 15 November 2020 (08.00 A.M. Greece time) to 30 November 2020 (08.00 A.M.) (first launch) and from 15 November 2021 (08.00 A.M.) to 30 November 2021 (08.00 A.M.) (second launch). The survey was conducted in the form of an online questionnaire, which was distributed to participants via their institutional e-mail ("name"@auth.gr). All of the questions were closed-type questions and it was obligatory for participants to choose an answer, once they agreed to participate, in order to complete the survey.

The questionnaire was divided into three thematic sets (Table 1):

**Table 1.** The questionnaire used in both years' surveys.


(a): evaluation of anxiety, (s): evaluation of stress, (d): evaluation of depression.

The first set was about the basic information and participant's profile, psychological evaluation, and experiences in relation to COVID-19 [16]. The aim was to extract information related to the basic characteristics of the respondents (questions 1–6) and their

familiarization with COVID-19 (questions 7–10). Moreover, there was a short evaluation of their psychological state (questions 12–14) and their social well-being (questions 16–18).

The second section was about the university status (questions 19–21)

Both sets 1 and 2 were composed of well-established questions, similar to other published studies [16].

The third and most significant set of questions (questions 22–42) assessed the levels of anxiety, stress, and depression, as well as their physical manifestations, through the DASS21 [16,19].

The DASS21 (Depression, Anxiety, and Stress Scale) was introduced in 1995 by Lovibond and Lovibond [21]. It consists of three self-report scales designed for screening of depression, anxiety, and stress [16,19]. In 1998, a final version of the DASS that consisted of 21-item (DASS21) was described [22]. Each of the three DASS21 scales contain seven elements, divided into subscales with similar content. The Depression Scale assesses discomfort, despair, life devaluation, self-devaluation, lack of interest/engagement, and inaction. The stress scale assesses autonomic arousal, signs of stress through skeletal muscle movements, stress-induced anxiety, and the subjective experience of anxiety. The stress scale is sensitive to chronic non-specific stimulation. This scale evaluates the difficulty of relaxation, nervous agitation and upset/agitation, the case of an irritable/hyper-reactive characters, and impatience. Scores for depression, anxiety, and stress are calculated by summing the scores for the relevant data [21,22]. The DASS21 rating scale is used internationally to assess stress, anxiety, and depression levels. It is a recognized and accepted tool by psychologists and psychiatrists with a very good internal consistency [22]. It is, therefore, a valid Likert-4 scale (0. Not at all, 1. A little, 2. much, 3. Too much), which calculates the negative emotional states experienced by the participants during the period of time that the survey is available. The Greek version of the DASS-21 scale, based on Greek sources and official translations, was described by Lyrakos et al. [23]. This particular version was used in this survey. The results can be either normal, mild, moderate, severe, or extremely severe. For stress, a normal score is 0–7, 8–9 for mild stress, 10–12 for moderate, 13–16 for severe, and above 17 for extreme severe stress [21]. Similarly, 0–3 is the normal score for anxiety, 4–5 is the prevalence of the mild anxiety, 6–7 moderate, 8–9 severe and above 10 is the extreme severe anxiety [21]. Finally, a score of 0–4 is normal for depression, 5–6 is mild, 7–10 is moderate, 11–13 is severe, and above 14 is the score for extreme severe depression [21]. The scores for depression, anxiety, and stress are calculated by summing the scores for the relevant items [21].

The above questionnaire was distributed for the second time on November 2021. The second launch included the same questions with two extra questions (vaccine coverage and concerns for an impending lockdown).

During the 2-year study, all personal data protection measures were obtained for the participants, who were informed that they had the right to terminate their participation at any time. They were also informed about the purpose of the research, the population target of the university, and the DASS21 evaluation scale. It was stated to them that any processing of personal data would be done in accordance with the General Regulation of Personal Data Protection, taking the appropriate technical and organizational measures. Personal data were kept only for the period required for the lawful purposes for which they were collected ensuring their safe destruction, when the legally abovementioned period had elapsed or the purpose of their processing ceased to exist. Finally, they were informed that for the purposes of the investigation it was not required to verify their identity by those responsible for processing the data, with the result that the latter were not obliged to obtain or retain or process additional information to verify the identity of each participant. Consequently, the following rights did not exist: (a) the right of access to personal data, (b) the right of correction, (c) the right of deletion, (d) the right of restriction of processing, and (e) the right of data portability in accordance with the General Regulation Personal Data Protection. Contact details were provided for anyone seeking more information and clarifications.

#### *2.5. Statistical Analysis*

The Cronbach "alpha" factor was excellent in both years. More specifically, it was estimated at 0.946 for the DASS21 launched in 2020 and 0.954 for the DASS21 launched in 2021. As mentioned, the DASS-21 is based on a multi-dimensional and not a categorical perception of psychological distress [21,22]. The hypothesis on which the development of DASS21 was based (and which was confirmed by research data) is that the differences between depression, anxiety, and stress experienced by normal individuals and clinical populations, are gradually different [21]. The demographic characteristics of the participants and the answers to the first set of questions (psychological assessment, COVID-19, academic capacity) were studied with Pearson's chi-square test. Because of the large set of categorical variables, the multiple correspondence (correlations) analysis was performed for the second questionnaire (November, 2021) [24–28]. The aim was an in-depth psychological evaluation of the students after 2 consecutive years of the ongoing pandemic, during the second and largest peak of COVID-19 outbreak in Greece. The results of the 21 questions through which the score of anxiety, stress, and depression was obtained, are summarized within the multiple correspondence analysis in in three grades (Normal, Mild to Sever, Extreme Severe). The gradation was delineated based on the quadrants Q1 and Q3 of a continuous distribution of samples with a value range of 0–21, which consist of the sum of the DASS21 scores [29]. The multiple correspondence processing leads to the construction of the Burt tables, which are multiple coincidence tables. These tables were produced by the intersection of the classes of each variable [30]. The purpose of the multiple correspondence analysis is the calculation of the coordinates of the rows and columns on the factorial axes that are formed during the analysis of the data, in order to interpret the extracted information [24–28]. In this case the columns represented the levels of stress, anxiety, and depression, while the rows were the responses to the basic information received by the first part of the survey (demographic, psychological etc.) The statistical processing of the results was performed with the program SPSS version 24.0 (IBM, SPSS Inc., Chicago, IL, USA), Microsoft Excel (2019) version 16.43, and the the Méthodes d 'Analyses des Données (MAD) software [31].

#### **3. Results**

#### *3.1. Two-Year Demographic Data*

Female participants outnumbered males in both years (70% and 73%, first and second launch respectively). The cohabitation status changed in the second year, with a higher percentage stating to dwell in their houses without roommates (31%) or with one person (25%) (chi-square 281.2, *p*-value < 0.00001), compared to 2020 (15% and 21% respectively). Significant difference was also observed in the work status, with more employees reporting changes in their job routine during the first year (27.5%) (chi-square 23.8, *p*-value < 0.00001). During the second launch (November 2021), the majority of the participants reported to know an acquaintance who was diagnosed with COVID-19 (94.4%). This rate was significantly increased in comparison to the first launch (chi-square 81.5, *p*-value < 0.00001). Meanwhile, the cases of acquaintances who were diagnosed positive and were seriously ill or died were also significantly increased in 2021 (21%) (chi-square 99.0, *p*-value < 0.00001). Psychiatric care (chi-square 78.3, *p*-value < 0.00001) and psychotropic drugs intake (chisquare 14.3, *p*-value 0.00015) were also increased during the second year of the pandemic (14.5% and 3.9% respectively). Also, compared to the first year, the respondents felt that the relationship between people confined in the same house was significantly deteriorated (50%) (chi-square 12.4, *p*-value 0.00042).

The majority of students who had not been vaccinated by the time of the second launch were undergraduate and master students (25% and 19% respectively).

A statistically significant difference was observed regarding gender and the fear of an impending lockdown, due to the raised numbers of COVID-19 cases (*p*-value < 0.00001). The majority of females (60%) were more concerned compared to male participants (50%) (Supplementary Material; Tables S1–S14).

#### *3.2. DASS-21 Results*

The results of the DASS21 surveys are presented below (Table 2). In 2020, moderate to severe stress and extreme anxiety was observed in high rates. In the same year, 60% of the depression prevalence rates in students were distributed to the scales from mild (13%) to extremely severe (16.1%). In 2021, a significant increase was observed in all extreme severe scales (Table 2).

**Table 2.** DASS21 results of the Aristotle University of Thessaloniki students during the 2 years of the pandemic (2020–2021). The comparisons were performed with the chi-square test (*p*-Value significant at 0.05).


In the second launch of the survey (November 2021), the extreme scale values for both female and male students were almost doubled (Tables 3 and 4).

**Table 3.** Chi-square statistical analysis between female students during the 2 years, based on the scores of the DASS21 scale (*p*-Value significant at 0.05).


**Table 4.** Chi-square statistical analysis between male students during the 2 years, based on the scores of the DASS21 scale (*p*-Value significant at 0.05).


A further study was performed on those who knew someone diagnosed with COVID-19 during the 2 years. The percentage of those who were familiar to someone who died of COVID-19 during the second year of the pandemic was doubled, in comparison to the first year (*p*-Value 0.0002).

In order for an in-depth analysis to be performed, the multiple correspondence analysis was implemented for the second-year responses (November 2021). DASS21 scores of our three variables stress, anxiety, and depression, were distributed in three grades: normal, mild to severe, and extremely severe (Table 5). From a total of 2916 students, 13 participants were removed. A total of 2903 students participated in the multiple correspondence analysis.


**Table 5.** The percentages of the three distribution grades for all participants.

#### *3.3. Multiple Correlations and Statistical Analyses*

The multiple statistical analysis was assessed on the prevalence of stress, anxiety, and depression (from November 2021), in association with the participants' demographic characteristics. Therefore, the Burt Tables were produced, alongside with the low-dimensional Euclidean spaces from which derived all the correlations (Table 6).

**Table 6.** The Burt table based on the variables and students' responses and statistical analysis. The variables are presented in odds ratios (ORs).



*p*-Value significant at 0.05.

As shown (Table 6), there were significant differences on the anxiety, stress, and depression levels and several sociodemographic and psychological features of the students. Younger students, female, and unmarried participants presented significantly increased levels of our three variables, in comparison to older students, males, and married respondents (*p*-Value < 0.00001). Similarly, all three variables were elevated for students who claimed to have received psychological or psychiatric treatment and those who were receiving psychotropic drugs during the surveys' launch periods (*p*-Value < 0.00001). Vaccinated students' responses revealed an augmentation in anxiety levels, in comparison to the unvaccinated respondents (*p*-Value = 0.0093). Also, concerns for an impending lockdown affected the levels of stress, anxiety, and depression (*p*-Value < 0.00001) (Table 7).


**Table 7.** The severe extreme stress anxiety depression odds ratios (ORs).

The same differences were also revealed on the gender-based analysis of our three variables regarding the psychological or psychiatric treatment, the psychotropic drugs intake, and the concerns on an impending lockdown (Tables 8 and 9). Female vaccinated students were significantly more anxious in comparison to the non-vaccinated (*p*-Value = 0.0079). Also, male students who lived alone presented higher anxiety levels in comparison to those who lived with one or more people (*p*-Value = 0.039).



**Table 9.**

Gender-based

 Burt table of the female students and statistical

 at

 The variables

 are

 in odds ratios

analysis(*p*-Valuessignificant0.05).presented(ORs).**Burt Table/ Odds Ratios Female Students Stress Anxiety Depression Normal Mild to Severe Extreme Severe Normal Mild to Severe Extreme Severe Normal Mild to Severe Extreme Severe** Marital status Unmarried 0.21 1.13 0.42 0.63 0.54 0.36 0.45 0.82 0.31 Other 0.54 1.00 0.18 1.52 0.34 0.17 1.10 0.48 0.18 *p*-Values <0.00001 <0.0001 <0.00001 Cohabitation status I live alone 0.21 1.17 0.41 0.61 0.53 0.38 0.44 0.85 0.31 With 1 person 0.24 1.11 0.39 0.69 0.52 0.33 0.49 0.75 0.32 With 2 or more 0.24 1.08 0.40 0.69 0.53 0.32 0.51 0.77 0.29 *p*-Values 0.83 0.60 0.68 Vaccinated Yes 0.22 1.11 0.41 0.63 0.52 0.37 0.47 0.79 0.31 No 0.26 1.15 0.35 0.81 0.55 0.24 0.53 0.81 0.26 *p*-Values 0.41 0.0079 0.34 Concerns about an impending lockdown None 0.45 0.84 0.31 1.04 0.45 0.22 0.73 0.65 0.23 Little 0.29 1.36 0.24 0.91 0.51 0.23 0.63 0.75 0.22 Much 0.19 1.19 0.41 0.60 0.57 0.36 0.44 0.87 0.30 Very Much 0.15 0.91 0.65 0.43 0.52 0.55 0.32 0.79 0.46 *p*-Values <0.00001 <0.00001 <0.00001 Psychological or psychiatric treatment in the past Yes 0.13 1.07 0.58 0.42 0.55 0.54 0.30 0.81 0.48 No 0.27 1.13 0.34 0.77 0.52 0.29 0.56 0.79 0.25 *p*-Values <0.00001 < 0.00001 <0.00001 Currently taking psychotropic drugs Yes 0.10 0.68 1.03 0.23 0.55 0.84 0.22 0.68 0.72 No 0.24 1.14 0.38 0.69 0.53 0.33 0.49 0.80 0.29 *p*-Values 0.00003 0.00002 0.00013

#### **4. Discussion**

This study assessed the stress, anxiety, and depression levels of the Aristotle University of Thessaloniki community, through the DASS21, during the 2-year ongoing pandemic. Severe and extremely severe prevalence was revealed in alarmingly high rates. The deterioration of students' mental health was conspicuous in both years. These findings were similar to initial research projects carried out in other Greek universities [17,32,33]. Our study revealed a significant upward pace in students' stress, anxiety, and depression levels during the 2-year evaluation.

Setting a pre-pandemic background, studies before the COVID-19 outbreak in Greece revealed a mild depression prevalence, but the results were in most cases conflicting [34–40]. A survey launched in 2015 reported mild to normal depression, related to marital status and previous psychiatric evaluation [34]. During the years 2009–2011, a study assessed the changes in mental health in the general population of Greece, due to the beginning of the economic crisis. Depression levels were increased, while stress remained stable [35]. Moreover, the fact that students, particularly from Greece, are prone to alcohol consumption during their first years of college, could lead to psychological imbalance and negative feelings [36,37]. On the contrary, another pre-pandemic study assessed the effects of the Mediterranean diet on Greek university students and found reduced levels of depression and stress, due to the consumption of specific local products [38]. Regarding mental illness, a pre-pandemic large study in the AUTh found that students claim to be familiar with mental illness, but through unreliable sources [39]. This finding reinforced fear of stigma, especially during the COVID-19 pandemic. Consequently, their imbalanced psychology, the fear of stigma, and further burden due to the daily changes of routine, posed severe danger to the mental wellness of AUTh students [40].

In the current research, approximately 22–26% of the students were integrated to the severe and extreme severe scales of the three study variables (stress, anxiety, and depression). The present 2-year analysis of the AUTh community revealed similarities with the international studies. According to Carr et al., a large percentage of a UK university community was on the verge of depression and severe anxiety disorder, with analogous proportions (30% of the students) [41]. Van Niekerk et al., launched a resemblant extensive survey during the two quarantine periods (2020 and 2021) in an Eastern Cape university and suggested that the risk of mental health deterioration should not be underestimated [42]. In the United Arab Emirates, half of the participants were at psychological distress, with those suffering of mental illness being at highest risk [43]. In China, an early COVID-19 study revealed increased levels of fear, which were associated with the prevalence of depression in students who were close to graduation [9]. Symptoms of depression and anxiety were also found in students during lockdown periods in China, 1 year after the onset of the pandemic [44,45]. Similarly, in Malaysia, a significant prevalence of anxiety in students was observed during the first year of the pandemic [14]. However, their extreme scores were significantly lower than the ones found in the AUTh students. A study conducted in seven united states of America revealed that students with low quality of life and health, of low income, and of young age were at psychological distress due to the pandemic [12]. However, the case of Sweden was different [46]. The DASS21 survey was launched in Swedish students during the first 3 months of the pandemic and no significant increase in stress, anxiety, and depression levels was revealed. On the contrary, Swedish students' mental state was improved, especially during the summer months of the first year of the pandemic [46].

Furthermore, the increased levels of stress and anxiety were correlated with female gender, between the AUTh students. More than the half of the AUth female participants suffered from mild to severe stress. The numbers of female participants that scored on the extreme stress, anxiety, and depression scales was high (above the expected scores on both launches) (Table 3). The male sample was also affected by the pandemic but not at such extreme scales. Stress and depression were above the expected values for male students during the second launch (Table 4). The higher risk of psychological distress in females was confirmed by our findings, in accordance to other studies [12,13,33]. The present study confirmed that the ongoing pandemic has significantly affected the female population of the AUTh, especially after 2 years of consecutive measures and restrictions.

Several students declared to have received psychological or psychiatric care in the past. There was a significant correlation of the increased psychiatric treatment and the elevated stress, anxiety, or depression levels of the AUTh community. These findings were also in line with evidence from pre-pandemic studies [34]. A balanced social environment and resilience of character are essential for preserving the mental well-being during the home-confinement periods [47].

The Aristotle University of Thessaloniki has established a 24-h-communication line for members who seek counseling and psychological support. Records so far were in alignment with the evidence presented in this study. The cases of the AUTh community members who seek consultation and psychological support has quintupled. Two hundred and fifty members of the university community are currently supported by two psychologists (support center of AUTh) [48]. Therefore, re-evaluation through similar studies by the end of the next years is vital.

#### *Strengths and Limitations*

In the present study, the DASS21 was used independently of any other psychometric scales. This way, the researchers tried to increase the chances of higher participation rates, by constructing a short and coherent questionnaire that could independently screen for stress, anxiety, and depression our sample. Limitations were the lack of evaluation on routine habits (e.g., alcohol consumption, food, etc.), the lack of detailed psychiatric background evaluation of the students, and the lack of specialized questions on the respondents' physical and mental health. The authors made an effort to avoid long questionnaires in order to minimize the risk of losing participations. Regarding the female students' significantly higher participation in the survey, it should be noted that the AUTh reports a female population of 54%. The responses here were about 70%, which is much higher. Even though expected based on the literature, the higher female participation should be considered in the interpretation of our findings.

The strengths of this study are the repetition of the analysis, the multiple correlations analysis, and the number of participants.

#### **5. Conclusions**

The community of Aristotle University of Thessaloniki has been greatly affected by COVID-19, with students presenting high levels of stress, anxiety, and depression during the pandemic. The deterioration of the mental health of AUTh students was in line with international and Greek research data. The analysis of the demographic and social variables indicated a statistically significant correlation between elevated stress, anxiety, and depression levels and gender, age, and psychiatric treatment. Vaccination against COVID-19 was not associated with any significant difference on stress, anxiety, and depression levels. The majority of the female participants were worried about the future and concerned for an impending quarantine confinement. The continuous implementation of restrictive measures poses significant risks to the mental health of students. It is necessary to continue the evaluation in universities and colleges every year, even after the de-escalation of the pandemic.

**Supplementary Materials:** The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jcm11154263/s1, Table S1: Demographic characteristics of the participants during the first and second year of completing the questionnaire. Table S2: Questions about COVID-19 during the 2 years, Table S3: Participants' mental health characteristics and social burden due to the pandemic, Table S4: University status of participants, Table S5: Vaccination against COVID-19 infection (second-year participants only), Table S6: Concern about impending lockdown (second year participants), Table S7: Study of gender correlation and basic questions during the first year of completing the questionnaire (2020), Table S8: Gender-based student scores during the

first year of the survey (2020), Table S9: Study of gender correlation and basic questions during the second year of completing the questionnaire (2021), Table S10: Gender-based student scores during the second year of the survey (2021), Table S11: Do you know anyone who has been diagnosed with COVID-19? (2020), Table S12: Do you know anyone who has been diagnosed with COVID-19? (2021), Table S13: Correlation of the question "Do you know anyone who has been diagnosed with COVID-19 infection?" with the DASS21 score during the 1st year of the survey, Table S14. Correlation of the question "Do you know anyone who has been diagnosed with COVID-19 infection?" with the score DASS21 during the second year of the research.

**Author Contributions:** Conceptualization, D.K. and S.C.; methodology, D.K., T.P., S.C. and S.K.; validation, A.K., S.K. and V.P.; formal analysis, D.K. and A.K.; investigation, A.K. and D.K.; resources, D.K. and T.P.; data curation, D.K.; writing—original draft preparation, D.K. and A.K.; writing—review and editing, A.K., V.P. and M.C. visualization, D.K. and A.K.; supervision, V.P., M.C. and T.P.; project administration, D.K.; funding acquisition, T.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Bioethics Committee of Aristotle University of Thessaloniki (1.254/20-10-20).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** We would like to acknowledge the valuable contribution of Dimitrios Karapistolis for the consultation on the multiple correlations' analysis and the permission to use his own developed statistical software "MAD". We would also like to acknowledge the valuable contribution of Konstantinos Stavrou (BSc English Literature) for the language revisions and Georgios Intzes for handling the data codification processes and assisting with the methodology.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Psychosocial Distress among Family Members of COVID-19 Patients Admitted to Hospital and Isolation Facilities in the Philippines: A Prospective Cohort Study**

**Leilanie Apostol-Nicodemus 1,\*,†, Ian Kim B. Tabios 2,3,\*,† , Anna Guia O. Limpoco 1, Gabriele Dominique P. Domingo <sup>4</sup> and Ourlad Alzeus G. Tantengco <sup>3</sup>**


**Abstract:** This study determined the psychosocial impact of COVID-19 on families of adult COVID-19 patients in isolation facilities in Metro Manila, Philippines. This prospective cohort study was conducted in COVID-19 healthcare facilities. Data collection was undertaken 2 weeks and 8 weeks after discharge. Logistic regression was performed to determine the socioeconomic and clinical factors influencing anxiety, depression, and family function. Based on HADS-P, 43.2% of the participants had anxiety symptoms, and 16.2% had depression symptoms 2 weeks after the discharge of their relative with COVID-19 infection. The prevalence of anxiety and depression significantly decreased to 24.3% and 5.4%, respectively, 8 weeks after discharge. The percentage of participants with a perceived moderate family dysfunction was 9.5% in the 2nd week and 6.8% in the 8th week post discharge. Participants with perceived severe family dysfunction increased from none to 4.1%. The most inadequate family resources for the participants were economic, medical, and educational resources. Patient anxiety (*p* = 0.010) and perceived inadequate family resources (*p* = 0.032) were associated with anxiety symptoms among family members. Patient anxiety (*p* = 0.013) and low educational attainment (*p* = 0.002) were associated with anxiety symptoms among family members 8 weeks after discharge. On the other hand, patient depression (*p* = 0.013) was a factor related to depressive symptoms among family members 2 weeks after discharge. This study provided an in-depth understanding of the mental health status of family members caring for relatives with COVID-19 infection. This can be used to guide healthcare professionals caring for COVID-19 patients and their family members.

**Keywords:** anxiety; coronavirus; depression; family; mental health; pandemic

#### **1. Introduction**

COVID-19 is a global pandemic caused by SARS-CoV-2. The Philippines is one of the countries in Southeast Asia that suffered greatly from COVID-19. There have been surges in cases due to the new variants and changes in the health response. Throughout the pandemic, containment and mitigation measures such as physical distancing, home quarantine, and self-isolation remain at the forefront of the country's response [1,2]. Families have an essential role in implementing these measures to control COVID-19.

Families are the basic social unit of society [3,4]. Filipinos are known for their closeknit extended family structures. Family members are the first line of support during times of sickness [5]. During disease outbreaks, families experience emotional upheaval due to anxiety and fear of the possibility of contracting the disease [6–8]. This emotional distress

**Citation:** Apostol-Nicodemus, L.; Tabios, I.K.B.; Limpoco, A.G.O.; Domingo, G.D.P.; Tantengco, O.A.G. Psychosocial Distress among Family Members of COVID-19 Patients Admitted to Hospital and Isolation Facilities in the Philippines: A Prospective Cohort Study. *J. Clin. Med.* **2022**, *11*, 5236. https://doi.org/ 10.3390/jcm11175236

Academic Editor: Alfonso Troisi

Received: 8 July 2022 Accepted: 2 September 2022 Published: 5 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

is further exacerbated by strict infection control measures that inadvertently promote stigmatization, social isolation, and economic problems stemming from losing income due to the lack of job opportunities [5,9].

The COVID-19 pandemic has caused radical changes to the average Filipino's life, such as loss of income and decreased social interaction due to the mitigating interventions implemented by the government [10,11]. Further psychosocial distress is likely to impact those directly affected by COVID-19 by either contracting the disease or having to take care of a family member who has been infected with COVID-19. Furthermore, COVID-19 severely affects the elderly, who require more attention and care [12–14].

Previous studies showed that COVID-19 infection affects the mental health of the patients [15,16]. Patients infected with COVID-19 experienced severe psychological distress, including symptoms of anxiety and depression [15,16]. Lower education level and family history of psychiatric disorder were risk factors for anxiety, and home isolation was a risk factor for depression [15]. Aside from patients, there were few studies that showed that family members taking care of COVID-19 patients also experienced mental health impacts, including anxiety and depressive symptoms [17–19].

COVID-19 infection negatively affected family function. Family members experienced the highest dysfunction in the areas of growth and affection [20]. Family members were dissatisfied with the support they received from their families regarding their decisions to take on new activities and directions, and the way their family members expressed affection and responded to their emotions, such as anger or love [20]. Family members, particularly parents, also reported increased family conflicts due to the pandemic [21]. However, our knowledge of the psychosocial effects of COVID-19 on the families and caregivers of COVID-19 patients in the Philippines is still limited.

Therefore, understanding the impact of COVID-19 illness on families taking care of COVID-19 patients will contribute to guidelines and policies for responding to pandemics such as COVID-19 effectively. Hence, this study determined the psychosocial impact of COVID-19 on families of adult patients in isolation facilities in Metro Manila, Philippines. We showed the proportion of COVID-19 patient family members with psychological symptoms of anxiety and depression, their perceived family function and resources, and the factors associated with psychological symptoms and family dysfunction among study participants.

#### **2. Materials and Methods**

#### *2.1. Ethical Consideration*

This study was approved by the University of the Philippines Manila Research Ethics Board (UPMREB Code: 2020-280-01) and the Philippine Department of Health Single Joint Research Ethics Board (SJREB Code: 2020-100). Written informed consent was obtained from all participants (family members of COVID-19 patients) and the patients at the beginning of the study and during the subsequent follow-up interviews.

#### *2.2. Study Design*

The study employed a cohort study design using quantitative methodologies. The participants were interviewed during their family member's two-week home quarantine after discharge from the facility. A follow-up data collection was performed 8 weeks post discharge of the participants' family member who had COVID-19 using the same procedure.

#### *2.3. Sampling Design*

The study employed non-probabilistic sampling. Participants were chosen as convenient from the list of patients of selected COVID-19 healthcare facilities. The selection was based on the availability of contact details and ease of coordination.

#### *2.4. Study Sites*

The following COVID-19 designated healthcare facilities in Metro Manila were included in the study: (1) two community isolation units (CIUs) in Metro Manila, namely the PNP Kiangan Quarantine Facility in Camp Crame, Quezon City, and the University of the Philippines Diliman Silungan Molave Quarantine Facility in Diliman, Quezon City; and (2) one hospital facility, namely the COVID-19 areas of the Philippine General Hospital (PGH) in Ermita, Manila.

#### *2.5. Study Population*

The study involved family members taking care of adult patients diagnosed with COVID-19 at the study sites. The inclusion criteria included: (1) age greater than or equal to 18 years; (2) family member (as previously defined) of a COVID-19 patient admitted in a hospital or quarantine facility; (3) has lived in the same household as the patient for at least 12 months before the interview; (4) involved in the care of the patient; (5) agrees to participate in the study with a signed informed consent form. The exclusion criteria included: (1) with non-consenting family members of COVID-19 patients; (2) with preexisting clinically diagnosed psychiatric disorder before COVID-19 admission; (3) unable to provide consent due to physical or mental illness, including cognitive impairment; (4) unable to participate fully and answer questions due to physical or mental illness. Family members were able to discontinue their study participation and cancel their consent.

#### *2.6. Data Collection Procedure*

Data collection was undertaken through telephone or online calls with family members at 2 weeks and 8 weeks after discharge of their relative with COVID-19 infection. The research assistant or field interviewer used a semi-structured questionnaire during the data collection (interviewer-administered). Handwritten notes and voice recordings of calls using a call recording application were made. Electronic data is being stored in a wellsecured data cloud for storage. If the participant warranted initial psychosocial supportive care during the interview, the research assistant or field interviewer referred the patient to the principal investigator and co-investigators. The co-investigators then provided initial counseling to the participants through telephone, online voice, or video calls. If further management was necessary, the participant was referred to the Family Health Unit (FHU) clinic at the Outpatient Department of the Philippine General Hospital.

#### *2.7. Data Collection Tools*

In this study, quantitative data were obtained to determine the impact of the COVID-19 experience on families caring for COVID-19 patients. The questionnaire was pre-tested practically with 20 family members of COVID-19 patients who were not included in this study, to increase data quality before data collection began. The assessment tools used in the study have validated Filipino translation.

#### 2.7.1. Hospital Anxiety and Depression Scale (HADS)

Psychological symptoms of anxiety and depression were assessed using the Hospital Anxiety and Depression Scale (HADS). It is a self-report instrument designed to detect symptoms related to anxiety and depression. Initially designed for hospitalized patients, HADS has been used and validated in community settings and primary care practice. In the Philippines, a Filipino translation (HADS-P) has been validated among Filipino patients. It has fourteen items and two subscales, anxiety and depression, and each item is scored on a four-point scale of 0 to 3. A score of ≥11 was interpreted as positive for the emotional illness being tested since similar studies which used HADS/HADS-P utilized the same cut-off [22–24]. This tool has been used and validated in previous studies using telephone interviews with a Cronbach alpha of 0.76 to 0.86 [25].

#### 2.7.2. Family Assessment Tools

The following family assessment tools were used in the study: Filipino family APGAR (Adaptability, Partnership, Growth, Affection, and Resolve), and SCREEM (Social, Cultural, Religious, Economic, Education, and Medical) Family Resources Survey (SCREEM-RES). Filipino family APGAR, a translated and validated Filipino version of Smilkstein's family APGAR, was used to assess family functioning based on five parameters: Adaptability, Partnership, Growth, Affection, and Resolve. Each parameter is scored with a three-point scale ranging from 0 (hardly ever) to 2 (almost always). The total scores range from 0 to 10, with higher scores indicating higher satisfaction with family functioning. A score of 0–3 shows severe family dysfunction, 4–7 moderate family dysfunction, and 8–10 highly functional families [26].

The SCREEM Family Resources Survey (SCREEM-RES) is a validated and reliable tool to measure family resources used to cope with difficult situations. This instrument is a brief twelve-item questionnaire containing all the six original SCREEM domains and includes two items per domain. Participants were asked to choose one of the following responses: strongly disagree, disagree, agree, and strongly agree. The total SCREEM-RES scores were grouped using the following key: Severely Inadequate Family Resources = 0 to 12, Moderately Inadequate Family Resources = 13 to 24, Adequate Family Resources = 25 to 36. For each domain subscale, scores were grouped using the following key: Severely Inadequate Family Resources = 0 to 2, Moderately Inadequate Family Resources = 3 to 4, Adequate Family Resources = 5 to 6. This was previously validated in Filipino patients with a Cronbach's alpha of 0.80 for the entire scale [26].

#### *2.8. Data Analysis*

Data were analyzed using SPSS statistical software version 28.0 (IBM Corp). Statistical significance was set at α = 0.05, and all tests were two-tailed. Collected data were summarized using descriptive statistics, tables, and graphs. Means, medians, standard deviations, and interquartile ranges were computed for continuous variables, whereas frequencies and percentages were obtained from categorical variables. The internal consistencies of the questionnaires were reported as Cronbach alpha coefficients. Statistical comparisons between continuous variables were performed with an independent Student t-test for normally distributed data, whereas a Mann–Whitney U test was used if otherwise. A χ<sup>2</sup> test or Fisher's exact test was done for categorical variables. To check for statistical differences in the proportions of mental and social outcomes between 2 and 8 weeks after discharge, a paired-samples proportion test using McNemar was used. Binary logistic regression analysis was performed to determine factors influencing anxiety, depression, and family dysfunction symptoms. Associations between the exposure and outcome variables are presented as odds ratios and 95% CIs, after adjustment for confounders defined as exposure variables with *p* > 0.25 based on univariate analysis.

#### **3. Results**

#### *3.1. Sociodemographic Characteristics of the Participants*

A total of 104 participants were recruited for the project and completed the first interview, and 74 participants completed the second interview. Baseline sociodemographic characteristics of the 74 participants who completed the first and second interviews are shown in Table 1.


**Table 1.** Baseline sociodemographic characteristics of the family members of patients who completed the two interviews (*n* = 74).

Abbreviations: SD, standard deviation; IQR, interquartile range; PGH, Philippine General Hospital; PNP, Philippine National Police; UP, University of the Philippines; PIDS, Philippine Institute for Development Studies; ND, No Data.

#### *3.2. Dynamics of Anxiety and Depression among Family Members of COVID-19 Patients*

At the cut-off HADS-P anxiety score of 11, 43.2% of the participants had anxiety symptoms 2 weeks after discharge of their relative with COVID-19 infection. The prevalence of anxiety significantly decreased to 24.3% 8 weeks after discharge (*p* = 0.002). At the cut-off HADS-P depression score of 11, 16.2% of the participants had symptoms of depression 2 weeks after the discharge of their relatives with COVID-19 infection. The prevalence of depression significantly decreased to only 5.4% at 8 weeks post discharge (*p* = 0.021). Lastly, 13.5% of the participants had a mixed diagnosis of anxiety and depression at 2 weeks and 4.1% at 8 weeks post discharge (*p* < 0.001) (Table 2 and Table S1).

**Table 2.** The proportion of family members with symptoms of anxiety and depression at 2 and 8 weeks after discharge of their relatives with COVID-19 from the study sites (*n* = 74).


Among participants with anxiety symptoms at 2 weeks post discharge, 53.1% had resolved symptoms, and 46.9% had persistent anxiety symptoms at the 8th-week postdischarge follow-up. Most participants without anxiety symptoms at the second-week follow-up remained asymptomatic during the 8th-week follow-up. However, 7.1% developed new-onset anxiety symptoms during the 8th-week follow-up (Table 3). Among participants with depression 2 weeks post discharge, 83.3% had resolved symptoms, and only 16.7% had persistent symptoms of depression during the 8th-week follow-up. Most of the participants (96.8%) without symptoms of depression during the 2nd-week follow-up remained asymptomatic, whereas 3.2% developed new-onset symptoms of depression during the 8th-week follow-up (Table 3).

**Table 3.** Dynamics of anxiety and depressive symptoms among family members at 2 and 8 weeks after discharge (*n* = 74).


#### *3.3. Perceived Family Dysfunction*

The Family APGAR index was used to measure the general family function. The percentage of participants with a perceived moderate family dysfunction decreased from 9.5% at 2 weeks post discharge to 6.8% at 8 weeks post discharge, whereas those with perceived severe dysfunction increased from none to 4.1% from the 2nd week to 8th week post discharge (Tables 4 and S1). However, these observed changes did not reach statistical significance.

Analysis of the dynamics of perceived family dysfunction showed that among those with perceived dysfunction at 2 weeks, 57.1% retained the same view at 8 weeks after discharge. On the other hand, 7.5% of those without perceived dysfunction at 2 weeks developed perceived family dysfunction 8 weeks post discharge (Table 5).


**Table 4.** The proportion of family members with perceived family dysfunction at 2 and 8 weeks after discharge of their relatives with COVID-19 from the study sites (*n* = 74).

**Table 5.** Dynamics of perceived family dysfunction among adult patients and family members at 2 and 8 weeks after discharge of their relatives with COVID-19 from the study sites (*n* = 74).


#### *3.4. Perceived Inadequacy of Family Resources*

Among the resources measured by the SCREEM-RES questionnaire, the most inadequate resources for the family member participants were the economic, medical, and educational resources. The prevalence of perceived economic resource inadequacy decreased at 8 weeks post discharge. However, medical inadequacy increased. The resources least perceived to be inadequate were social, cultural, and religion. Perceived inadequacy in these resources increased at 8 weeks compared with 2 weeks after discharge. There was no increase in the overall perceived inadequacy from 2 weeks to 8 weeks post discharge (Tables 6 and S1).

**Table 6.** The proportion of patients' family members with perceived inadequate family resources at 2 and 8 weeks after discharge of their relatives with COVID-19 infection from the study sites (*n* = 74).


Analysis of the dynamics of perceived inadequacy of family resources showed that most participants with perceived inadequacy at 2 weeks retained the same view 8 weeks after discharge (59.3%). Meanwhile, 25.5% of those without perceived inadequacy at 2 weeks developed it 8 weeks post discharge (Table 7).

**Table 7.** Dynamics of perceived inadequacy of family resources among patients and family members at 2 and 8 weeks after discharge (*n* = 74).


#### *3.5. Factors Influencing Psychological Impact of COVID-19 Experience*

At 2 weeks after discharge, patient anxiety (*p* = 0.010) and perceived inadequate family resources (*p* = 0.032) were associated with anxiety symptoms among family members. Patient anxiety (*p* = 0.013) and low educational attainment (*p* = 0.002) were associated with anxiety symptoms among family members 8 weeks after discharge. On the other hand, patient depression (*p* = 0.013) was a factor related to depressive symptoms among family members 2 weeks after discharge (Table 8). Using multivariate logistic regression analysis, no identified factors were associated with depressive symptoms among family members at 8 weeks after discharge.

**Table 8.** Factors associated with psychological symptoms in patients' family members identified by multivariate logistic regression analysis.


<sup>1</sup> Adjusted for number of household members, diagnosed with chronic disease, employment status, perceived family functioning at 2 and 8 weeks after facility discharge, social resources at 2 weeks after facility discharge, overall family resources at 8 weeks after facility discharge, presence of patient anxiety at 2 and 8 weeks after facility discharge; <sup>2</sup> adjusted for highest educational attainment, perceived family functioning at 2 weeks after facility discharge, patient anxiety and depression at 2 weeks after facility discharge; <sup>3</sup> adjusted for marital status, number of household members, presence of chronic disease, knew somebody who died due to COVID-19, perceived family functioning at 2 weeks after facility discharge, medical resources at 2 weeks after facility discharge, patient anxiety and depression at 2 and 8 weeks after facility discharge.

#### **4. Discussion**

This prospective cohort study revealed a high prevalence of psychosocial symptoms among participants at 2 and 8 weeks after the discharge of their family member admitted for COVID-19 infection. Overall, family function and family resources contributed to anxiety and depression among patients and families post COVID. The study explored the impact of COVID-19 on the psychological symptoms of anxiety and depression and its associated sociodemographic, economic, and clinical factors. Anxiety was high in family members at 2 weeks and 8 weeks post COVID infection. Similarly, common mental health conditions such as anxiety, depression, post-traumatic stress disorder, and overall lower quality of life occur for up to 3 months or 12 weeks post COVID-19 infection.

The unpredictability of COVID-19 infection contributes to anxiety and depression among family members of patients with COVID-19. It impairs work, family engagements, and health [27]. Families may experience high stress, anxiety, and financial burden from missing work and unemployment concerns [28]. Moreover, family members' anxiety also stemmed from their inability to feel connected to the patient and informed about care [29]. During the pandemic, family members were not allowed to stay and visit their relatives who were admitted to the hospital for COVID-19. Family members struggled to feel informed about the care they could not witness and had difficulty understanding information. A previous study showed that visits to COVID-19 patients in the ICU reduced anxiety among family members [19]. Another study showed that the family members of COVID-19 patients experienced mental health symptoms 12 months after ICU admission of their relatives for COVID-19 infection. Family members also experienced disruption of quality of life and work-related problems [17].

In this study, risk factors identified for anxiety among family members were patient depression and a low level of educational attainment. Similar findings have been made in other studies on family members of COVID-19 patients [18,29]. COVID-19 has resulted in massive unemployment worldwide. Several studies have already shown that the unemployment rate increased negative mental health outcomes during the COVID-19 pandemic [30–32]. It exacerbated pre-existing mental health disorders and created new disorders for others [32]. Collectively, these data showed that the government and the health care system should support patients' families financially.

The family function measurements taken using APGAR scores showed that the participants' perceived family dysfunction increased during the period from 2 weeks to 8 weeks after the discharge of their family member. This suggests that family dysfunction exacerbated by admission of a family member to a health facility due to COVID-19 could have long-term effects. Previous studies showed that social and family relationships were disrupted for patients and their caregivers [33]. The stresses created by the COVID-19 pandemic have put families and their interrelationships under tremendous pressure [34]. A previous study in Portugal showed that almost 20% of the participants perceived their families to have severe dysfunction or moderate dysfunction [20]. Family dysfunction is a predisposing factor for developing the emotional problems of anxiety and depression during the COVID-19 pandemic [35].

Interestingly, despite the decrease in the prevalence of anxiety and depression in the participants during the period from 2 weeks to 8 weeks after discharge, perceived severe family dysfunction still increased. We surmised that family-level dysfunction manifests later than personal-level conditions, such as anxiety and depression. The family members and the patient need to adapt to the consequences of COVID-19 infection [36], such as long COVID-19 symptoms, financial responsibility accrued from hospitalization, loss of productivity due to inability to go to work, and unemployment. Caregiver fatigue may occur later, hence, the appearance of severe family dysfunction later after the patient's discharge. Previous studies showed that the COVID-19 pandemic changed the structure and routine of the family, especially for those who suffered from the disease [21,37,38]. The disruptions in the usual routine resulted in physical and mental health problems and

family matters [37]. Family members, particularly parents, also reported increased family conflicts due to the pandemic [21].

The perceived inadequate family resources did not decrease 8 weeks post COVID-19 discharge. The economic burden did not decrease during the period from 2 weeks to 8 weeks post discharge. Financial factors may contribute to severe family dysfunction, as shown by the APGAR score. COVID-19 has generated a considerable economic and financial burden on patients. Aside from the high cost of hospitalization, long-term health effects of COVID-19 such as kidney disease and long COVID-19 symptoms may induce chronic medical needs that expose patients and their families to long-term financial risk [39,40]. A previous study conducted in the Philippine General Hospital showed that the average out-of-pocket payment for COVID-19 patients less than 60 years old ranged from Php 25,899 to Php 44,428.63 (\$538 to \$924.44), whereas for patients older than 60 this ranged from Php 4005.60 to Php 32,920.20 (\$83.35 to \$684.98) [41]. Despite the financial help from national health insurance in the Philippines, the patients still need to pay out of pocket. This puts a financial burden on the patients and their families, especially since the daily minimum wage of an average worker in the National Capital Region of the Philippines only ranges from Php 533 to Php 570 (\$9.54 to \$10.20) [42].

The participants perceived a significant lack of access to medical resources at 8 weeks post discharge. The persistent perception of the participants of having inadequate access to medical resources is not surprising. The pandemic has brought disruption and barriers to accessing medical care. The availability of healthcare services related to COVID-19 disease and other chronic diseases has deteriorated due to the diversion of health services for urgent COVID-19 cases [43,44]. This lack of access to medical care was more pronounced among those belonging to the lower socioeconomic strata [45,46].

This study also showed that inadequate family resources and low educational attainment were associated with anxiety in the participants. Previous studies showed that low educational levels were significantly associated with both anxiety and depression [47,48]. On the other hand, higher educational attainment is protective against developing a spectrum of psychiatric disorders [48]. Higher educational attainment is also associated with higher income [49] and being more capable of shouldering medical expenses from COVID-19 hospitalization. The material advantage is protective of the negative effect of COVID-19 on the mental health of individuals, as shown in a previous study [50].

This study supports the need for more holistic COVID-19 practice guidelines to include psychosocial interventions among family caregivers. We emphasize the need to have family assessments in routine medical history taking. The use of validated tools for early detection and screening of mental disorders such as the Patient Health Questionnaire (PHQ) and Depression, Anxiety, and Stress Scale (DASS) is recommended. In addition, the Family APGAR and SCREEM-RES are good family assessment tools to check functional family relationships and family resource adequacy. Healthcare professionals should involve family members during active treatment and post-COVID-19 care of hospitalized patients and those in the quarantine facilities as part of treatment protocols. A multi-disciplinary approach to the active and follow-up care of COVID-19 patients and their family caregiver is needed. The care team must include health professionals who can provide psychological, social support, and home care.

This study has several limitations. The study only recruited family members of patients from selected COVID-19 healthcare facilities within Metro Manila, the epicenter of the pandemic in the Philippines. Experiences in areas outside Metro Manila and other metropolitan cities, where healthcare facilities have significantly different situations, may vary substantially from those recorded in the study. Second, the structural distance inherent in telephone interviews affected the engagement and retention of samples of the study because of the absence of an interpersonal relationship commonly established in face-to-face interviews. Lastly, symptoms of anxiety and depression were detected using a screening tool, and the presence of either anxiety or depression disorders was not confirmed with a diagnostic tool commonly used in psychiatry, such as DSM-5.

#### **5. Conclusions**

This study provided a valuable in-depth understanding of the mental health status of family members caring for relatives with COVID-19 infection. This study described the prevalence of and factors associated with psychological distress, particularly symptoms of anxiety and depression, in family members of patients with COVID-19 admitted to hospital and quarantine facilities. They experienced symptoms of anxiety and depression even after the discharge of their relative with COVID-19 infection. They also perceived moderate to severe family dysfunction and inadequacy of economic, medical, and educational resources in the family. These symptoms and perceptions persisted for 2 to 8 weeks after the discharge of their relative with COVID-19 infection. Depressive symptoms in a relative with COVID-19 infection tend to influence the occurrence of anxiety among family members. Our findings can be used to guide healthcare professionals caring for COVID-19 patients and their family members. COVID-19 infection generates a secondary public health crisis through stress-related disorders among family members of COVID-19 patients. Therefore, relevant interventions are recommended.

**Supplementary Materials:** The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jcm11175236/s1, Table S1: Instrument scores of patients' family members at 2 and 8 weeks after discharge of their relatives with COVID-19 from the study sites (*n* = 74).

**Author Contributions:** Conceptualization, L.A.-N., I.K.B.T., A.G.O.L. and G.D.P.D.; methodology, L.A.-N., I.K.B.T., A.G.O.L. and G.D.P.D.; Formal analysis, L.A.-N., I.K.B.T., A.G.O.L., G.D.P.D. and O.A.G.T.; Investigation, L.A.-N., I.K.B.T., A.G.O.L. and G.D.P.D.; data curation, L.A.-N., I.K.B.T., A.G.O.L., G.D.P.D. and O.A.G.T.; writing—original draft preparation: L.A.-N., I.K.B.T., A.G.O.L., G.D.P.D. and O.A.G.T.; writing—review and editing, I.K.B.T. and O.A.G.T.; supervision: L.A.-N.; funding acquisition, L.A.-N. and I.K.B.T. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study was funded by the Department of Health AHEAD Program 2018.

**Institutional Review Board Statement:** This study was approved by the University of the Philippines Manila Research Ethics Board (UPMREB Code: 2020-280-01) and the Philippine Department of Health Single Joint Research Ethics Board (SJREB Code: 2020-100).

**Informed Consent Statement:** Written informed consent has been obtained from the participants to publish this paper.

**Data Availability Statement:** The data that supports the findings of this study are available in the Supplementary Material of this article.

**Acknowledgments:** We would like to thank DOH, DOST-PCHRD, and the University of the Philippines Manila for supporting the study. We would also like to express our utmost gratitude to the officials and staff of the Philippine General Hospital, UP Diliman Silungan Molave Quarantine Facility, and PNP Kiangan Quarantine Facility who helped us in the participant identification, recruitment, and interviews.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

#### **References**


## *Review* **Prevalence of Depression among Healthcare Workers during the COVID-19 Outbreak: A Systematic Review and Meta-Analysis**

**Beatriz Olaya 1,2 , María Pérez-Moreno 3, Juan Bueno-Notivol 4,\*, Patricia Gracia-García 4, Isabel Lasheras <sup>5</sup> and Javier Santabárbara 2,5,6**


**Abstract:** Background: There is evidence of a high psychological toll from the COVID-19 pandemic in healthcare workers. This paper was aimed at conducting a systematic review and meta-analysis of studies reporting levels of depression among healthcare workers during the COVID-19 and estimating the pooled prevalence of depression. Methods: We searched for cross-sectional studies listed on PubMed from 1 December 2019 to 15 September 2020 that reported prevalence of depression in healthcare workers, nurses, medical doctors, and COVID-19 frontline professionals. The pooled proportions of depression were calculated with random effects models. Results: We identified 57 studies from seventeen countries. The pooled prevalence of depression in healthcare workers was 24% (95% CI: 20–28%), 25% for nurses (95% CI: 18–33%), 24% for medical doctors (95% CI: 16–31%), and 43% for frontline professionals (95% CI: 28–59%). Conclusions: The proportion of depression in nurses and medical doctors during the COVID-19 pandemic was similar to that found in the general population as previously reported in other meta-analyses conducted with smaller numbers of studies. Importantly, almost half of the frontline healthcare workers showed increased levels of depression. There is need for a comprehensive, international response to prevent and treat common mental health problems in healthcare workers.

**Keywords:** depressive symptoms; COVID-19; nurses; medical doctors; frontline; pooled prevalence

#### **1. Introduction**

The new coronavirus (SARS-CoV-2) was first identified in a wet market in Wuhan, Hubei province, China, in December 2019 [1]. This virus causes a highly infectious acute respiratory syndrome (COVID-19) that can be associated with serious pneumonia and eventually lead to death. Due to its rapid spread around the world, the World Health Organization [2] declared the COVID-19 as a pandemic in March 2020, and from its identification to this date (12th November 2020), more than 51.9 million people have been confirmed as cases worldwide, with 1.2 million deaths [3]. The enormous impact on people's physical and mental health, and on economic systems worldwide, is one of the main challenges for society in this century [4].

Healthcare workers (HCW) are a fundamental part of the global response to COVID-19. Because of their close personal exposure to patients with COVID-19, their risk of infection is

**Citation:** Olaya, B.; Pérez-Moreno, M.; Bueno-Notivol, J.; Gracia-García, P.; Lasheras, I.; Santabárbara, J. Prevalence of Depression among Healthcare Workers during the COVID-19 Outbreak: A Systematic Review and Meta-Analysis. *J. Clin. Med.* **2021**, *10*, 3406. https:// doi.org/10.3390/jcm10153406

Academic Editor: Emmanuel Andrès

Received: 2 July 2021 Accepted: 26 July 2021 Published: 30 July 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

very high. A recent study reports HCW to be at an 11.6 times higher risk of infection than the general population, although this risk decreases to 3.4 after accounting for the differences in testing frequency between HCW and the general community [5]. Besides this higher risk of infection, several observational studies conducted during the COVID-19 pandemic have shown that health professionals are at a higher risk of developing psychological problems [6]. Growing patient load and working under pressure in resource-deprived settings might increase psychological stress among HCW [7,8]. They are also more exposed to prolonged work shifts, lack of adequate equipment (i.e., protective equipment (PPE)), and fear of infecting themselves or relatives [9,10]. This fear, in turn, may be associated with anxiety, depression, and insomnia [11–13].

HCW thus constitute one of the groups most vulnerable to psychological distress, requiring immediate interventions to improve their wellbeing and the healthcare system capacity. Two very recent systematic reviews and meta-analyses on the prevalence of anxiety and depression have been published, reporting the pooled prevalence among HCW. The first one, conducted by Pappa et al. [14] in April 2020, included a total of thirteen crosssectional studies (all of them conducted in China except one, from Singapore) reporting a pooled prevalence of anxiety of 23.2% and 22.8% for depression. The second [15] was based on seven studies conducted in China and found an increased risk of anxiety and depression in HCW, compared with other professionals (OR = 1.61; 95%CI 1.33 to 1.96 and OR = 1.32; 95%CI 1.09 to 1.60, respectively).

Due to the rapid, evolving nature of this health emergency, an increasing number of other studies from different countries addressing mental health problems among HCW have been published in recent months. Thus, the present study is aimed at updating and extending the previous work of Pappa et al. [14] and da Silva and Neto [15] by conducting a systematic review and meta-analysis of studies published afterwards reporting a global prevalence of anxiety and depression among HCW during the COVID-19 outbreak.

#### **2. Materials and Methods**

The present study followed the PRISMA guidelines for reporting systematic reviews and meta-analysis [16] (Supplementary Table S1).

#### *2.1. Search Strategy*

The search strategy (Supplementary Table S2) included all cross-sectional studies informing about the prevalence of depression that were published from 1 December 2019 to 15 September 2020. The search was conducted by two researchers (MPM and JBN) using MEDLINE via PubMed. Briefly, they focused on depression, although an anxiety term was additionally included to examine whether these articles also included relevant information about depression. Depression could be measured either using diagnostic tools (e.g., structured interviews) or standardized scales to assess depressive symptomatology. As our main objective was to calculate the overall prevalence of depression, in case we found a study using scales, we considered the presence of depression reported according to a certain cut-off point for that given scale. Thus, henceforth we use the term "depression" to refer to either a full-blown diagnosis or presence of depression according to a cut-off point.

Search terms also included samples of HCW, nurses, medical doctors, and/or frontline HCW. There was no language restriction. We inspected references from selected articles to detect additional studies. In case of disagreement, a third and fourth reviewer (JS and IL) were consulted to reach a consensus.

#### *2.2. Selection Criteria*

The following inclusion criteria for studies were used: (1) studies providing crosssectional data on the proportion of depression during the COVID-19 outbreak; (2) studies conducted with samples of health care workers; (3) studies in which the assessment methods for depression were described; and (4) studies for which the full-text was available.

Studies that used other specific samples (e.g., adolescents and patients) and review articles were excluded from the present study.

We extracted the following data using a pre-designed form: country, sample size, prevalence rates of depression, proportion of women, average age, instruments used to assess depression, response rate, and sampling methods.

#### *2.3. Assessment of Methodological Quality*

Two independent reviewers (JS and JBN) rated the methodological validity of selected articles before their inclusion in the review using the Joanna Briggs Institute (JBI) standardized critical appraisal instrument for prevalence studies [17]. This tool uses nine criteria to evaluate quality with a score ranging from zero ('No') to one ('Yes').

In case of disagreement between the two reviewers, there was a discussion to resolve it between them or with a third reviewer (PGG).

#### *2.4. Statistical Analysis*

We used a generic inverse variance method with a random effect model [18]. To check heterogeneity across studies, we calculated the Hedges *Q* statistic (a *p* value < 0.10 indicates statistical significance) and the *I* <sup>2</sup> statistic and 95% confidence interval [19]. *I* <sup>2</sup> values between 25% and 50% are considered low, 50%–75% moderate, and 75% or greater high [20]. Different study designs or demographic characteristics may explain the heterogeneity. Thus, we calculated meta-regression and carried out subgroup analyses [21] to find potential sources of heterogeneity [22]. A sensitivity analysis was also made by omitting studies one by one. This allowed us to learn how each individual study influenced the overall result. Publication bias was determined by visually inspecting a funnel plot. Since funnel plots might inaccurately assess publication bias in meta-analyses of proportion studies [23], we additionally calculated Egger's [24] and Begg's tests [25], with *p* values < 0.05 indicating publication bias.

Despite the fact that one inclusion criterion for a given study was the use of HCW samples, we were interested in separately calculating the pooled prevalence for the following groups: HCW in general (with no distinction of the type of worker or working in the frontline), nurses, medical doctors, and frontline HCW. Frontline HCW were those who provided direct care to patients with a diagnosis of infection by COVID-19 or who worked in units where care was provided. Additionally, for practical purposes, the pediatric HCW, physical therapists, and laboratory HCW were considered as HCW. Statistical analyses were conducted with STATA statistical software (version 10.0; College Station, TX, USA) and R [26].

#### **3. Results**

Flowchart of the search strategy and study selection process is shown in Figure 1. We initially identified 354 studies. After removing duplicates and studies after the first screening, 186 articles were read in full. Finally, a total of 57 studies were included in the present meta-analysis [6,12,13,27–80].

The results are organized as follows: Table 1 shows the characteristics of those studies (46) that reported prevalence rates of depression in HCW (without distinction in the type of workers); Table 2 displays characteristics of studies reporting data from nurses (14); Table 3 characteristics for medical doctors (10); and Table 4 for frontline HCW (12).

Approximately half of the studies were conducted in China (*n* = 24), but we also found studies from India (*n* = 4), Italy (*n* = 3), Turkey (*n* = 3), Singapore (*n* = 2), and one study from each of the following countries: Cameroon, Croatia, Jordan, Kosovo, Libya, Nepal, Poland, Serbia, South Korea, Spain, Switzerland, and the USA. The sample size ranged from 46 to 14,825 participants, and the mean age ranged from 29 to 47 years. All studies except one included both men and women, with women predominating in most of the studies that reported this (40 out of 43). All studies used online questionnaires and all except two used non-random methods. Twenty-three studies reported response rate, ranging from 20.4% to

94.9%. All studies measured depression using standardized scales, most commonly the Patient Health Questionnaire-9 (PHQ-9, *n* = 21 studies), the Depression, Anxiety, and Stress Scale (DASS-21, *n* = 8 studies), and the Hospital Anxiety and Depression Scale (HADS, *n* = 8 studies).

The risk of bias ranged from 5 to 8, with a mean score of 6.95 (Supplementary Table S3, Table 1). The most common limitations were (a) recruitment of participants not appropriate (56 studies), (b) response rate not reported or large number of non-responders (33 studies), and (c) sample size too small to ensure good precision of the final estimate (19 studies).

**Figure 1.** Flowchart of the study selection.


**Table1.**Characteristicsofthestudiesincludedinthemeta-analysisbasedonsamplesofhealthcareworkers.


68

Anxiety and Depression

 Scale; HAMD = Hamilton Depression

 Rating Scale; PHQ-9 = Patient Health

Questionnaire;

 SDS = Zung's Self-Rating

 Depression

 Scale.


**Table2.**Characteristicsofthestudiesincludedinthemeta-analysisbasedonsamplesofnurses.

AnxietyandStressscales;HADS=HospitalAnxietyandDepressionScale;PHQ-9=PatientHealthQuestionnaire;SDS=Zung'sSelf-RatingDepressionScale.


**Table 3.** Characteristics of the studies included in the meta-analysis based on samples of medical doctors.

**Table 4.** Characteristics of the studies included in the meta-analysis based on samples of frontline healthcare workers.


Figure 2 shows the estimated overall prevalence of depression in HCW (24%; 95% CI: 20%–28%), 25% in nurses (95% CI: 18%–33%) (Figure 3), 24% in medical doctors (95% CI: 16%–31%) (Figure 4), and 43% in frontline HCW (95% CI: 28%–59%) (Figure 5), with significant heterogeneity between studies (Q test: *p* < 0.001) across these four categories. Additionally, the prevalence of depression in frontline HCW was significantly higher than in HCW overall (*p* < 0.05).

**Figure 2.** Forest plot for the prevalence of depression among healthcare workers.


**Figure 3.** Forest plot for the prevalence of depression among nurses.

**Figure 4.** Forest plot for the prevalence of depression among medical doctors.

## **Figure 5.** Forest plot for the prevalence of depression among frontline healthcare workers.

Potential sources of heterogeneity were investigated across the studies. Our subgroup analysis showed that prevalence of depression was lower in studies using the DASS-21, those carried out in China, studies using convenience sampling methods and those of high methodological quality (Table 5).


**Table 5.** Overall prevalence rates of depression according to study characteristics.

\* *p* value obtained from univariate meta-regression. In bold, significant associations.

The exclusion of studies one-by-one from the analysis did not substantially change the overall prevalence rate of depression. Thus, no single study had a disproportional impact on the overall prevalence (data not shown).

Visual inspection of the funnel plot (Figure 6) suggested a small publication bias for the prevalence estimate in HWC, nurses, and medical doctors, confirmed by significant results in the Egger's test (*p* < 0.05). However, no publication bias was detected for frontline HCW (Egger's test: *p* = 0.928).

**Figure 6.** Funnel plot for the prevalence of depression.

#### **4. Discussion**

The present systematic review and meta-analysis identified a total of 57 cross-sectional studies reporting rates of depression among HCW. The pooled prevalence rate of depression in HCW was 24%, and when analyzing professional groups, we found that the rates were similar in nurses (25%) and medical doctors (24%), whereas up to 43% of frontline HCW report depression. The overall prevalence of depression found in HCW, nurses and medical doctors is similar to that found in a recent meta-analysis conducted from January 2020 to May 2020. This meta-analysis was based on 12 population-based studies conducted during the COVID-19 outbreak, finding that the overall prevalence was 25% in the general population [81].

Since the outbreak of COVID-19 pandemic in January 2020, the attention paid to the impact on mental health among HCW has grown exponentially, as indicated by the large number of studies found. A recent systematic review and meta-analysis of the prevalence of depression, anxiety, and insomnia among HCW during the pandemic included thirteen studies published up to 17 April 2020. In that study, the pooled prevalence of depression was 22.8%, based on ten studies [14]. A subgroup analysis for different occupational categories found that the pooled prevalence for nurses was 30.3% and for medical doctors 25.9%. These figures are slightly higher than those reported in our meta-analysis. These discrepancies might be explained by the different number of studies included. In the Pappa et al. meta-analysis, only five studies were considered in calculating separate pooled prevalence of depression for nurses and doctors, whereas our study was based on 14 and 10 studies to calculate prevalence depression in nurses and doctors, respectively. Another remarkable difference between the two meta-analyses is the origin of the samples, with the Pappa et al. study mainly focused on a Chinese population. However, our meta-analysis includes a broad range of countries from very different regions worldwide. This regional heterogeneity, along with a greater sample size, allows us to provide an updated estimation of the pooled prevalence of depression among HCW.

Our pooled prevalence of depression found in HCW (25%) is also higher compared with another systematic review and meta-analysis based on samples of HCW that reported a prevalence of 12.2% [15]. The study was conducted April–May 2020 and was based on seven cross-sectional studies, all of them conducted in China. Again, the diversity of the origin of the samples and/or the number of studies included might explain these discrepancies.

Note that all the studies included in our meta-analysis used self-reported standardized questionnaires to assess depressive symptomatology. Additionally, the use of a great variety of scales might have led to differences in the estimation of the presence of depression. In fact, our results show that those studies using the DASS-21 questionnaire reported lower prevalence rates of depression. Despite the convenience of using the same instruments and the inclusion of a diagnosis based on clinical interviews, this is not always possible in epidemiological studies.

Similarly, Muller et al. [82] conducted a rapid systematic review in May 2020 focusing on several outcomes such as mental health problems and risk or resilience factors from quantitative and qualitative data. They found a total of 19 studies, with a percentage of depression ranging from 5% to 51%, and a median of 21%. According to their systematic review, the most common risk factors for mental health problems in HCW were being a woman, being exposed to infected patients, and the worry of being infected.

An important contribution of the present meta-analysis is the calculation of the pooled prevalence of depression in frontline HCW. The prevalence is significantly higher (43%) compared with other types of HCW. The mental toll of working at the frontline during previous pandemic outbreaks, such as that of severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS), has previously been reported to be high [83,84]. The fear of being infected, stigmatization and uncertainty put these workers under extraordinary stress. A qualitative study conducted among frontline HCW in Wuhan (China) pointed up the intensive work (i.e., long working hours and use of personal protective equipment), fear of infecting others or being infected, managing relationships under stressful situations, and feeling powerless to handle patients' conditions as common experiences during the COVID-19 outbreak [85].

However, factors contributing to increased vulnerability to depression among HCW as well as resilience characteristics (such as coping strategies) might be culturally different. Additionally, inequalities related to health systems and resources across high- and lowincome countries might also contribute to the differing impact of COVID-19 on mental health among HCW from diverse settings. Furthermore, note that the first wave of COVID-19 was characterized by a high degree of uncertainty about the illness, its treatment, and its prognosis. Since the outbreak, there has been an intensive international response to fight the virus along with a research agenda aimed at finding effective treatments for infected patients and preventing the spread of the virus (e.g., vaccines). This means that COVID-19 is a rapid, evolving health challenge that requires up-to-date data to ensure appropriate surveillance of mental health, and specifically for vulnerable subpopulations such as HCW.

There are limitations to be considered when interpreting our results. First, the majority of studies included in the present meta-analysis used convenience samples, so representativeness of HCW might be jeopardized. Second, depression was mainly assessed by means of self-reported data drawn from questionnaires, which might have introduced biases such as social desirability [86], as well as being less accurate than clinical interviews. Third, the inclusion of cross-sectional studies makes it difficult to determine causal associations between the pandemic and depression. Fourth, some of the studies included in

the calculation of the pooled prevalence for the three groups of HCW (i.e., nurses, medical doctors, and frontline HCW) might have also been included in the calculation of the pooled prevalence for HCW. Thus, caution should be taken when interpreting our results. Fifth, our systematic review was only conducted in a medical database (MEDLINE); thus, some articles, especially those related to psychology, might not be included. Finally, we found some sources of heterogeneity. For example, using convenience sampling methods, conducting studies in China, and using the DASS-21 questionnaire were associated with lower prevalence rates of depression. Related to this, half of the studies were carried out with Chinese samples, so the results of the present meta-analysis should be approached with caution. Future studies should endeavor to investigate the prevalence of depression among HCW in other countries, and use randomized sampling designs whenever possible, as well as longitudinal designs to determine the evolution of mental health problems in this population.

In summary, our meta-analysis shows that depression during the COVID-19 pandemic is a common mental condition among HCW, with the frontline HCW especially affected. A unified response to help HCW during the pandemic should be placed in the international agenda. Comprehensive psychological support, along with regular and intensive training for HCW, can help safeguard their well-being [87]. Common mental problems, such as depression, should be routinely assessed to detect those HCW at high risk of mental disorders and in need of intensive interventions to alleviate their symptomatology.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/jcm10153406/s1, Table S1: PRISMA checklist, Table S2: Search strategy in MEDLINE via Pubmed, Table S3: Quality assessment with the JBI Appraisal Checklist for Prevalence Studies.

**Author Contributions:** Conceptualization, B.O., J.B.-N., and J.S.; methodology, M.P.-M., J.B.-N., P.G.- G., and I.L.; software, J.S.; validation, B.O., J.B.-N., P.G.-G., and J.S.; formal analysis, J.S.; resources, J.S.; data curation, M.P.-M. and J.B.-N.; writing—original draft preparation, B.O., M.P.-M., J.B.-N., and P.G.-G.; writing—review and editing, B.O., J.B.-N., and J.S.; visualization, I.L. and J.S.; supervision, I.L. and J.S.; project administration, B.O. and J.S.; funding acquisition, B.O. All authors have read and agreed to the published version of the manuscript.

**Funding:** B.O. was supported by the Miguel Servet (CP20/00040) contract, funded by the Instituto de Salud Carlos III and co-funded by the European Union (ERDF/ESF, "Investing in your future").

**Institutional Review Board Statement:** No institutional approval was necessary for this study.

**Informed Consent Statement:** Not applicable to this study.

**Data Availability Statement:** Data sharing is not applicable to this article as no new data were created or analyzed in it.

**Conflicts of Interest:** We declare that P.G.-G. has received grant support from Janssen, AstraZeneca and the Ilustre Colegio de Médicos de Zaragoza; she has received honoraria from AstraZeneca and Lilly; and she has received travel support from Lilly, Almirall, Lundbeck, Rovi, Pfizer and Janssen-Cilag. None of these activities was related to the current project. The remaining authors declare no conflicts of interest.

#### **References**


## *Article* **Influence of the Cumulative Incidence of COVID-19 Cases on the Mental Health of the Spanish Out-of-Hospital Professionals**

**Raúl Soto-Cámara 1,2,3 , Susana Navalpotro-Pascual 3,4,5,\*, José Julio Jiménez-Alegre 3,4,6, Noemí García-Santa-Basilia 2,3, Henar Onrubia-Baticón 2,3, José M. Navalpotro-Pascual 3,4 , Israel John Thuissard 3,7 , Juan José Fernández-Domínguez 3,4,7,8, María Paz Matellán-Hernández 2,3, Elena Pastor-Benito 3,4, Carlos Eduardo Polo-Portes 3,4,7, Rosa M. Cárdaba-García 2,3,9,\* and on behalf of the IMPSYCOVID-19 Study Group †**


**Abstract:** This study aimed to analyze the psychological affectation of health professionals (HPs) of Spanish Emergency Medical Services (EMSs) according to the cumulative incidence (CI) of COVID-19 cases in the regions in which they worked. A cross-sectional descriptive study was designed, including all HPs working in any EMS of the Spanish geography between 1 February 2021 and 30 April 2021. Their level of stress, anxiety and depression (DASS-21) and the perception of self-efficacy (G-SES) were the study's main results. A 2-factor analysis of covariance was used to determine if the CI regions of COVID-19 cases determined the psychological impact on each of the studied variables. A total of 1710 HPs were included. A third presented psychological impairment classified as severe. The interaction of CI regions with the studied variables did not influence their levels of stress, anxiety, depression or self-efficacy. Women, younger HPs or those with less EMS work experience, emergency medical technicians (EMT), workers who had to modify their working conditions or those who lived with minors or dependents suffered a greater impact from the COVID-19 pandemic in certain regions. These HPs have shown high levels of stress, anxiety, depression and medium levels of self-efficacy, with similar data in the different geographical areas. Psychological support is essential to mitigate their suffering and teach them to react to adverse events.

**Keywords:** coronavirus infections; health personnel; emergency medical services; psychological stress; anxiety; depression; self-efficacy; incidence

**Citation:** Soto-Cámara, R.; Navalpotro-Pascual, S.; Jiménez-Alegre, J.J.; García-Santa-Basilia, N.; Onrubia-Baticón, H.; Navalpotro-Pascual, J.M.; Thuissard, I.J.; Fernández-Domínguez, J.J.; Matellán-Hernández, M.P.; Pastor-Benito, E.; et al. Influence of the Cumulative Incidence of COVID-19 Cases on the Mental Health of the Spanish Out-of-Hospital Professionals. *J. Clin. Med.* **2022**, *11*, 2227. https://doi.org/ 10.3390/jcm11082227

Academic Editor: Mauro Giovanni Carta

Received: 29 March 2022 Accepted: 13 April 2022 Published: 15 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

#### **1. Introduction**

The declaration of the disease caused by the virus SARS-CoV-2, named COVID-19, as "The Sixth International Public Health Emergency" and the proclamation of the resulting situation as a pandemic by the World Health Organization (WHO) has produced important changes at the economic, social and health levels in all countries [1,2]. As in other areas, in Spain, this affectation has not remained uniform over time but has fluctuated depending on the cumulative incidence (CI) of cases in the different waves and geographical regions. For this, the Center for the Coordination of Health Alerts and Emergencies of the Ministry of Health and the National Epidemiological Surveillance Network of the National Center for Epidemiology of the Carlos III Health Institute have been designated as the organizations in charge of collecting CI information in the different Spanish regions and monitoring possible change from the start of the COVID-19 pandemic to the present [3,4].

During this period, numerous studies have analyzed the impact of the COVID-19 pandemic on the mental health of the general population and certain sectors [5,6]. Health professionals (HPs) have been among the most affected groups, focusing most of this research on primary care or hospital workers and specific national care models [7–11]. HPs have faced very intense and stressful work situations, such as work overload, prolonged work shifts, fewer hours of rest, no clear and defined protocols for action, strict safety instructions and measures, the constant need for concentration and vigilance, the lack of personal protective equipment and reduced social contact, as well as having to perform tasks for which many professionals have not been prepared [12,13]. This situation of stress has put both the physical and mental health of the HPs at risk. Their general well-being has been altered, and they have started to show high levels of anxiety and depression, other emotional disorders, sleep problems, difficulty in interpersonal relations, dysfunctional cognitive reactions, substances use behaviors, post-traumatic stress, and even vicarious traumatization stemming from compassion towards the patients that they were treating [14–17].

Generally, emergency medical service (EMS) is the department in charge of out-ofhospital care for critically ill patients in most countries. During the COVID-19 pandemic, this service had had to develop new policies, procedures, and protocols to address the consequences of this epidemiological situation, characterized by an increase in the volume of calls and the care of patients with suspected signs or confirmed cases [18]. However, the specific scientific research referring to out-of-hospital EMSs has been very limited [18,19], even though they continue to be one of the frontline healthcare providers. Like HPs in other settings, the findings of these studies have shown a negative impact of the COVID-19 pandemic on the mental health of out-of-hospital workers, with an increase in the prevalence of disorders due to stress, anxiety, depression, insomnia or burnout [18,19].

For all these reasons, the objective of the present study was to analyze the level of psychological affectation of the HPs of the Spanish EMSs, according to the CI of COVID-19 cases of the geographical regions in which they worked.

#### **2. Materials and Methods**

#### *2.1. Study Design-Participants*

A cross-sectional descriptive study was designed. The study population was all HPs working in any EMS in the Spanish geography between 1 February and 30 April 2021. Not accepting voluntary participation in the study or not completing the entire questionnaire were considered exclusion criteria.

For the estimation of the sample size, it was considered that 23,467 HPs worked in EMS in Spain in 2020, according to data from the Statistical Portal of the Primary Care Information System of the Ministry of Health [20]. It was necessary to recruit at least 1066 subjects to achieve a confidence level of 95% and an accuracy of 3%, considering a 15% possible loss.

#### *2.2. Procedure—Data Collection*

Participants were selected using non-probabilistic snowball sampling. An online questionnaire was used for data collection. The link to the questionnaire was distributed through the Prehospital Emergency Research Network (RINVEMER) of the Spanish Society of Emergency Medicine (SEMES) and the managers of the different EMSs. In the first part of the questionnaire, the participants were informed of the characteristics and objectives of the study and its anonymous and voluntary nature. Its completed return implied the person's informed consent to participate in the research. To guarantee the anonymity of the HPs, no personal data was collected that could allow their identification, even in those cases in which they specifically requested feedback on the results obtained, for which a personal alphanumeric code was created. Participants could withdraw from the study at any time without giving any reason. The time required to answer the questionnaire was approximately 15–20 min. All doubts were resolved by email.

The research protocol was approved by the Medicine Ethics and Research Committee of the East Valladolid Health Area (PI-20-2052), respecting the principles of the Declaration of Helsinki and its successive revisions [21].

#### *2.3. Main Outcomes—Instruments*

The study's main results were the level of stress, anxiety and depression of the HPs and their perception of self-efficacy.

The reduced version of the Depression Anxiety Stress Scale (DASS-21) was used as a self-reported instrument to assess the intensity of 21 different symptoms associated with a negative emotional state [22]. It consists of 3 subscales, with 7 items each one: (i) stress, which evaluates tension, irritability, nervousness, impatience, agitation, and negative affect; (ii) anxiety, which assesses physiological activation, musculoskeletal symptoms and subjective sensation of anxiety; and (iii) depression, which evaluates hopelessness, dysphoria, sadness, anhedonia, low self-esteem, and low positive affect. The HPs must indicate the frequency with which they have experienced these symptoms in the previous 2 weeks using a 4-point Likert scale (0: Never; 3: Always). In each subscale, the total score is obtained by adding the points of each item and multiplying it by 2. The score of the subscales ranges between 0 and 42, so the higher the value, the greater the degree of symptomatology. Similarly, this score can be categorized as normal, mild, moderate, severe or extremely severe. Its adaptation and validation to the Spanish population were carried out by Bados et al., with acceptable psychometric properties [23]. It has been widely used to assess the psychological impact of the COVID-19 pandemic on the general population [6] and HPs [18] as it has good discriminant validity in screening for mental disorders [24].

To evaluate the person's perception of their ability to adequately handle different stressful situations, the Spanish adaptation of the General Self-Efficacy Scale (G-SES) was used [25,26]. It is made up of 10 items, with 10 response options (1: Never; 10: Always). The score ranges between 10 and 100, associating higher values with greater perceived self-efficacy. It presents good psychometric properties, with predictive capacity on coping styles, and internal consistency of 0.87 [26].

Other variables were also collected through an ad hoc questionnaire: sex, age, living with minors or dependent persons, professional category, previous work experience in EMS, change in working conditions, previous diagnosis of COVID-19 or CI of COVID-19 cases. For analytical purposes and based on the CI per 100,000 inhabitants defined by the Health Authorities on 1 February 2021, the Spanish geography was divided into 3 areas: region with low CI if ≤4999 cases, region with medium CI if 5000–6999 cases, and region with high CI if ≥7000 cases [27] (Figure 1).

**Figure 1.** Distribution of the CI regions of COVID-19 cases per 100,000 inhabitants.

#### *2.4. Statistical Analysis*

Categorical variables were summarized as absolute frequencies and percentages, while quantitative ones were in terms of mean and standard deviation (SD). The compliance of the normality criteria of the quantitative variables was evaluated using the Kolmogorov– Smirnov test; in those cases in which they did not follow a normal distribution, the criteria proposed by Blanca et al. were considered [28]. To contrast the levels of stress, anxiety, depression and self-efficacy in regions with the same CI of COVID-19 cases or between the 3 regions considered, the χ<sup>2</sup> test, the Student's *t*-test for independent samples, the one-way analysis of variance or the Pearson's correlation were calculated, depending on the nature of the variables. For multiple comparisons, post hoc tests were corrected by Bonferroni adjustment. In addition, to find out if the different regions were a determining factor in the psychological impact of each of the variables, a 2-factor analysis of covariance (study variables × region) was performed. Statistical significance was considered if *p* < 0.05. Statistical analysis was carried out with SPSS version 25.0 software (IBM-Inc, Chicago, IL, USA).

#### **3. Results**

The sample consisted of 1710 participants; 50.58% were women, with a mean age of 43.54 years (SD ± 9.94). The most represented professional category was emergency medical technicians (EMT) (*n* = 765), followed by doctors (*n* = 474) and nurses (*n* = 453), with a mean work experience in EMS of 15.22 years (SD ±9.15). In relation to the mental health of these HPs, 37.39% (*n* = 639), 39.36% (*n* = 673) and 30.46% (*n* = 521) presented levels of stress, anxiety and depression categorized as severe or extremely severe. The mean scores obtained in stress, anxiety, depression and self-efficacy were 20.61 (SD ± 11.08), 13.08

(SD ± 11.17), 15.74 (SD ± 11.11) and 70.78 (SD ± 15.75), respectively. The distribution of their descriptive characteristics in the different CI regions is summarized in Table 1.

**Table 1.** Descriptive characteristics of the sample based on the CI of COVID-19 cases from the different regions.


Values are expressed as mean ± standard deviation or frequencies (percentages). Abbreviations: CI—Cumulative Incidence; EMT—Emergency Medical Technicians; EMS—Emergency Medical Service; COVID-19—Coronavirus Disease-19.

In areas with medium or high CI, women presented greater stress, anxiety, and depression; men who worked in areas with low CI reported less stress than those employed in areas with a higher number of COVID-19 cases. Regarding self-efficacy, men perceived higher values in areas with low CI. The interaction of gender and region did not affect the psychological variables analyzed (Table 2).

**Table 2.** Level of stress, anxiety, depression and self-efficacy according to sex and the CI regions of COVID-19 cases.


Values are expressed as mean ± standard deviation. Abbreviation: CI—Cumulative Incidence. \* *p* < 0.05 between sexes in the same CI region. \*\* *p* < 0.01 between sexes in the same CI region. \*\*\* *p* < 0.001 between sexes in the same CI region. & *p* < 0.05 between CI regions in the same sex. <sup>a</sup> *p* < 0.05 in the post-hoc analysis (Bonferroni test).

EMTs who worked in regions with low or high CI reported negative emotional states compatible with stress or depression more frequently than other professional categories. Their anxiety levels were also significantly higher in the three areas, regardless of the number of COVID-19 cases. The professional category and region combination did not influence the mean scores obtained on the DASS-21 and the G-SES (Table 3).

**Table 3.** Level of stress, anxiety, depression and self-efficacy according to professional categories and the CI regions of COVID-19 cases.


Values are expressed as mean ± standard deviation. Abbreviation: CI—Cumulative Incidence; EMT—Emergency Medical Technicians. \* *p* < 0.05 between professional categories in the same CI region. \*\* *p* < 0.01 between professional categories in the same CI region. \*\*\* *p* < 0.001 between professional categories in the same CI region. a,b *p* < 0.05 in the post-hoc analysis (Bonferroni test).

HPs who were forced to change their work schedule, location, or dedication reported higher levels of stress, anxiety and depression in the three regions of CI. When the psychological impact of the COVID-19 pandemic was analyzed, considering the need or not for changes in working conditions, it was concluded that the different regions were not a determining factor (Table 4).

**Table 4.** Level of stress, anxiety, depression and self-efficacy according to change in working conditions and the CI regions of COVID-19 cases.


Values are expressed as mean ± standard deviation. Abbreviation: CI—Cumulative Incidence. \* *p* < 0.05 between professional categories in the same CI region. \*\* *p* < 0.01 between professional categories in the same CI region. \*\*\* *p* < 0.001 between professional categories in the same CI region.

Having a previous diagnosis of COVID-19 or living with minors and/or dependents was not related to significant changes in the values of stress, anxiety, depression and selfefficacy, except for a higher level of anxiety among those HPs with vulnerable dependents in regions with low CI. In both cases, when the interaction of these variables with the CI region was analyzed, no influence was observed on the psychological parameters (Tables 5 and 6).

**Table 5.** Level of stress, anxiety, depression and self-efficacy according to the previous diagnosis of COVID-19 and the CI regions of COVID-19 cases.


Values are expressed as mean ± standard deviation. Abbreviation: COVID-19—Coronavirus Disease-19; CI— Cumulative Incidence.

**Table 6.** Level of stress, anxiety, depression and self-efficacy according to living with minors/dependents and the CI regions of COVID-19 cases.


Values are expressed as mean ± standard deviation. Abbreviation: CI—Cumulative Incidence. \* *p* < 0.05 between professional categories in the same CI region.

Both HPs' age and EMS work experience were indirectly and weakly correlated with stress levels, anxiety, depression and self-efficacy, regardless of the number of COVID-19 cases in the region they worked (Table 7).


**Table 7.** Level of stress, anxiety, depression and self-efficacy according to age, EMS work experience and the CI regions of COVID-19 cases.

Values are expressed as Spearman's r. Abbreviation: IA—Cumulative Incidence; EMS: Emergency Medical Service; \* *p* < 0.05. \*\* *p* < 0.01. \*\*\* *p* < 0.001.

#### **4. Discussion**

This study is proposed to identify the impact of the COVID-19 pandemic on the mental health of HPs in Spanish EMSs and its influence on certain socio-demographic and labor variables, according to the number of cases registered in each region. The interaction of the CI regions with the other study variables considered has not altered the levels of stress, anxiety, depression and self-efficacy of the HPs, unlike what was observed by Brillon et al. [29]. However, in certain regions, a greater impact of the COVID-19 pandemic was observed on women, younger HPs or those with less EMS work experience, EMTs, workers who had to modify their working conditions or those who lived with minors or dependents.

Around a third of the participants presented severe or extremely severe levels of stress, anxiety and depression, data higher than those reported by HPs from other care settings [9,30–32]. Working on the front line, in areas where the unpredictability of the attended cases is greater or in environments with a high probability of contagion, as is the case of the EMSs, is becoming a risk factor for the development of negative responses to challenging situations [33,34]. The high number of HPs with scores in psychopathological alarm ranges should be considered a warning sign of the future psychosocial consequences of the acute phase of the COVID-19 pandemic, such as post-traumatic stress or burnout [35,36].

In this study, the levels of stress, anxiety or depression of the HPs have not been influenced by the number of COVID-19 cases declared in the different geographical areas. However, several authors have shown the existence of an "epicentric effect," which explains a higher prevalence of these psychological conditions the closer the HPs are to the most affected regions [29,34,37–40]. Continuous exposure to stressful elements for long periods of time, lack of social support, or living the same reality as the patients have contributed to exacerbating this effect among HPs from regions with high CI [29].

A higher level of stress and emotional burden has been observed in women who worked in regions with medium or high CI and lower use of coping strategies. The less time dedicated to self-care or self-compassion and the high work pressure during the health emergency has led to the appearance and maintenance of this situation [41,42]. All this has been favored by factors such as gender discrimination, the progressive feminization of the health sector, the difficulties in reconciling work and family, the traditional assumption of the role of primary caregiver at home, the lack of sufficient support systems, the greater empathy in providing care, or the greater ability to express feelings to others and develop emotional responses to stressful events [41,43–47]. The lower psychological affectation of men, especially in regions with low CI, is related to their relative underreporting of symptoms and underuse of health services [48,49] and the widespread use of coping strategies focused on the problem. These strategies limit the ability to recognize their emotional difficulties and become aware of their own experiences [50].

Younger HPs and those with less EMS work experience were more vulnerable to developing symptoms compatible with stress, anxiety, or depression disorders, regardless of geographic area. Some authors have speculated that these workers, whatever their professional role, have less self-confidence and less resistance at a psychological

level, and a greater degree of uncertainty in how to act in unforeseen and/or complex situations [37,41,51,52].

The psychological well-being of all participants has been affected during the COVID-19 pandemic. This finding suggests their great personal and emotional involvement, being more notable in the EMTs who worked in areas with low and high CI. The lesser affectation of doctors and nurses could be related to the use of coping strategies based on intellectualization and denial and greater resistance to somatization, related to personal achievements, professional experience or self-awareness [53,54].

The modification of working conditions increased the vulnerability of HPs to stress, anxiety and depression in all regions. The reorganization and restructuring of the EMSs and the adaptation of the workplace for health reasons may be the main causes of these changes. The sudden outbreak of COVID-19 has caused unpredictable changes in the work of HPs, with an increase in the demand for care, greater contact with patients suffering from serious and complex diseases, a reduction in rest times and a lack of socio-occupational support [29,55,56]. To deal with this situation, the EMSs have created units specifically dedicated to the care of patients with COVID-19 and have moved part of the HPs from one job to another [29,52]. The adaptation of these displaced HPs to this new context, in constant change, has caused them an additional mental burden [52]. On the other hand, some authors have observed a greater impact of the COVID-19 pandemic on HPs whose workplace had to be adapted. Among the possible explanations for this result are fear and lack of information about the interaction of their previous diseases with SARS-CoV-2 infection and its possible consequences in the medium and long term [57–60]. Furthermore, HPs with previous mental diseases are more likely to present this type of psychological symptoms [61].

Only HPs who lived with vulnerable people in regions with low CI had higher anxiety levels. This finding can be attributed to the fear of becoming infected and the consequent risk of transmitting the disease to their relatives [9,31,62].

The psychological discomfort of HPs must be considered beyond a merely individual level, as it directly impacts patient care. The most distressed HPs participate less in the therapeutic relationship, make more mistakes and even compromise clinical results [40]. Based on this premise, the need for health authorities to design psychological support strategies in which HPs reflect on their psycho-emotional reactions to adverse events is reinforced [63].

These results must be interpreted within the context of their limitations. It has not been possible to determine a causal relationship between variables due to the study's cross-sectional nature. Psychological distress has been assessed only through self-report measures administered online, limiting access to HPs less accustomed to the use of new technologies. The use of non-probabilistic snowball sampling may have induced a selfselection bias by favoring the participation of HPs who are particularly sensitive to the issue and those who have a greater degree of affectation. Data collection lasted 12 weeks. This fact may have affected the quality of the responses since the CI of COVID-19 cases, and the perception related to the infection have differed between the first and last day. The lack of studies on this topic in the out-of-hospital setting has hindered comparing and contrasting the results obtained. Among its strengths is the collection of data from a large sample of HPs from all EMSs of the Spanish geography and the use of validated questionnaires with excellent psychometric properties.

#### **5. Conclusions**

The HPs from the Spanish EMSs present high levels of stress, anxiety, depression and medium levels of self-efficacy. Similar data were observed in different geographical areas. A greater impact of the COVID-19 pandemic has been observed on women, younger HPs or those with less EMS work experience, EMTs, workers who had to modify their working conditions or those who lived with minors or dependents in certain regions. In these HPs, psychological support is essential to mitigate their suffering, helping them to reflect on their psycho-emotional reactions to adverse events.

**Author Contributions:** Conceptualization, R.S.-C. and S.N.-P.; methodology, R.S.-C., S.N.-P., J.J.J.-A., N.G.-S.-B., H.O.-B. and J.M.N.-P.; software, R.S.-C., J.M.N.-P. and I.J.T.; validation, R.S.-C., S.N.-P. and R.M.C.-G.; formal analysis, R.S.-C., S.N.-P., J.J.J.-A., N.G.-S.-B., H.O.-B., J.M.N.-P., I.J.T., J.J.F.-D., M.P.M.-H., E.P.-B., C.E.P.-P. and R.M.C.-G.; investigation, R.S.-C., S.N.-P., J.J.J.-A., N.G.-S.-B., H.O.-B., J.M.N.-P., I.J.T., J.J.F.-D., M.P.M.-H., E.P.-B., C.E.P.-P. and R.M.C.-G.; resources, J.J.F.-D. and S.N.-P.; writing—original draft preparation, R.S.-C., S.N.-P., J.J.J.-A., N.G.-S.-B., H.O.-B. and J.M.N.-P.; writing—review and editing, R.S.-C., R.M.C.-G. and S.N.-P.; visualization, R.S.-C., S.N.-P., J.J.J.-A., N.G.-S.-B., H.O.-B., J.M.N.-P., I.J.T., J.J.F.-D., M.P.M.-H., E.P.-B., C.E.P.-P. and R.M.C.-G.; supervision, R.S.-C. and S.N.-P.; project administration, S.N.-P.; funding acquisition, J.J.F.-D. and S.N.-P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Fundación ASISA and Sociedad Española de Urgencias y Emergencias (SEMES).

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of East Valladolid Health Area (PI-20-2052).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data for this study are available by contacting the corresponding authors.

**Acknowledgments:** European University of Madrid (Spain), for the statistical analysis.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **References**


## *Article* **The "Healthcare Workers' Wellbeing [Benessere Operatori]" Project: A Longitudinal Evaluation of Psychological Responses of Italian Healthcare Workers during the COVID-19 Pandemic**

**Gaia Perego 1,\* , Federica Cugnata 2,3, Chiara Brombin 2,3, Francesca Milano <sup>4</sup> , Emanuele Preti <sup>1</sup> , Rossella Di Pierro <sup>1</sup> , Chiara De Panfilis 5, Fabio Madeddu <sup>1</sup> and Valentina Elisabetta Di Mattei 2,4**


**Abstract:** Background: COVID-19 forced healthcare workers to work in unprecedented and critical circumstances, exacerbating already-problematic and stressful working conditions. The "Healthcare workers' wellbeing (Benessere Operatori)" project aimed at identifying psychological and personal factors, influencing individuals' responses to the COVID-19 pandemic. Methods: 291 healthcare workers took part in the project by answering an online questionnaire twice (after the first wave of COVID-19 and during the second wave) and completing questions on socio-demographic and work-related information, the Depression Anxiety Stress Scale-21, the Insomnia Severity Index, the Impact of Event Scale-Revised, the State-Trait Anger Expression Inventory-2, the Maslach Burnout Inventory, the Multidimensional Scale of Perceived Social Support, and the Brief Cope. Results: Higher levels of worry, worse working conditions, a previous history of psychiatric illness, being a nurse, older age, and avoidant and emotion-focused coping strategies seem to be risk factors for healthcare workers' mental health. High levels of perceived social support, the attendance of emergency training, and problem-focused coping strategies play a protective role. Conclusions: An innovative, and more flexible, data mining statistical approach (i.e., a regression trees approach for repeated measures data) allowed us to identify risk factors and derive classification rules that could be helpful to implement targeted interventions for healthcare workers.

**Keywords:** COVID-19; healthcare workers; mental health; mixed effects model; Random Effects/ Expectation Maximization (RE-EM) Tree

#### **1. Introduction**

Healthcare settings may represent a challenging workplace, characterized by long and undefined working hours, excessive workloads, competitiveness of training, high responsibility, and constant exposure to suffering, illness, and mourning [1,2]. However, lack of time, stigma, and concerns around confidentiality may prevent seeking psychological support [2,3].

Consistently, increasing evidence shows that healthcare workers around the world report high levels of depression, anxiety, stress, burnout, and post-traumatic stress disorder (PTSD) [1,4]. However, the literature is limited, and the samples analyzed are heterogeneous, resulting in a prevalence of psychiatric symptoms ranging from 30% to 60% for physicians [5–7] and from 11% to 73% for nurses [8].

**Citation:** Perego, G.; Cugnata, F.; Brombin, C.; Milano, F.; Preti, E.; Di Pierro, R.; De Panfilis, C.; Madeddu, F.; Di Mattei, V.E. The "Healthcare Workers' Wellbeing [Benessere Operatori]" Project: A Longitudinal Evaluation of Psychological Responses of Italian Healthcare Workers during the COVID-19 Pandemic. *J. Clin. Med.* **2022**, *11*, 2317. https://doi.org/10.3390/jcm11092317

Academic Editor: Alfonso Troisi

Received: 16 March 2022 Accepted: 19 April 2022 Published: 21 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

In turn, neglected mental health issues in healthcare workers can affect both team and individual work performance, resulting in a reduced quality of care [9], lower patient satisfaction, and higher rates of medical errors and staff turnover [1,9,10], with a remarkable impact on the healthcare economy [1,10,11].

This topic is currently of critical importance, given the detrimental consequences of the COVID-19 pandemic on the entire population, e.g., ref. [12] and healthcare workers in particular [13], as shown in previous epidemics [14], worsening already-problematic and stressful workplace conditions. Indeed, frontline healthcare workers have been working for more than a year in unprecedented and critical circumstances to cope with the COVID-19 pandemic, while being exposed to potentially traumatic or stressful factors such as fear of contagion, a lack of personal protection equipment, longer working hours, countless patient deaths and numerous critical patients, and continuous updates to hospital procedures [10,15].

Although healthcare workers faced the second wave of the pandemic with more therapeutic knowledge than the first, they still had limited resources to care for COVID-19 patients. In fact, the unpredictability of the disease and the pandemic's course, the extremely high number of deaths and critical patients, and the necessity to make difficult choices about prioritizing care remained serious concerns about COVID-19 [15].

Several reviews and meta-analyses show that working in COVID-19 wards affected healthcare workers' mental health in terms of high rates of depression, anxiety, insomnia, burnout, and PTSD symptoms [13,15–18]. A recent meta-analysis [10] found the following pooled prevalence of psychiatric outcomes among healthcare workers: 30% for anxiety, 31.1% for depression, 56.5% for acute stress, 20.2% for post-traumatic stress, and 44% for sleep disorders. However, as other two meta-analyses pointed out [19,20], most healthcare workers actually experienced mild psychiatric symptoms, with moderate and severe symptoms being less common.

This is in line with our baseline findings, which point to low or mild mental health issues among healthcare workers after the main peak of the outbreak's first phase [21]. Notwithstanding, the pandemic has evolved quickly, and early studies were unable to capture post-traumatic stress disorders and the mental health outcomes associated with a state of prolonged stress. Longitudinal studies are, thus, required to analyze the effect of time on these psychiatric outcomes [10] and to differentiate the effect of the pandemic from all other pre-existing stressors in the hospital work environment [22].

In this contribution, we present findings of the "Healthcare workers' wellbeing [Benessere Operatori]" project, which aims at evaluating psychological distress, as well as socio-demographic, situational, and personal factors that may affect individuals' psychological responses to the COVID-19 pandemic. All these factors have been assessed twice: at baseline (between 9 May and 13 July 2020, after the main peak of the COVID-19 outbreak in Italy) and during the second wave (between 5 and 30 December 2020). Along with more traditional mixed effects modelling approaches, an alternative advanced data mining approach, extending regression trees methodology to repeated measures data, has been applied. The applied procedure is extremely flexible and appealing since it allows for the identification of the best variables with the best cut-off values for discriminating among different outcome responses, while uncovering complex relationships among predictors. It thus provides an effective tool to be used in clinical practice to support decision making process.

#### **2. Materials and Methods**

#### *2.1. Participants and Procedure*

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the University of Milano-Bicocca (protocol n. 0024531/20), the Ethics Committee of the IRCCS San Raffaele Scientific Institute (protocol n. 109/2020), and the Ethics Committee of the Parma Local Health Authority (protocol n. PG0019826\_2020).

This study is part of a web-based longitudinal project to examine the psychological impact of COVID-19 on a sample of Italian healthcare workers involved in the management of the pandemic. After reading the informed consent, participants voluntarily completed an online survey, administered through Qualtrics and sent to the e-mail address provided, during the baseline assessment. We assessed participants' working conditions, individual perception of the COVID-19 situation, anxiety, depression, and insomnia symptoms, posttraumatic stress, state anger, and burnout levels. In the present study, we will also analyze the coping strategies and perceived social support measured during the baseline study.

In total, 344 healthcare workers participated in the second survey, ten of whom stated that they had not worked in the previous three months and were thus excluded from the analysis. Finally, statistical analyses have been carried out on a sample of 291 respondents with complete records on both demographic and psychological variables.

#### *2.2. Measures*

A self-report questionnaire was used to collect socio-demographic and work-related information from participants, including their age, gender, psychological/psychiatric history, ward in which they worked, and whether they had received emergency training.

The Depression Anxiety Stress Scale (DASS-21) [23,24] is a 21-item scale that assesses general distress using a tripartite model of psychopathology. This questionnaire includes three subscales: depression, anxiety, and stress. Each item is rated on a four-level Likert scale (0 = never; 3 = almost always). The total score is calculated by adding together the response values for each item. Higher scores suggest severe levels of depressive, anxiety, and stress symptoms. The original version of the questionnaire showed an internal reliability with Cronbach's alpha coefficient of 0.91 for the depression scale, 0.84 for the anxiety scale, and 0.90 for the stress scale [24]. The total score of the Italian version reported a Cronbach's alpha value of 0.90, with subscale values ranging from 0.74 to 0.85 [23].

The Insomnia Severity Index (ISI) [25,26] is a self-report questionnaire that assesses the nature, severity, and impact of insomnia using seven items rated on a five-level Likert scale (0 = "no problem"; 4 = "very severe problem"), with scores ranging from 0 to 28. The dimensions evaluated are severity of sleep onset, sleep maintenance, early morning awakening problems, sleep dissatisfaction, interference of sleep difficulties with daytime functioning, noticeability of sleep problems by others, and distress caused by the sleep difficulties. The original version of the ISI reported a Cronbach's alpha coefficient of 0.74 [26]. The Italian version showed a Cronbach's alpha coefficient of 0.75 [25].

The Impact of Event Scale-Revised (IES-R) [27,28] is a 22-item self-report questionnaire for evaluating the frequency of intrusive and avoidant thoughts and behaviors associated with a traumatic event. Items are rated on a five-point Likert scale (0 = "not at all"; 4 = "extremely"). The IES-R is divided into three subscales. Intrusion (8 items) assesses intrusive thoughts, nightmares, intrusive feelings, and imagery related to the traumatic event; Avoidance (8 items) evaluates avoidance of feelings, situations, and ideas; Hyperarousal (6 items) measures difficulty in concentrating, anger and irritability, psychophysiological arousal in response to reminders, and hypervigilance. The original version showed high levels of internal consistency (Intrusion: α = 0.87–0.94, Avoidance: α = 0.84–0.87, Hyperarousal: α = 0.79–0.91) [28]. The Italian version shows good (0.84) and acceptable (0.71) internal consistency for the intrusion subscale and the avoidance subscale, respectively [27].

The State-Trait Anger Expression Inventory-2 (STAXI-2) [29] is a 57-item self-report questionnaire that measures five domains of anger: State-Anger, Trait-Anger, Anger Expression-In, Anger Expression-Out, and Anger-Control. Responses are rated on a fourpoint Likert scale, ranging from 1 (not at all) to 4 (almost always). Cronbach's α coefficients range from 0.73 to 0.76, indicating high internal reliability for all the subscales except for the Trait Anger Scale/Angry Reaction [29]. In the present study, we only used the State-Anger subscale to assess healthcare workers' acute reaction to the pandemic.

The Maslach Burnout Inventory (MBI) [30,31] consists of 22 items, divided into three subscales, that assess the three components of the burnout syndrome: emotional exhaustion (9 items), depersonalization (5 items), and professional realization (8 items). Each item is rated on a seven-point Likert scale (0 = "never"; 6 = "every day"). The subscales showed good internal consistency both for the original version (α = 0.71–0.90) [30] and for the Italian version (α = 0.68–0.87) [31].

The Brief Cope [32,33] is a 28-item questionnaire, divided into 14 subscales, measuring coping responses. Each item is rated on a four-level Likert scale (0 = "I have not been doing this at all"; 3 = "I have been doing this a lot"). Coping strategies can be grouped into problem-focused (strategies aimed at changing a stressful situation: active coping, use of instrumental support, positive reframing, and planning), emotion-focused (strategies to regulate emotions associated with a stressful situation: use of emotional support, venting, humor, acceptance, self-blame, religion), and avoidance coping strategies (physical or cognitive efforts to disengage from the stressor: self-distraction, denial, substance use, behavioral disengagement) [34,35]. The original version of the questionnaire showed Cronbach's alpha coefficients, ranging from 0.50 to 0.90 [32], while the Italian version of the questionnaire revealed omega coefficients for reliability, ranging from 0.439 to 0.959 [33].

The Multidimensional Scale of Perceived Social Support (MSPSS) [36,37] is a 12-item self-administered questionnaire evaluating social support perceived by family, friends, and significant others, rated on a 7-point Likert scale (1 = "very strongly disagree"; 7 = "very strongly agree"). Higher scores indicate higher perceived social support. The internal reliability of the questionnaire is good, with Cronbach's alpha coefficients ranging from 0.85 to 0.91 [38]. The Italian version shows good indices of reliability with Cronbach's alpha coefficients ranging from 0.81 to 0.98 [36].

Furthermore, we assessed how worried participants were about the possibility that themselves, their relatives, their friends, and their colleagues could contract COVID-19. Four items were rated on a five-point Likert scale (1 = "not at all"; 5 = "extremely"). A total score of worry was obtained by averaging items scores.

Finally, we evaluated participants' working conditions over the previous three months in several areas, including eating, sleeping, working shifts, isolation, and wearing appropriate protective equipment. Seven items were rated on a five-point Likert scale (1 = "not at all"; 5 = "very much"). A total score of working conditions was obtained by averaging item scores. Higher scores indicate worse working conditions.

#### *2.3. Statistical Analysis*

Median and interquartile range (IQR) were used as summary statistics to describe continuous variables, while categorical variables were expressed as frequencies and percentages. Radar plots were used to visualize differences between measurements collected during the two evaluations in the different healthcare worker categories.

Linear mixed-effects (LME) models [39] were applied to evaluate the changes in the psychological outcomes over time while accounting for respondent-specific heterogeneity through random effects specification. The variables included in the models were: time (categorical with two levels, T0 and T1, respectively, at baseline and during the second wave), gender, occupation, working or having worked in COVID-19 wards (time dependent variable), worry scores and the evaluation of working conditions, the presence of psychological or psychiatric symptoms in the past, having attended emergency training, perceived social support as measured by the MSPSS, and the three Brief COPE subscales. To highlight specific differences in the outcome variables over time for the different healthcare workers categories, we also entered in the model the interaction between occupation and time.

Standard transformations (square root, power, ordered quantile normalization) were applied to outcome variables to satisfy model regression assumptions.

To examine psychological measure dynamics over time, within a data mining framework, an extension of regression trees methodology accounting for correlation structure among observations was considered. This approach is suited for studies with repeated measures and longitudinal data. The tree-based estimation method, proposed and implemented in the R RE-EM tree package by Sela and Simonoff [40], was considered.

A regression tree [41] implements a binary recursive partitioning in which predictor variables that best discriminate among response profiles, along with the optimal cut-points, are automatically chosen. The splitting criterion is based on maximizing the reduction in the sum of squares, for the node, until convergence is reached. The approach is very flexible in handling missing values and both quantitative and qualitative predictors.

Following the branches of the resulting tree, hence considering rules derived from the variables' best cut-off values, it is possible to derive different patterns of longitudinal responses. Regression trees provide an effective tool for supporting decision-making in clinical practice and identifying relevant variables associated with different outcomes, while uncovering complex relationships among predictors. Trees were estimated using the same covariates included in the mixed models. All the analyses were performed using R statistical software (version 4.1.1, https://cran.r-project.org/index.html, accessed on 10 October 2021). The significance level was set at 0.05.

#### **3. Results**

Participants' characteristics are shown in Table 1. The final sample included 291 participants. The median age was 46 years (IQR = [35.00, 54.00]), ranging from 23 to 72 years; 239 (82.1%) were female.

Among the sample, 23% reported having a psychological/psychiatric history, and only 16.8% reported having undergone emergency training.

Concerning their occupation, 31.3% of the participants were physicians, 33.3% were nurses, 27.9% were other healthcare workers, and 7.6% were clerks. At T0, 33.3% of the participants worked in a COVID-19 ward, and 15.5% worked in a COVID-19 ward at T1.


**Table 1.** Demographic, clinical, and occupational characteristics (*n* = 291).

Abbreviation: no., number; Psych History, psychiatric history.

Table 2 shows the characteristics of the participants stratified by occupation, and radar plots in Figure 1 display the average score of the psychological constructs of interest for each occupation group at each time point. Overall clerks reported higher average scores on almost all the scales, and their psychological condition seems to worsen, at least at a descriptive level, at the second time point. State-anger, DASS-21 scales, and emotional exhaustion subscale are, on average, higher for all the healthcare workers at the second time point (complete descriptive statistics are reported in Table S1). Observed ranges for all the psychometric scales are reported in Table S2.

**Figure 1.** Radar charts displaying average scores of the investigated psychometric variables at T0 and T1, stratified by occupation (Table S1). Out of the 291 participants, 91 were physicians, 97 were nurses, 81 were other healthcare workers, and 22 were clerks. Abbreviation: DASS Depr, Depression Anxiety Stress Scale-Depression; DASS Anx, Depression Anxiety Stress Scale-Anxiety; DASS Stress, Depression Anxiety Stress Scale-Stress; MBI Prof Real, Maslach Burnout Inventory-Professional Realization; MBI Depers, Maslach Burnout Inventory-Depersonalization; MBI Emo Ex, Maslach Burnout Inventory-Emotional Exhaustion; STATE Ang, STATE Anger; IES Hypar, Impact of Event Scale- Hyperarousal; IES Avoid, Impact of Event Scale-Avoidance; IES Intr, Impact of Event Scale-Intrusion; ISI Tot, Insomnia Severity Index Total score.


**Table 2.** Demographic, clinical, and occupational characteristics stratified by occupation (*n* = 291).

Abbreviation: no., number; MSPSS, Multidimensional Scale of Perceived Social Support; COPE, Brief Cope; IQR, InterQuartile Range; Psych History, psychiatric history; CONDWORK, working conditions.

#### *3.1. DASS-21—Depression, Anxiety, and Stress*

Table 3 displays the estimated models for the DASS-21 subscales. DASS-21 subscales are not significantly different at T1, with respect to the baseline values (T0). Higher levels of worry, worse working conditions, and having a psychological/psychiatric history significantly increase anxiety, depression, and stress levels. Having undergone emergency training significantly decreases all DASS-21 subscales. Higher levels of perceived social support decrease only participants' depression levels. Considering the effects of coping strategies, we found that the use of problem-focused coping significantly decreases all DASS-21 subscales, avoidant coping significantly increases all DASS-21 subscales, and emotional-focused coping significantly increases only depression and stress levels. Finally, nurses show higher anxiety levels than physicians.

Figure 2 shows the estimated regression trees for the prediction of the DASS-21 subscales scores. For depression, among all the variables, the algorithm selected both avoidant coping and working conditions as the variables best discriminating among participants' response profiles. High levels of avoidant coping and worse working conditions predict the highest levels of depression, whereas low levels of avoidant coping and better working conditions predict lower levels of depression symptoms. For subjects with avoidant coping ≥ 2.2, the mean depression level is 4.4. For subjects having a working conditions score < 2.4 and avoidant coping scores lower than 1.7, instead, the predicted depression level is 2.

When anxiety is considered as an outcome, in addition to avoidant coping and working conditions, the algorithm selected the worry scale.

Following the tree branches, participants with higher scores on the worry scale (≥3.4) and avoidant coping scores, greater than or equal to 2.2, show the highest average value equal to 3.9. Conversely, participants with scores on the worry scale lower than 3.4, and showing avoidant coping scores lower than 1.9, show the lowest average value equal to 1.5. In the tree for stress, psychiatric/psychological history and problem-focused coping

strategy were selected, in addition to splitting variables characterizing the other two trees, to best discriminate among different response profiles.

**Figure 2.** RE-EM tree for the DASS-21 subscales. Depression and anxiety scales were square root transformed. Each tree represents a series of splits starting at the top of the tree. Starting from the top node, a series of questions are presented based on the splitting variables and corresponding cut-off values. Depending on the answer, other branches may appear until the final node, which displays the average predicted outcome value for participants, satisfying all the conditions, leading to that node and the proportion of subjects falling in the node itself. For example, in the first tree for the depression scale, the top split assigns observations having avoidant coping scores greater than or equal to 2.2 to the right branch. The predicted depression level for these subjects is given by the mean response value for the individuals in the data set with avoidant coping ≥ 2.2. For such subjects, the mean depression level is 4.4. Among subjects who have avoidant coping scores < 2.2, the working conditions also affect depression level. For subjects with avoidant coping score < 2.2 and working conditions ≥ 2.4, the predicted depression level is 2.9. For subjects having working conditions < 2.4 and avoidant coping scores between 1.7 and 2.2, the predicted depression level is 2.7, whereas for subjects having a working conditions score < 2.4 and avoidant coping scores lower than 1.7, the predicted depression level is 2. The same logic applies to all the other trees. Abbreviation: CONDWORK, working conditions; Psych History, psychiatric history.


**Table 3.** Estimates (standard-errors) of the models for the DASS-21 subscales. Depression and anxiety scales were square root transformed.

\*\*\* *p* < 0.0001; \*\* *p* < 0.01; \* *p* < 0.05. Abbreviation: Ref, reference; CONDWORK, working conditions; Psych History, psychiatric history; MSPSS, Multidimensional Scale of Perceived Social Support, COPE, Brief Cope.

#### *3.2. MBI Emotional Exhaustion, Insomnia, IES-R Intrusion, and State Anger*

Table 4 shows the estimated models for the MBI emotional exhaustion scale, the ISI score, the IES-R Intrusion score, and the State anger score. When comparing the two time points, the state anger score significantly increases at T1 with respect to the baseline values (T0). Only for clerks, the MBI emotional exhaustions score also significantly increases at T1 compared to T0. Focusing on the occupation, overall, nurses report higher ISI levels and IES-R Intrusion levels than physicians. Higher levels of worry and worse working conditions significantly increase all of the considered scales. A previous history of psychological or psychiatric symptoms significantly increases the MBI emotional exhaustion score, the ISI score, and the IES-R Intrusion score but not the state anger score. Older age significantly increases the ISI score and the IES-R Intrusion score. Having undergone emergency training significantly decreases the ISI and IES-R Intrusion scores. Higher levels of perceived social support decrease the MBI emotional exhaustion score. Finally, we found that using avoidant coping significantly increases all considered scales, and the use of problem-focused and emotion-focused coping strategies significantly influences the MBI emotional exhaustion scale and state anger scale.

Figure 3 shows the estimated regression trees for the same outcomes.

For emotional exhaustion, the algorithm selected working conditions, avoidant coping strategies, and survey administration time as the best discriminating variables. Worse working conditions and higher use of avoidant coping strategies will lead to the highest predicted levels of emotional exhaustion, whereas the lowest outcome score is predicted for participants with better working conditions (<2.4). Moreover, worse working conditions and avoidant coping scores lower than 2.3, depending on the survey administration time, will lead to different predicted outcome scores, with higher predicted values at the time of the second survey.

For insomnia, the algorithm selected working conditions and the worry scale as the best discriminating variables. Worse working conditions and higher levels of worry will lead to the highest predicted levels of insomnia, whereas the lowest outcome score is predicted for participants with better working conditions (<2.4). Participants with worse working conditions and a level of worry lower than 3.4 will have predicted values lying in the middle and between those found in the previous two scenarios.

With reference to intrusion, working conditions, avoidant coping strategies, and scores on the worry scale are selected as the best splitting variables to identify different response profiles. Higher use of avoidant coping strategies (≥1.8) will lead to higher predicted levels of intrusion.

When avoidant coping strategies are lower than 1.8 and associated with better working conditions and lower levels of worry, the lowest levels of intrusion are predicted.

When avoidant coping strategies are lower than 1.8 and associated with worse working conditions (>2.7), participant profiles, with levels of intrusion lying in the middle, between the best and the worst scenarios, are identified.

For state anger, out of all the variables, only the worry scale plays a role in discriminating among participants' response profiles, with higher levels of worry leading to the highest predicted state anger score.

Estimated LME models for depersonalization and professional realization MBI subscales are reported in Table S3. In Table S4, LME models for the IES-R subscales of avoidance and hyperarousal are reported.

**Table 4.** Estimates (standard-errors) of the models for the MBI Emotional Exhaustion, ISI total score, IES-R Intrusion, and State Anger. Square root transformation was applied to the MBI Emotional Exhaustion scale, ISI total score, and IES-R Intrusion scale, while ordered quantile normalization was used for the State Anger scale.


\*\*\* *p* < 0.0001; \*\* *p* < 0.01; \* *p* < 0.05. Abbreviation: Ref, reference; CONDWORK, working conditions; Psych History, psychiatric history; MSPSS, Multidimensional Scale of Perceived Social Support, COPE, Brief Cope; MBI Emo Ex, Maslach Burnout Inventory-Emotional Exhaustion; ISI TOT, Insomnia Severity Index Total score; IES Intrusion, Impact of Event Scale-Intrusion.

**Figure 3.** Estimated regression trees for Emotional exhaustion, Insomnia, Intrusion, and State Anger. Square root transformation was applied to the MBI Emotional Exhaustion scale, ISI total score, and IES-R Intrusion scale, while ordered quantile normalization was used for the State Anger scale. Abbreviation: CONDWORK, working conditions.

#### **4. Discussion**

The present study is the second phase of a longitudinal study investigating the psychological consequences of the COVID-19 outbreak and their predictive factors in a sample of Italian healthcare workers. To our knowledge, this is one of the few longitudinal studies that monitored the mental health of healthcare workers during the COVID-19 outbreak [22,42,43].

In general, the scores obtained by our sample on the psychological scales during the second phase do not significantly differ from the scores obtained during the first phase. Overall, regardless of the category of healthcare workers, our sample only showed an increase in state anger levels. Furthermore, during the second wave, clerks seemed to experience higher levels of burnout symptoms.

These results are in line with the scant literature, showing a substantial invariance of psychiatric symptoms among healthcare workers throughout the epidemic [22,42,43].

However, working under chronic stress conditions and experiencing a poor quality of sleep both contribute to increased feelings of anger among healthcare workers during the COVID-19 outbreak [44,45]. Moreover, during the second wave of the pandemic, public opinion appears to have shifted, and healthcare workers now feel less support, appreciation, and trust from the general population, and they are viewed less favorably than in the past [22]. Consistently, during previous epidemics, feelings of anger seemed to be frequent among healthcare workers, caused by both long-term stressful working conditions and poor adherence to infection control guidelines, as well as rejection and stigma experienced from relatives and public opinion [46].

Furthermore, the increasing levels of burnout experienced by clerks during the second phase may be explained by the end of smart-working and the subsequent return to work in hospitals.

Concerning predictive factors, our results highlight that higher levels of worry, worse working conditions, a previous history of psychiatric illness, being a nurse, older age, and avoidant and emotion-focused coping strategies seem to be risk factors for healthcare workers' mental health. Conversely, high levels of perceived social support, the attendance of emergency training, and problem-focused coping strategies seem to be protective factors for healthcare workers' mental health.

Specifically, worse working conditions and worry about the infection represent risk factors for higher levels of depression, anxiety, stress, burnout, PTSD, insomnia, and state anger. Considering the regression trees, a score greater than or equal to 2.4 in working conditions is a cut-off for higher depression scores (for participants showing avoidant coping scores lower than 2.2), stress, burnout, and insomnia; a high worry score (≥3.9) is the only factor that distinguishes between the highest and lowest levels of state anger, whereas a score greater than or equal to 3.4 discriminates among participants with more extreme levels of anxiety and insomnia.

These findings are in line with the literature showing that difficult working environments, including poor supervision and organizational support, intense workloads, a lack of personal protective equipment or its continuous use for many hours, and fear and concern about becoming infected or infecting relatives or colleagues, increase the psychological distress' levels of healthcare workers [47–49].

Concerning demographic variables, older age seems a risk factor for higher levels of insomnia and intrusion symptoms. This result is consistent with the literature, which shows an association between increasing age and a physiological decline in sleep quality [50]. Additionally, the literature identifies older age as a risk factor for PTSD symptoms during the COVID-19 pandemic, probably due to the elderly being a high-risk category for infection and death [51,52].

Moreover, a previous history of psychiatric illness seems to be a risk factor for higher levels of depression, anxiety, stress, burnout, PTSD, and insomnia symptoms. The current literature highlights that a history of psychiatric symptoms seems to make healthcare workers more vulnerable and more likely to experience psychological distress during the COVID-19 emergency [53,54].

Furthermore, being a nurse seems to be a risk factor for higher levels of anxiety, PTSD intrusion symptoms, and insomnia. This result is consistent with the literature showing that being a nurse represents a risk factor for worse mental health, which is probably due to the longer time spent with patients in contact with their fears, suffering, and death [18,49,55,56].

Concerning protective factors, having undergone emergency training predicts low levels of depression, anxiety, stress, PTSD, and insomnia symptoms. This result is in line with the literature showing perceived adequacy of training as a protective factor for longterm psychiatric morbidity [57], post-traumatic stress [58,59], and burnout [58] in previous epidemics. This is understandable given that the goal of emergency training is to improve healthcare workers' skills and abilities to increase their sense of control and avoid being overwhelmed during a real emergency [60].

High levels of perceived social support from family and friends seem to be a protective factor for lower depression and burnout symptoms. In line with the literature, high perceived social support from family, colleagues, and friends is helpful to deal with workrelated stress and to increase self-confidence [18,47].

Concerning coping strategies, regression trees highlighted that higher avoidant coping scores lead to the highest scores in the depression, anxiety, stress, burnout, and PTSD subscales.

The detrimental role of avoidant coping strategies is consistent with current and past epidemic literature [17,61–63]. Avoidance strategies, such as denial or self-distraction, may be helpful for short periods of time to allow the person to continue with their tasks while also giving them some time to think. However, in the long-term, these coping strategies are dangerous for mental health, provoking dysfunctional detachment and distance from the problem, while changing neither the situation nor the associated psychological distress [64,65].

Conversely, the use of problem-focused coping strategies seems to be a protective factor for lower levels of depression, anxiety, stress, burnout, and state anger. This result is in accordance with the literature, showing that these strategies can increase feelings of autonomy and self-efficacy while reducing psychological distress [17,61,66]. These coping strategies may help in changing the meaning of the event and focusing on a specific goal, thereby increasing the perception of control, and avoiding being overwhelmed by the stressful situation [64,65].

Finally, the use of emotion-focused coping strategies seems to be a risk factor for higher levels of depression, stress, burnout, and state anger. The literature concerning emotion-focused coping strategies is contradictory [17,64], which is probably due to the heterogeneous nature of the strategies included in this subscale (e.g., use of emotional support and self-blaming). Moreover, during an emergency, problem-focused strategies appear more effective and adaptive than emotion-focused strategies [67]. Making difficult decisions in a stressful and extraordinary situation, with little knowledge about the disease and poor support from family and friends due to isolation, may be overwhelming if emotions are prioritized over the problem. Indeed, because of the governments' social restrictions policy in response to the COVID-19 emergency, a typically positive and protective coping strategy, such as the use of emotional support [66,68], could be a risk factor for poor mental health among healthcare workers [63,66,67,69].

In general, our findings align with other Mediterranean studies on the psychological impact of the COVID-19 pandemic on healthcare workers. Indeed, studies conducted in Italy, Greece, Portugal, Spain, and France highlighted a more resilient attitude among healthcare workers compared to studies conducted in China, the United Kingdom, and the Middle East [70]. Moreover, worse working conditions, higher levels of worry about the infection, and a previous history of psychiatric illness seem to be the main risk factors for poor mental health among Mediterranean healthcare workers [71–74].

#### *Limitations and Strengths*

Among limitations to be acknowledged, a self-selection bias may have occurred, as only participants experiencing low levels of psychological distress may have taken part in this research. Then, as only a subset of participants completed the assessment twice, the study sample may not be representative of both the healthcare worker population and the initial study sample.

However, this longitudinal study allowed us to monitor the mental health of healthcare workers over time and investigate relationships among variables to identify potential target risk and protective factors. Another strength of our study is the inclusion of healthcare workers from various hospitals located throughout Italy. Finally, the use of an online survey may have encouraged participants to reveal sensitive aspects of their work without worrying about confidentiality [75].

#### **5. Conclusions**

Healthcare workers seemed to be resilient in the face of the pandemic. Their mental health and general wellbeing is a complex issue that the government should monitor. In the current context, much can be offered, such as virtual clinics and remotely delivered psychological therapies and psychoeducation. However, it is also necessary to reduce mental health stigma, as physicians appear generally reluctant to disclose their problems, even when they are experiencing significant psychological distress [76]. The proposed data mining approach allowed us to derive classification rules and to identify risk and protective factors for healthcare workers' psychological wellbeing, which should be monitored during emergency situations. The proposed statistical methodology could provide

more insight into the psychological aspects which are leveraged when implementing training/intervention programs.

**Supplementary Materials:** The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jcm11092317/s1, Table S1: Descriptive statistics of investigated psychometric variables; Table S2: Observed ranges of investigated psychometric variables; Table S3: Estimates (standard-errors) of the models. Square root transformation was applied to MBI depersonalization scale, while scores on the MBI professional realization scale were raised to the power of two; Table S4: Estimates (standard-errors) of the models. Square root transformation was applied to scores on both IES-R Avoidance and Hyperarousal scales.

**Author Contributions:** Conceptualization, G.P., E.P. and V.E.D.M.; Data curation, F.C. and C.B.; Formal analysis, F.C. and C.B.; Investigation, G.P. and C.D.P.; Methodology, G.P.; Project administration, E.P. and V.E.D.M.; Supervision, F.M. (Fabio Madeddu) and V.E.D.M.; Visualization, G.P. and F.M. (Francesca Milano); Writing—original draft, G.P. and F.M. (Francesca Milano); Writing—review & editing, E.P., R.D.P. and V.E.D.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the University of Milano-Bicocca (protocol n. 0024531/20), the Ethics Committee of the IRCCS San Raffaele Scientific Institute (protocol n. 109/2020), and the Ethics Committee of the Parma Local Health Authority (protocol n. PG0019826\_2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data cannot be shared because participants did not provide written informed consent for it.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Fear of COVID-19 among Healthcare Workers: The Role of Neuroticism and Fearful Attachment**

**Alfonso Troisi 1,\*, Roberta Croce Nanni 2, Alessandra Riconi 3, Valeria Carola <sup>4</sup> and David Di Cave <sup>5</sup>**


**Abstract:** Fear of becoming infected is an important factor of the complex suite of emotional reactions triggered by the COVID-19 pandemic. Among healthcare workers (HWs), fear of infection can put at risk their psychological well-being and occupational efficiency. The aim of this study was to analyze the role of personality (i.e., the big five traits) and adult attachment in predicting levels of fear (as measured by the FCV-19S) in 101 HWs employed in a COVID-19 university hospital. The three significant predictors retained by the stepwise regression model were age (beta = 0.26, *t* = 2.89, *p* < 0.01), emotional stability (i.e., the inverse of neuroticism) (beta = −0.26, *t* = −2.89, *p* < 0.01), and fearful attachment (beta = 0.25, *t* = 2.75, *p* < 0.01). Older HWs with higher levels of neuroticism and fearful attachment reported more intense fear of COVID-19. Our results can be useful to identify vulnerable subgroups of HWs and to implement selective programs of prevention based on counseling and psychological support.

**Keywords:** COVID-19; fear; healthcare workers; neuroticism; insecure attachment

#### **1. Introduction**

COVID-19 has exposed healthcare workers (HWs) and their families to unprecedented levels of risk. While carrying out their duties, HWs face the occupational risk of being infected or unknowingly infecting others. Although HWs represent less than 3% of the population in the large majority of countries and less than 2% in almost all low- and middle-income countries, around 14% of COVID-19 cases reported to WHO is among HWs [1]. As of 1 June 2021, the number of coronavirus cases recorded among medical staff in Italy reached 135,054 [2].

Fear of becoming infected is an important factor of the complex suite of emotional reactions triggered by the COVID-19 pandemic [3]. There is a substantial difference between fear of infection and fear of noninfectious medical conditions (i.e., cancer, Alzheimer's disease, heart disease, stroke, and diabetes) that are feared the most in ordinary times [4]. Fear of these degenerative diseases is largely cognitive and prompted by cultural inputs because their etiology and pathogenesis are largely dependent on risk factors and life habits that are typical of modern environments (e.g., extended longevity, high calories diet, sedentary lifestyle, obesity, smoking, drinking alcohol, pollution, etc.). By contrast, fear of infection is deeply rooted in our emotional brain because it reflects a psychological adaptation evolved to minimize the exposure to a wide and varying array of pathogens that were relatively common throughout the evolutionary history of *Homo sapiens* [5,6].

Studies conducted on the general population during the current pandemic showed that fear of COVID-19 is more intense among women and tends to increase with age [7–10].

**Citation:** Troisi, A.; Nanni, R.C.; Riconi, A.; Carola, V.; Di Cave, D. Fear of COVID-19 among Healthcare Workers: The Role of Neuroticism and Fearful Attachment. *J. Clin. Med.* **2021**, *10*, 4358. https://doi.org/ 10.3390/jcm10194358

Academic Editors: Emmanuel Andrès and Icro Maremmani

Received: 3 September 2021 Accepted: 20 September 2021 Published: 24 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Research on the psychological reactions to previous pandemics and epidemics suggests that increased levels of fear of infection are a risk factor for developing depression, hypochondriasis, and post-traumatic stress disorder [3]. A study of 256 adults in the United States found that fear of COVID-19 predicted both depressive symptoms and generalized anxiety [11]. Moreover, fear was also found to be strongly associated with other indicators of emotional distress, such as suicidal ideation, alcohol and drug use, and extreme hopelessness [12].

These findings from the general population gain even greater importance when applied to HWs. Fear of COVID-19 is an expected emotional reaction among HWs because the increased morbidity risk due to their occupational role adds to the natural fear of infection. Previous studies conducted during the current pandemic have confirmed the relevant incidence of fear of infection among HWs and the negative impact on their psychological well-being [13–15]. However, among HWs, fear of infection can put at risk their psychological well-being as well as their occupational efficiency. For example, frontline nurses with greater fear of COVID-19 report less job satisfaction and higher intent to leave the profession [16], and fear of infection has been shown to be a predictor of burnout [17]. Given the strong correlation between fear of infection and the development of negative psychological and occupational outcomes, an enhanced understanding of which HWs are more vulnerable has implications for the treatment and prevention of a broad range of pathologies (e.g., depression and post-traumatic stress disorder) and for the optimization of their professional performance. In addition, the identification of subgroups of HWs with greater levels of fear of infection can allow the implementation of personalized psychological support and programs to facilitate open communication [18,19].

The aim of the present exploratory cross-sectional study was to analyze the role of the big five personality traits and adult attachment style in predicting levels of fear of COVID-19 in a convenience sample of HWs employed in a COVID-19 university hospital. The rationale inspiring the choice of these individual variables was the large body of evidence showing the consistent association between personality traits, attachment style, and vulnerability or resilience to different types of stressful events [20–23], including stress response to COVID-19 pandemic [24,25]. In pre-COVID times, Taylor [26] predicted that individuals high in neuroticism are vulnerable to elevated distress during pandemics because they are sensitive to stress and threats of infection. His prediction has been confirmed by studies conducted during the current pandemic in the general population in the United States [11], Canada [24], and Italy [27]. Similar to neuroticism, insecure attachment has also been linked with enhanced stress sensitivity, emotional dysregulation, and propensity to experience negative affectivity [28,29]. Based on these previous studies, we hypothesized that higher levels of neuroticism and insecure attachment correlated with greater fear of COVID-19.

#### **2. Materials and Methods**

#### *2.1. Participants*

Participants were 101 healthcare professionals working in a major university hospital that was converted into a COVID hospital in spring 2020. Participants were recruited in the period between June and August 2020 by snowball sampling. In Italy, the COVID-19 pandemic was particularly invasive during the period between March and late April, then decreased in both the number of infections and in the seriousness of the illness throughout the summer of 2020 [30]. The study was conducted when vaccination was not yet available. Thus, all HWs attending the hospital (including the participants of this study) were obliged to adhere to the same strict preventive measures to reduce the risk of infection, independently of their professional roles. Participants' mean age was 39.35 years (SD = 11.52, range: 21–70). In total, 64 were women and 67 were physicians. Other professional roles included nurses and laboratory technicians. Paper questionnaires were used to collect data. Participation was voluntary, and anonymity was guaranteed. To limit the selection biases of snowball sampling, we began with a set of initial informants that were as diverse as possible in terms of age, gender, and professional role followed by

respondent-driven sampling method (i.e., weighting the sample in order to compensate for the initial non-random selection). Written informed consent was obtained prior to participation. The study was approved by the Ethical Committee of the Department of Dynamic and Clinical Psychology, Sapienza, University of Rome (Prot. n. 0000453 and Prot. n. 0000112).

#### *2.2. Psychometric Measures*

#### 2.2.1. Fear of COVID-19

Ahorsu et al. [31] have recently developed a brief and valid scale (FCV-19S) to capture an individual's fear of COVID-19. The FCV-19S is a seven-item scale (e.g., "I am most afraid of COVID-19", "My heart races or palpitates when I think about getting COVID-19"). The participants are asked to indicate their level of agreement with the statements using a five-item Likert-type scale. Answers included "strongly disagree", "disagree", "neither agree nor disagree", "agree", and "strongly agree". The minimum score possible for each question is 1, and the maximum is 5. A total score is calculated by adding up each item score (ranging from 7 to 35). The higher the score, the greater is the fear of COVID-19. The Italian validation of the FCV-19S used in this study [32] showed robust psychometric properties (alpha = 0.82 and ICC = 0.72) and confirmed its stable unidimensional structure.

#### 2.2.2. Big Five Personality Traits

The Ten-Item Personality Inventory (TIPI) [33] is a short scale developed to measure personality traits according to the big five models (also known as the OCEAN model: openness to experience, conscientiousness, extraversion, agreeableness, neuroticism) in working or clinical settings in which assessment time is limited. The TIPI was developed using descriptors from other well-established big five instruments. Each of the 10 items is rated on a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree). The version used in this study was the revised Italian version (I-TIPI-R) [34], which showed adequate factor structure, test–retest reliability, self-observer agreement, and convergent and discriminative validity with the Big Five Inventory (BFI). In the I-TIPI-R, the scale measuring neuroticism is inverted and named emotional stability (i.e., people scoring low on emotional stability have high levels of neuroticism). When reporting the results, we refer to emotional stability. Yet, in the discussion, to facilitate the comparison of our findings with those of previous studies, we refer to neuroticism.

#### 2.2.3. Attachment Style

To measure adult attachment style, we used the Italian version [35] of the Relationship Questionnaire (RQ) [36]. The RQ is a single-item measure made up of four short paragraphs, each describing a prototypical attachment pattern as it applies in close adult peer relationships. Participants are asked to rate their degree of correspondence to each prototype on a 7-point scale. The four attachment patterns (i.e., secure, preoccupied, fearful, and dismissing) are defined in terms of two dimensions: anxiety (i.e., a strong need for care and attention from attachment figures coupled with a pervasive uncertainty about the willingness of attachment figures to respond to such needs) and avoidance (i.e., discomfort with psychological intimacy and the desire to maintain psychological independence). The preoccupied, fearful, and dismissing patterns reflect different forms of insecure attachment.

The reliability estimates for the RQ self-ratings are comparable to those for other short questionnaires assessing adult attachment styles (test–retest *r*'s around 0.50) [37]. The RQ shows convergent validity with interview ratings of adult attachment [36]. As for discriminant validity, several studies have demonstrated that the RQ explains individual differences in cognition, emotions, and behaviors even after controlling for the big five personality traits [38].

#### *2.3. Statistical Analysis*

Statistical analysis was performed on a personal computer using SPSS for Windows, version 25.0 (SPSS, Inc., Chicago, IL, USA). Spearman's rho was used to calculate bivariate correlations. Stepwise multiple regression analysis was used to identify significant predictors of infection fear. Although the primary aim of our study was to focus on personality traits and attachment as predictors of fear, in the first step of the stepwise multiple regression analysis, we included age, gender, and professional role to control for their possible confounding effects. There were no violations of the assumptions required by multiple regression. In particular, we used the Durbin–Watson statistic (value = 1.481) to check that the values of the residuals were independent, and variation inflation factors (VIF) scores (ranging from 1.012 to 1.033) and tolerances scores (ranging from 0.968 to 0.988) to check that there was no multicollinearity among the independent variables. The software G\*Power 3.1.9.7 was used to calculate the minimum sample size for multivariate analysis.

#### **3. Results**

Table 1 reports the psychometric data for the entire sample. High levels of fear of infection (FCV-19S score > 18) were reported by 18% of the participants.


**Table 1.** Psychometric scores for the entire sample (*n* = 101).

Legend: FCV-19S, Fear of COVID-19 scale; I-TIPI- R, Ten-Item Personality Inventory, Revised Italian Version; EXT, extraversion; AGR, agreeableness; CON, conscientiousness; EMS, emotional stability (the inverse of neuroticism); OPE, openness to experiences; RQ, Relationship Questionnaire.

Nonparametric bivariate correlations between the I-TIPI-R, the RQ, and the FCV-19S showed higher levels of fear of COVID-19 in participants scoring lower on emotional stability (rho = −0.32, *p* < 0.01) and higher on preoccupied attachment (rho = 0.28, *p* < 0.01) and fearful attachment (rho = 0.27, *p* < 0.01). A stepwise multiple regression was conducted to determine which individual variables were the best predictors of fear of COVID-19, as measured by the FCV-19S. At step 1 of the analysis, age, gender (women vs. men), and professional role (medical doctors vs. other HWs) were entered into the regression model to control for their possible confounding effects. At step 2, the big five dimensions (i.e., extraversion, agreeableness, conscientiousness, emotional stability, and openness to experiences), as measured by the I-TIPI-R scores, were entered into the regression model. In step 3, the RQ scores for the four attachment patterns (i.e., secure, preoccupied, fearful, and dismissing) were entered into the regression model.

The final model explained 24% (R2) of the variance in the FCV-19S scores. The three significant predictors retained by the final model were age (beta = 0.26, *t* = 2.89, *p* < 0.01), I-TIPI-R emotional stability (beta = −0.26, *t* = −2.89, *p* < 0.01), and RQ fearful attachment (beta = 0.25, *t* = 2.75, *p* < 0.01) (Table 2). Older HWs with lower levels of emotional stability (i.e., higher levels of neuroticism) and higher levels of fearful attachment reported more intense fear of COVID-19.


**Table 2.** Results of stepwise regression analysis with fear of COVID-19 (FCV-19S) as the dependent variable, and sociodemographic data (step 1), big five personality traits (I-TIPI-R) (step 2), and adult attachment style (RQ) (step 3) as independent variables.

Legend: FCV-19S, Fear of COVID-19 scale; I-TIPI- R, Ten-Item Personality Inventory, Revised Italian Version; EMS, emotional stability; RQ, Relationship Questionnaire.

#### **4. Discussion**

We found that older age predicted greater fear of infection. One possible explanation is that older HWs knew that they were at higher risk of critical COVID-19 symptoms [39]. In contrast, we found no correlations between gender, professional role, and fear of infection. It is likely that such missing correlations were idiosyncratic to our sample because a recent systematic review of 55 articles found that being a nurse and being female appeared to confer greater risk in terms of fear of infection [14].

We found that two personality traits, neuroticism and fearful attachment, were independent predictors of fear of infection. Neuroticism is a personality trait originally defined to include anxiety, emotional instability, worry, tension, and self-pity. This negative affectivity is accompanied by a pervasive perception that the world is a dangerous and threatening place, along with beliefs about one's inability to manage or cope with challenging events [40]. In accordance with previous studies [11,24,27], our findings confirm the prediction by Taylor [26] that individuals high in neuroticism are vulnerable to elevated distress during pandemics because they are sensitive to stress and threats of infection. The original contribution of our study is that neuroticism is associated with a specific facet of emotional distress (i.e., fear of infection) and that such an association can be found among HWs.

The pattern of insecure attachment that emerged as a significant predictor of fear of COVID-19 over and above the effect of neuroticism was fearful attachment. The finding that fearful attachment was a significant predictor independent of neuroticism was expected because previous studies showed that correlations between attachment patterns and the big five personality traits are weak [41]. The RQ paragraph describing fearful attachment reads as follows: "I am uncomfortable getting close to others. I want emotionally close relationships, but I find it difficult to trust others completely or to depend on them. I worry that I will be hurt if I allow myself to become too close to others".

We hypothesize that the psychological mechanisms linking fearful attachment with fear of infection are mainly related to dysfunctional coping strategies. In general, people with secure attachment tend to appraise stressful events in less threatening ways and to appraise themselves as able to cope effectively. In contrast, insecure attachment (including the fearful pattern) is associated with distress-intensifying appraisals (i.e., appraising threats as extreme and coping resources as deficient). Among people with fearful attachment, there is an additional factor that may increase fear of infection. They have a pervasive uncertainty about the willingness of significant others to respond to their needs for emotional support. Their typical discomfort with psychological intimacy and preference for emotional distance preclude self-disclosure, promote social avoidance, and can also work against attendance at support programs.

The few studies that have analyzed the relationship between adult attachment style and emotional reaction to the COVID-19 pandemic do not allow the assessment of the validity of our hypothesis. The theoretical paper by Rajkumar [42] makes no specific prediction about the pattern of insecure attachment that is expected to correlate with increased stress sensitivity to the COVID-19 outbreak. The report by Moccia et al. [43] on the Italian general population used the Attachment Style Questionnaire (ASQ) which, unlike the RQ used in the present study, does not measure the dimension of fearful attachment. Finally, the study by Lozano and Fraley [44] focused on sentinel behavior (only indirectly related to fear of infection) and found that people higher in attachment avoidance were less likely to protect themselves and protect others. We need further research to ascertain how different patterns of insecure attachment are associated with stress and coping during the current pandemic.

Based on the findings of the present study, neuroticism and fearful attachment may be viewed as vulnerability traits because of their link with fear of infection and the associated higher risk of developing stress-related psychiatric conditions. However, it is worth noting that fear of infection evolved as an adaptation to reduce the risk of contracting deadly diseases and that bold personality traits and lack of fear can lead to underestimating the risk of COVID-19 infection and eluding containment measures [45,46]. It is likely that the most adaptive emotional response to infection risk is to experience intermediate levels of fear (neither too high nor too low).

The main limitations of this study are related to the sampling method. By using snowball sampling, we had no information on how many HWs were approached and declined to participate. In addition, although the inclusion of HWs with different occupations and working in different wards provided a more complete picture of the impact of the pandemic, the limited number of participants and the variety of their duties limit the generalizability of our findings.

#### **5. Conclusions**

If confirmed by future studies based on larger samples, our results are relevant for policymakers and mental health professionals engaged to preserve HWs' well-being and professional efficiency. The psychometric battery used in this study includes brief selfreport scales that are easy to complete and useful to predict which HWs will be more inclined to react fearfully toward the COVID-19 outbreak. The identification of vulnerable subgroups would allow the selective implementation of prevention programs based on counseling and psychological support [18,19].

**Author Contributions:** Conceptualization, A.T. and D.D.C.; methodology, R.C.N.; software, V.C.; validation, A.T. and R.C.N.; formal analysis, A.T. and V.C.; investigation, A.R.; resources, A.T. and D.D.C.; data curation, A.R.; original draft preparation, A.T.; review and editing; R.C.N., V.C. and D.D.C.; funding acquisition, not applicable. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethical Committee of the Department of Dynamic and Clinical Psychology, Sapienza, the University of Rome (Prot. n. 0000453 and Prot. n. 0000112).

**Informed Consent Statement:** Informed consent was obtained from all individual participants included in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Can Psychological Empowerment Prevent Emotional Disorders in Presence of Fear of COVID-19 in Health Workers? A Cross-Sectional Validation Study**

**Marta Llorente-Alonso 1,2,\* , Cristina García-Ael <sup>3</sup> , Gabriela Topa <sup>3</sup> , María Luisa Sanz-Muñoz 2, Irene Muñoz-Alcalde <sup>2</sup> and Beatriz Cortés-Abejer <sup>2</sup>**


**Abstract:** The global emergency produced by COVID-19 has been a turning point for health organizations. Healthcare professionals have been exposed to high levels of stress and workload. Close contact with infected patients and the infectious capacity of COVID-19 mean that this group is especially vulnerable to contagion. In various countries, the Fear of COVID-19 Scale has been shown to be a fast and reliable tool. Early detection of fear complements clinical efforts to prevent emotional disorders. Thus, concepts focused on positive occupational health, such as Job Crafting or psychological empowerment (PE), have been examined as a tool to prevent mental health problems at work. In this work, we intended to adapt and validate the 7-item Fear of COVID-19 Scale in health workers (*N* = 194). The interpretation of the measurement model indicates adequate values of internal consistency reliability, and convergent and discriminant validity. The overall goodness of fit of the model was also adequate. The structural model indicates that the implementation of job crafting measures in health services leads to workers' greater PE. High levels of anxiety and depression prevent health professionals from psychologically detaching from work. In turn, PE can reduce the emotional disorders caused by the fear of COVID-19.

**Keywords:** psychological empowerment; fear of COVID-19; collaborative crafting; job crafting; emotional disorders; psychological detachment

#### **1. Introduction**

The health emergency caused by the SARS-COV-2 virus has led to serious physical and psychological problems worldwide [1,2]. In Spain, the "Center for Sociological Research" (October 2020) [3] reports that 79.3% of the Spanish population considers that the pandemic has affected the emotional health of the entire population. This survey was conducted on 2861 people and it values the effects and consequences of the Coronavirus in the Spanish population. Of the participants, 50.6% expressed anxiety during the health crisis, 29.3% felt depressed, and 57.5% were afraid of getting sick. In addition, subsequent surveys have shown that the pandemic has changed the way a large part of Spanish society thinks. In fact, 12.3% of the respondents consider that they live in fear, unease, or apprehension of the pandemic (CIS, December 2020) [4].

The high contagion rate and increased mortality of the SARS-COV-2 virus compared with other respiratory pathologies [5] have caused feelings of fear and uncertainty about the future in part of the population [4]. Fear consists of anguish over a real or imaginary

**Citation:** Llorente-Alonso, M.; García-Ael, C.; Topa, G.; Sanz-Muñoz, M.L.; Muñoz-Alcalde, I.; Cortés-Abejer, B. Can Psychological Empowerment Prevent Emotional Disorders in Presence of Fear of COVID-19 in Health Workers? A Cross-Sectional Validation Study. *J. Clin. Med.* **2021**, *10*, 1614. https:// doi.org/10.3390/jcm10081614

Academic Editor: Alfonso Troisi

Received: 21 March 2021 Accepted: 7 April 2021 Published: 10 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

risk or harm [6]. Extreme fear has even led to cases of suicide in people not diagnosed with COVID-19 [7,8]. Gunnell et al. suggest that the COVID-19 pandemic can trigger profound effects on mental health, and that suicide rates may increase, given the increase in the number of suicides in previous epidemics (in the USA during the 1918–1919 flu and among older people, in Hong Kong during the 2003 Severe Acute Respiratory Syndrome (SARS) epidemic) [9]. Therefore, fear assessment can be an important mechanism for preventing emotional or mental health disorders.

The general objective of this study is to adapt and validate the Fear of COVID-19 Scale of Ahorsu et al. [10], made up of 7 items, in health workers. This research also comprises several specific objectives. First, we aim to examine the role that fear of COVID-19 plays in emotional disorders and, in turn, in psychological detachment. Second, we will assess whether positive occupational health, through collaborative crafting and psychological empowerment (PE), can act as a relevant factor in preventing emotional disorders and lead to better recovery experiences after the workday. Finally, we intend to determine whether the fear of COVID-19 has any negative effect on PE and, in turn, whether it affects how healthcare professionals' distance themselves from work.

#### *1.1. Current Situation of the Fear of COVID-19*

Ahorsu et al. developed a brief instrument to detect fear of COVID-19 in the general population [10]. As these authors explained, fear is directly associated with the transmission rate and morbidity. This scale has been adapted to other cultures in general population [11–19]. However, no psychometric adaptations and assessments of the Fear of COVID-19 Scale in health workers were found in the literature. The assessment of fear of COVID-19 levels in different sociodemographic groups is relevant for the implementation of specific prevention programs [8]. We also consider it especially important to know whether there is fear in professional sectors that are in direct contact with the virus and that have a high risk of exposure contagion. Early detection of fear of COVID-19 can act as an alarm signal to health workers to prevent the development of emotional disorders and, therefore, be able to recover from stressful work situations.

On the other hand, a recent meta-analysis showed that health professionals working to combat COVID-19 are more affected by psychiatric disorders, sleep disorders, stress, and indirect trauma than other occupational groups [20]. Pappa et al. Additionally suggested that a significant proportion of health workers have experienced mood and sleep disorders [21]. De Brier et al. stated that the level of exposure to the disease and fear for health were significantly associated with worse mental health outcomes [2]. Additionally, finding oneself in stressful, scary situations can lead to emotional disorders that, in turn, can prevent proper psychological detachment from work. This detachment is part of a process called recovery, through which people stop facing a demanding situation to regain energy to continue and renew the resources invested in that situation [22]. Stressful experiences are considered the opposite process of recovery [22]. Sonnentag et al. reported that employees who felt exhausted found it more difficult to disconnect psychologically from work [23]. They also stressed the importance of time pressure in the increase of the association between exhaustion and lower psychological detachment. In this sense, the duration of stressful experiences and feelings of fear during the COVID-19 pandemic can contribute to health workers' being unable to disconnect from work, thus preventing adequate recovery experiences.

#### *1.2. Job Crafting and Psychological Empowerment*

Job crafting is defined as employee's proactive behavior aiming to modify the relational, cognitive, or task limits to shape or redesign a job [24]. Most research has studied job crafting on an individual level. However, Leana et al. consider job crafting not only to consist of an individual employee's activity, but of the fact that workers participate in similar work processes, relate to each other, and experience common events [25]. This refers to collaborative crafting, whereby employees can team up and decide how to modify

tasks to achieve their goals. Moreover, in certain professions, such as in health care or education, it is difficult to adapt individual work due to the high degree of interdependence between groups [26]. Research linking job crafting to PE is still scarce. However, several authors have shown that job crafting is strongly related to PE [27,28]. Specifically, Harbridge conducted a study on registered nurses and highlighted job crafting as an important predictor of PE [28]. Demerouti proposed that job crafting may lead to greater motivation, performance, or engagement [29].

On the other hand, PE consists of a subjective, cognitive, and attitudinal process through which the individual feels effective, competent, and authorized to perform tasks. Spreitzer considers that PE reflects an active orientation and self-perception of the ability to shape one's working role [30]. While PE is not synonymous with intrinsic motivation, it can be considered a predictor of it [31], and therefore a motivational factor. The four components of PE are a proximal cause of intrinsic task motivation and satisfaction [31]. Schermuly and Meyer showed that PE leads to less emotional fatigue and depression [32]. It also strongly influences the degree of work stress experienced by workers [33]. Petersen et al. found evidence that self-efficacy is amendable to change and exerts an effect on protective behavior. The effects of fear were small among those who felt efficacious [34].

On the other hand, Ghosh et al. found that psychological detachment acts as a moderator between intrinsic motivation and engagement [35]. Employees who feel motivated and psychologically detached from work in their free time are also more creative [35].

Thirdly, to test the last specific objective, we aim to determine whether fear of COVID-19 has any negative effect on psychological detachment, through the mediation of PE. There is no literature linking the fear of COVID-19 to organizational variables such as PE. However, other types of fear, such as fear of success, have been linked to self-efficacy and intrinsic motivation. Specifically, both of them can be used to mitigate the potentially adverse effects of this type of fear [36]. In this sense, in a situation of fear of COVID-19, workers are expected to have reduced PE and be incapable of activating the cognitive processes that enable them to perform tasks effectively and competently. In this way, they will not be able to distance themselves from their work in their free time.

**Hypothesis 1.** *Emotional disorders will play a mediating role in the relationship between the fear of COVID-19 and psychological detachment*.

**Hypothesis 2.** *PE will mediate the relationship between collaborative crafting and emotional disorders*.

**Hypothesis 3.** *Emotional disorders will mediate between PE and psychological detachment*.

**Hypothesis 4.** *PE will mediate the relationship between fear of COVID-19 and the psychological detachment of health professionals*.

#### **2. Materials and Methods**

#### *2.1. Participants and Procedure*

The sample was made up of a total of 194 workers from the health centers of the province of Soria belonging to the Health Service of Castilla y León (Sacyl) in Spain. Permission was requested from the organization's Ethics Committee, and the questionnaires were forwarded to a total of 1056 workers, obtaining a response rate of 18.37%. Data collection took place in July 2020 via email. Through this means, participants accessed a link in Google Forms by which, after providing their informed consent, access was given to fill out the questionnaire. The final sample consisted of 162 women and 32 men, with an average age of 45.94 (SD = 12.39). Of the sample, 28.4% had been diagnosed with COVID-19 or had been detected to have antibodies after a test. Concerning their employment status, 50.5% of the participants were nurses or specialist nurses, 26.3% were specialized graduates, and 12.4% were assistant nursing technicians. Hence, most of the participants were healthcare professionals. Additionally, 45.9% of the sample considered that their tasks or

work activities during the COVID-19 pandemic had changed compared with those they had performed previously. (see Table 1).

**Table 1.** Demographics.


Note: *N,* Sample Size; M, Mean; SD, Standard Deviation; TCAE, Technician in Auxiliary Nursing Care.

#### *2.2. Instrument*

To assess fear of COVID-19, the Fear of COVID-19 Scale was used [10]. As a preliminary step, the questionnaire was translated into Spanish by a blind back-translation process [37]. Two methodology experts compared the original scale and the final Spanish version [38]. The questionnaire is a one-dimensional scale consisting of seven items measured on a Likert-like response scale ranging from 1 (Totally Disagree) to 5 (Totally Agree). The internal consistency index of the scale (α = 0.90) was higher than that obtained for the original scale (α = 0.82) [10].

As mentioned above, PE refers to several cognitive processes that modify the subjective self-perception, by which the worker feels intrinsically motivated and effective to perform tasks [30]. In this study, PE was evaluated using the Psychological Empowerment Scale [30], adapted to Spanish [39] consisting of four subscales: (a) Meaning (three items, e.g., "My work activities have been personally valuable"); (b) Competence (three items, e.g., "I trust my ability to get the job done"); (c) Self-determination (three items, e.g., "I have had the autonomy to determine how to do my job"); and (d) Impact (three items, e.g., "I've had enough influence on what was going on in my work"). The Likert-type response scale ranged from 1 (Totally Disagree) to 5 (Totally Agree). High scores indicate greater PE. The reliability coefficient was high (α = 0.84).

To assess participants' anxiety and depression, we used the Hospital Scale of Anxiety and Depression (HADS) [40], consisting of two subscales of seven items each. Items in the Anxiety subscale aim to detect generalized anxiety ("I feel tense or nervous"), and the subscale of Depression primarily assesses the state of anhedonia ("I feel like I'm slowing down every day"). The response range ranged from 1 (Totally Disagree) to 5 (Totally Agree). The internal consistency in this study was high for both the total scale (α = 0.92), the Anxiety subscale (α = 0.91), and the Depression subscale (α = 0.87).

To evaluate collaborative crafting, the two-dimensional Spanish-validated Job Crafting Questionnaire [25,26] was used. In the study, we used the 6-item Collaborative Crafting Subscale, of which only five items were used (e.g., "You work together with your peers to introduce new approaches to improving your work"). Specifically, the item that refers to celebrations or events at work was removed, as it was deemed inappropriate in the pandemic situation. The response scale ranged from 1 (Totally Disagree) to 5 (Totally Agree). The reliability coefficient was high (α = 0.91).

To measure psychological detachment, we used the Recovery ExperienceQuestionnaire [41] validated in Spanish [22], which presents four subdimensions: Psychological Detachment, Relaxation, Challenge-Seeking, and Control. For the study, three items of the Psychological Detachment subscale were used (e.g., "After work, I can disconnect) measured on a Likerttype scale ranging from 1 (Totally disagree) to 5 (Totally agree). The reliability coefficient was high (α = 0.95).

Finally, the following demographic data were collected: age, gender, professional category, organizational rank, job tenure in the current contract, diagnosis of Covid, place of work, type of contract, and modification of tasks or activities during the pandemic.

#### *2.3. Data Analysis*

To evaluate the descriptive statistics and bivariate correlations of the study variables, we used the IBM SPSS Statistics 26 program [42]. The data was then analyzed using a structural equations model (SEM) based on the variance, with the method of partial least squares (PLS) [43]. This procedure allows simultaneously assessing the reliability and validity of the measures of the theoretical construct (measurement model) and estimating the relationships between constructs (structural model) [44]. A new approach called consistent PLS was used because, if the common factor model is retained, consistent PLS or covariance-based SEM should be the first choice of researchers over traditional PLS [45]. Additionally, when comparing PLS-SEM with CB-SEM, PLS can handle small sample sizes and discard the assumption of normality, so it is recommended for social science research [46]. Thus, in the present investigation the method of choice is PLS-SEM, which is considered a more robust method when the sample size is reduced. The data were analyzed with the statistical software SmartPLS (v.3.3.2) [47].

#### **3. Results**

The mean, standard deviations, and correlations of the study variables are presented in Table 2.


Note: *N* = 302. \* *p* < 0.05, \*\* *p* < 0.01. - indicates a blank space.

Before analyzing the data for the adaptation of the Fear to COVID-19 scale, we examined skewness and kurtosis to verify that the data did not stray excessively from a normal distribution. PLS-SEM is a non-parametric statistical method [48]. Although the data are not required to have a normal distribution, if extremely non-normal data were present, the standard errors obtained through bootstrapping could be inflated, and the probability of finding significant relationships between variables would decrease [48]. These authors recommend examining two distribution measures, skewness and kurtosis. In most indicators, values between −0.99 and +0.98 were obtained, so we decided not to eliminate any of them, as there was no problem of non-normality.

The interpretation of the PLS model comprises three phases: (a) evaluation of the global model, (b) measurement model (external model), and (c) structural model (internal model). We also performed an analysis of the invariance of the measurement model to determine whether similar measures were obtained in different groups.

#### *3.1. Global Model*

To evaluate the global model, the Standardized Root Mean Square Residual (SRMR) parameter was used, which measures the difference between the observed correlation matrix and the correlation matrix implied by the model. Hu and Bentler proposed values of SRMR < 0.08 to achieve a good fit of the data [49]. Ringle proposed a more flexible option (SRMR < 0.10) [50]. In this study, we obtained an SRMR of 0.077 both in the saturated and estimated models.

#### *3.2. Measurement Model*

The evaluation of the reflective measurement models includes composite reliability (to assess internal consistency), the reliability of the individual indicator, and the meanvariance extracted (AVE) to assess convergent validity [48]. In addition, such measurement models also assess discriminant validity.

First, the internal consistency reliability of the Fear of COVID-19 Scale was tested. Cronbach's alpha coefficient obtained a high value of 0.90. Composite reliability and reliability rho\_A obtained values of 0.89 and 0.90, respectively. These composite reliability values are considered satisfactory. Values greater than 0.95 are not adequate because they could suggest that all indicators are measuring the same phenomenon [48]. The rest of the variables also obtained high levels of internal consistency.

Second, convergent validity was assessed, examining the loadings or simple correlations of indicators with their construct. The external loadings of the indicator should be greater than 0.707 [51]. In the case of our Fear of COVID-19 Scale, items 5, 6, and 7 exceeded the value of 0.707, whereas the rest obtained values between 0.58 and 0.67. Indicators with loadings between 0.40 and 0.70 should be removed if there is an increase in composite reliability [52]. After performing the analyses without these items, the composite reliability only decreased from 0.89 to 0.84, so we decided to maintain them. For all other variables, several analyses were performed, eliminating indicators with values between 0.40 and 0.70. In the case of emotional disorders, there was an increase in composite reliability (0.92) and AVE (0.50), so item 4 was removed from the Anxiety scale and item 4 from the Depression scale. On the PE scale, items 1 and 2 (Meaning subscale), 5 (Competency subscale), 7 (Self-Determination subscale), and 10 (Impact subscale) were also removed. Thus, a high composite reliability value (0.84) was obtained.

Convergent validity was evaluated through AVE. The Fear of COVID-19 Scale obtained a value of 0.55, above the recommended value of 0.50. This indicates that the construct explains more than half of the variance of its indicators. The rest of the variables, except for PE, achieved values greater than 0.50.

Finally, discriminant validity was assessed through the cross-loads, following the criterion of Fornell and Larcker and the Heterotrait-Monotrait ratio (HTMT). The load of the indicators on the Fear of COVID-19 Scale was higher than their cross-loads with other constructs, indicating discriminant validity. Concerning the Fornell criterion, the

square root of the AVE of the Fear of COVID-19 (0.75) was not higher than the correlation between fear and emotional disorders (0.87). However, given the absence of discrepancies between the two criteria, we valued the HTMT. The Fornell and Larcker criterion is not appropriate when the loadings of the indicators of the constructs differ only slightly [48]. HTMT should be lower than 0.85. All the variables had lower values, so discriminant validity was achieved (see Table 3) [53].


**Table 3.** Measurement model: loads, construct reliability, and convergent validity.

Note: λ = Loadings. CR = Composite reliability. Rho\_A = Dijkstra-Henseler's rho (ρA). AVE = Average variance extracted. A = Cronbach's alpha. Items removed: Psychological empowerment 1, 2, 5, 7, and 10; Emotional disorders 4 and 11.

#### *3.3. Structural Model*

Having verified that the measures of the constructs are reliable and valid, we valued the structural model. First, we evaluated the collinearity of the structural model, using the variance inflation factor (VIF) whose value must be 5 or less [52]. The results showed that all VIF values were below 5, indicating the absence of collinearity between predictors. Specifically, the VIF value between EP and Collaborative Crafting, and between EP and fear of COVID-19 was 1.01. Between emotional disorders and PE, and emotional disorders and fear, the VIF was 1.05, whereas between PE and psychological detachment, it was 1.12. The highest values of VIF were 4.53 and 4.30, between detachment and emotional disorders, and detachment and fear, respectively.

The algebraic sign, magnitude, and statistical significance of the path coefficients were also evaluated. The signs of the path coefficients matched the hypotheses raised. The highest values of the standardized beta coefficients (β) were between fear of COVID-19 and emotional disorders (β = 0.85, *p* < 0.001) and between emotional disorders and psychological detachment (β = −0.82, *p* < 0.001).

Bootstrapping was used for consistent PLS (10,000 subsamples) to assess the meaning of the path coefficients. The relationships between Collaborative Crafting and PE (*t* = 7.90, *p* < 0.001), between PE and emotional disorders (*t* = 2.28, *p* = 0.023), between fear of COVID-19 and emotional disorders (*t* = 24.01, *p* < 0.001), and between emotional disorders and psychological detachment (*t* = 3.95, *p* < 0.001) were significant. In contrast, fear of COVID-19 was not directly related to psychological detachment (*t* = 1.24, *p* = 0.21), or to PE (*t* = 1.90, *p* = 0.05).

On the other hand, we calculated the indirect effects between the variables. The indirect effect of fear of COVID-19 on psychological detachment (β = −0.69, *p* < 0.001), through the mediation of emotional disorders, was significant. The indirect effect of collaborative crafting on emotional disorders was also significant, through the mediation of PE (β = −0.06, *p* = 0.033). Additionally, the mediation of emotional disorders in the relationship between PE and psychological detachment was significant (β = 0.10, *p* < 0.001). These results support Hypotheses 1, 2, and 3. However, Hypothesis 4 could not be confirmed. In Figure 1, we specify the structural model. In Table 4, we specify the total effects of the model.

**Table 4.** Total effects.


With regard to the coefficient of determination, the model explained 40.9% of the variance of psychological detachment, 77.9% of the variance of emotional disorders, and 30.6% of PE.

#### *3.4. MICOM Model: Analysis of the Invariance of the Measurement Model*

Measurement invariance, or measure equivalence, means that group differences in model estimates are not due to the different content or meaning of the latent variables between groups [48], but as to whether, under different conditions of observation and study of the phenomena, the measurement operations produce measurements of the same attribute [54]. Henseler et al. developed a procedure for calculating the measurement invariance of composite models (MICOM) [55]. This method is developed in three hierarchically interrelated stages. As Henseler et al. explain, variance-based SEM techniques model latent variables as composite variables, so this procedure is considered appropriate for assessing common factor models, such as the one presented in this study [55].

**Figure 1.** Structural model. \* *p* < 0.05. \*\*\* *p* < 0.001.

### 3.4.1. First Stage: Configuration Invariance

This determines whether a composite has been specified equally in all groups and whether it emerges as a one-dimensional entity in the same nomological network for all groups [55]. In this study, the initial qualitative evaluation ensures that the same indicators are used in each measurement model, the data are processed in the same way, and the algorithm is also configured identically.

#### 3.4.2. Second Stage: Composite Invariance

It analyzes whether a composite is formed in the same way in all groups [55]. To evaluate composite invariance, we performed a permutation algorithm with PLS (5000 permutations), in which the selected groups were, on the one hand, the participants whose work tasks or activities had been changed during the COVID-19 pandemic, and on the other hand, those whose activity was similar to before the health emergency. To test composite invariance, the original correlation must be greater than or equal to the 5% quantile.


two groups.

3.4.3. Third Stage: Evaluation of Equal Means and Variances of the Composite Variables

In Table 5, MICOM results show that the composite scores did not differ between the

In the third step, we determined whether the original differences in means and variances were between 2.5% and 97.5%, and complete invariance was established. If only one of these variances fell between 2.5% and 97.5%, then partial invariance of means and variances would be considered. In Table 6, the results of the third MICOM stage suggest equality of means and variances.



Therefore, after observing that all three stages were met, we performed multigroup analysis based on group data [48]. We ran the multigroup analysis in PLS and analyzed the non-parametric PLS-MGA approach, which compares each bootstrap estimate of a given parameter between each of the groups [48]. Again, we performed the calculation with 5000 subsamples. All values were nonsignificant, indicating that no bootstrap estimate of a parameter differed between groups. As can be seen in Table 7, our results suggest that there are no significant differences between the two groups (changes in work activities vs. no changes in work activities).

**Table 7.** PLS-MGA (PLS-Multigroup Analysis results).


Note: The contrasting groups were: Participants with changes in their tasks or work activities during the COVID-19 pandemic vs. participants with no changes in their tasks.

#### **4. Discussion**

The main objective of this work was to adapt and validate the Fear of COVID-19 Scale [10], composed of seven items, in health workers. The results show that it is a valid and reliable scale and that its structure consists of a single factor, as other authors have suggested [10,12,13,15,16,18]. First, the evaluation of the overall model based on the SRMR criterion showed adequate goodness of fit, thus indicating that the model is probably appropriate. Second, by rating the external or measurement model, we conclude that both the composite reliability of the scale and the internal consistency reliability are adequate. Appropriate values of convergent and discriminant validity were also obtained. Other authors have found similar results in terms of composite reliability [10], internal consistency [13], convergent validity [10], and discriminant validity [56]. Finally, to ensure that the differences between groups are not due to the content or meaning of the latent variables, we tested the measurement invariance of composite models (MICOM). The results showed that there were no differences between the groups of participants (with and without changes in their tasks or activities), indicating the existence of measurement equivalence between the two groups.

The evaluation of the structural model determines whether the postulated assumptions are met. Concerning the first hypothesis, this research was intended to assess the role of fear of COVID-19 in the development of anxious-depressive disorders, and in health workers' ability to distance themselves psychologically from their work. The results have shown that fear of COVID-19 is a strong predictor of emotional disorders, such that health professionals who score higher in fear of COVID-19 are more likely to develop anxiety and/or depression. Ahorsu et al. also found significant relationships between fear and anxiety or depression, suggesting that people with severe fear may have these emotional disorders [10]. Additionally, in this study, we found that fear determines health workers' inability to distance themselves from work, but only if they suffer some degree of comorbid anxiety and depression. This result is in line with the results of Sonnentag et al., because health workers who feel more exhausted find it more difficult to detach psychologically from work [23].

Concerning the second hypothesis, which sought to test the ability of positive occupational health to prevent mental health problems at work, we found that collaborative crafting behaviors lead to health-care workers' greater PE. In the first months of the pandemic, uncertainty due to the lack of knowledge of the coronavirus disease and the hospital collapse determined the need to work collaboratively and to modify the usual tasks of healthcare professionals (e.g., nurses or technicians who changed their jobs to work on Covid floors or doctors with non-COVID-19 specialties working together with internists or intensive care doctors). These collaborative crafting behaviors have proven to be important predictors of PE. Other authors have shown that PE leads to less emotional fatigue and depression [32]. In this study, we highlight the importance of PE as a mediator in the relationship between collaborative crafting and emotional disorders. These results provide evidence to the literature about the importance of job crafting interventions to enable employees to proactively create a motivating work environment and improve their well-being [57].

Next, we discuss the confirmation of the third hypothesis. We expected that the most empowered workers could psychologically distance more from work than those who are not empowered. In this relationship, we observed the importance of having no emotional disorders for health workers to recover after their working day, as anxiety and depression problems acted as a total mediator. Hochwälder and Brucefors also found that greater PE at work generally corresponds with fewer health problems [58].

Concerning the fourth hypothesis, which tested the role of fear in detachment through the mediation of PE, we found no significant relationships. Faced with a situation of fear of COVID-19, workers' PE is not reduced, but other organizational variables, such as job crafting, activate the cognitive processes that enable them to perform their tasks.

Finally, we consider that one of the strengths of this research has been to relate positive organizational psychology to the prevention of basic emotions such as fear to decrease emotional disorders. Thus, we link lower fear scores with the preventive role of PE (after the development of collaborative crafting interventions) to develop fewer mental health problems and recover psychologically from work.

#### *4.1. Limitations*

Firstly, and as the main limitation, we highlight the impossibility of establishing relationships of direct causality. The findings should be interpreted with caution due to the cross-sectional nature of the data and the lack of longitudinal research on COVID-19 fear and emotional disorders. Secondly, the data are self-reported, so they should be treated with caution. Additionally, we consider a possible threat to external validity. When health workers responded to the questionnaire, they could react to the pandemic situation and respond according to the social norm, as a function of what is expected of the group of health professionals. For example, if a healthcare provider verbalizes that he or she is afraid of COVID-19, this may be criticized from the point of view of normative influence. Thirdly, as a threat to internal validity, we highlight the motivation to answer the questionnaire, or self-selection bias. The participants in this study may have had different expectations than those who chose not to participate. In future research focused on health workers, it would be advisable to assess the participants' degree of social desirability.

On the other hand, this study was carried out in a single health institution and in the province with the highest incidence of seroprevalence in Spain [59]. This could pose a problem for the generalization of the results to the rest of health professionals in this country. In addition, another important limitation is the small size of the sample that can cause low representativeness.

In addition, this study was carried out in July 2020, after three months of quarantine in Spain, and Soria was the most affected province, with a 14% seroprevalence of SARS-COV-2 [59]. However, the study was conducted at a time when the hospital situation was adequate. There was no hospital overload or collapse. In the questionnaire, professionals were instructed to evaluate the previous three months. However, these previous circumstances could have altered the answers.

Finally, it should be noted that these results were obtained in an exceptional situation of a global pandemic. Health workers were exposed to contagion due to lack of personal protective equipment, uncertainty, fear of infecting family members, helplessness from lack of knowledge about the disease, etc. Collaborative crafting was analyzed in a situation where health workers performed as a team and collaboratively more than ever. For this reason, we must be cautious and consider the exceptionality of the situation as a limitation of this study and analyze these organizational variables (job crafting and PE) in times of non-pandemic normality.

#### *4.2. Future Lines of Research and Practical Implications*

This study has important practical implications. We emphasize the importance of early detection of fear of COVID-19 to prevent emotional disorders, as well as to improve recovery experiences after the workday. Additionally, the Fear of COVID-19 Scale was adapted [10] for health workers, so it can be appropriately used to analyze this emotion in them. We also know the importance of promoting positive occupational health to prevent these anxious-depressive disorders. Through practices that enhance collaborative crafting, health workers can be empowered and thereby, reduce mental health problems.

We found no research papers in the literature that relate positive organizational psychology and the psychology of emotions to mental health problems and recovery experiences. Coelho et al. recommended that future research on fear of COVID-19 and anxiety should focus on pointing to protective and risk factors of psychological well-being [60]. Other authors consider that leaders need to have the appropriate communication that provides up-to-date information and encourages individual empowerment to support their

staff [61]. As future lines of research, more studies are proposed that investigate these models and analyze them over time through longitudinal research.

**Author Contributions:** Writing—Original draft, M.L.-A., C.G.-A. and G.T.; Writing—review and editing, M.L.-A., C.G.-A. and G.T. Resources, M.L.S.-M., I.M.-A., and B.C.-A. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of Burgos and Soria health area of Castile and Leon Health Service (Protocol Code REF CEim 2349 and date of approval 12 June 2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **How Did COVID-19 Affect Suicidality? Data from a Multicentric Study in Lombardy**

**Camilla Gesi 1, Federico Grasso 2,3,\* , Filippo Dragogna 1, Marco Vercesi 2,3, Silvia Paletta 2, Pierluigi Politi <sup>3</sup> , Claudio Mencacci <sup>1</sup> and Giancarlo Cerveri <sup>2</sup>**


**Abstract:** The aim of the study was to describe the characteristics of subjects accessing the emergency rooms for suicidal behavior during the first epidemic wave of COVID-19 in three Emergency Departments (EDs) in Lombardy (Italy). A retrospective chart review was conducted for the period 8 March–3 June 2020, and during the same time frame in 2019. For all subjects accessing for suicidality, socio-demographic and clinical data were collected and compared between the two years. The proportion of subjects accessing for suicidality was significantly higher in 2020 than in 2019 (13.0 vs. 17.2%, *p* = 0.03). No differences between the two years were found for sex, triage priority level, history of substance abuse, factor triggering suicidality and discharge diagnosis. During 2020 a greater proportion of subjects did not show any mental disorders and were psychotropic drug-free. Women were more likely than men to receive inpatient psychiatric treatment, while men were more likely to be discharged with a diagnosis of acute alcohol/drug intoxication. Our study provides hints for managing suicidal behaviors during the still ongoing emergency and may be primary ground for further studies on suicidality in the course of or after massive infectious outbreaks.

**Keywords:** COVID-19; SARS-CoV-2; suicidal behavior; first emergency care; multicentric; Lombardy

#### **1. Introduction**

About 800,000 people worldwide die every year due to suicide and an even greater number attempts suicide or engages in self-injuring behaviors [1]. Up to 90% of suicides around the world are associated with mental disorders and substance abuse, including harmful use of alcohol [2]. However, a broad variety of environmental factors also contribute to suicidal behavior, many of which originate from the concurrent cultural, social and economic context [1]. Suicidal spectrum behaviors include a broad variety of manifestations, from suicidal thoughts and plans, to suicidal self-injuring and suicide attempts, to completed suicide [3,4]. Despite the fact that most subjects with suicidal thoughts do not attempt suicide, suicidal ideation may often precede suicide attempts. However, according to the ideation-to-action framework, the development of suicidal ideation and the progression from ideation to suicide attempts are distinct phenomena with distinct explanations and predictors [5].

Individuals with suicidality often are referred to Emergency Departments (EDs), and EDs also frequently provide care for people with other risk factors for suicide, such as serious mental illness, substance use, and chronic pain. Every month, the number of visits to EDs prompted by suicidality is considerable, accounting for about 4% of accesses yearly in the US [6,7]. In addition, suicidal behaviors may not only represent the overt reason for the access, but also emerge as part of a broader constellation of psychiatric symptoms or

**Citation:** Gesi, C.; Grasso, F.; Dragogna, F.; Vercesi, M.; Paletta, S.; Politi, P.; Mencacci, C.; Cerveri, G. How Did COVID-19 Affect Suicidality? Data from a Multicentric Study in Lombardy. *J. Clin. Med.* **2021**, *10*, 2410. https://doi.org/10.3390/ jcm10112410

Academic Editor: Laurent Boyer

Received: 25 March 2021 Accepted: 26 May 2021 Published: 29 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

be hidden by other complaints, so that the ED consultation itself may end up unraveling a current suicide risk [8,9]. Therefore, the emergency room of the EDs is an especially privileged observatory for the whole spectrum of suicidal behaviors [10].

The effect of natural and man-made disasters on suicidality has been evaluated in previous studies. Despite some inconsistent report, most data indicate a significant impact, either immediate or delayed, of disasters on suicide behaviors [11–14]. This is likely due to the detrimental effect of collective emergencies on mental health and psychosocial wellbeing, as well as to the socio-economic upheaval brought about by a range of consequences of disasters, such as the death or injury of family members, the loss of employment and properties, and the disruption of community cohesion and support [15,16].

Data focusing on the effect of massive infectious outbreaks on suicide behaviors are sparse, consistently with the relatively rare occurrence of epidemics in the last decades. Only poor evidence is available about the Spanish Flu, infecting 500 million people between 1918 and 1919 and narratively associated with a high risk of enacting suicidal behaviors among survivors [17]. During the more recent outbreak of SARS in Honk-Hong in 2003, rates of suicide were shown to rise compared to the previous year among elderly females, but not among elderly males or younger age groups. A recent nationwide cohort study conducted in Taiwan found significant higher rates of suicide, anxiety, depression, sleepand trauma-related disorders among SARS survivors compared to non-affected subjects in the five years following the 2003 outbreak [18]. As for referral to EDs for suicidality, a study evaluating accesses to the emergency room in a SARS-dedicated hospital in northern Taiwan during 2003 SARS outbreak found an increased number of suicide attempts from drug overdoses during peak- versus pre-epidemic stages, despite the difference not being statistically significant [19].

The ongoing pandemic due to Sars-Cov-2 has obvious similarities with previous outbreaks, but also bears a few differences. After beginning in China in 2019, the COVID-19 has rapidly spread on a global scale with multiple epidemic waves in 2020. At the time of writing this paper, about 100 million people [20] have contracted the virus globally and more than two million have died. Besides the massive toll in terms of mortality, the health-related and social costs of COVID-19 are thought to be as much as significant. The impact on mental health is expected to be especially severe as the coronavirus epidemic has shown to enhance several relevant risk factors for mental illness, spanning from the loss of community life to widespread poverty, from unemployment to disruption of critical mental health and social services. Noteworthily, the compulsory quarantine enforced for preventing the propagation of the virus led to a sharp increase in social isolation and to a significant decrease in social support, which are among the most important risk factors for any kind of suicidal behavior [21,22]. Conversely, although data on deaths by suicide during the lockdown are still scarce, the first months of the pandemic might have been characterized by a lower suicide mortality rate [23]. For instance, a decrease of suicidal behaviors was observed in France during the strict lockdown. This decrease may be explained by several factors: the so-called "pulling-together effect", observed in times of national tragedies, the work adaptation (reduced working hours and work-fromhome policies), the subsidies limiting financial distress, the reduced access to illegal drugs. However, the absolute number of violent or severe suicide attempts remained relatively stable [24].

Italy was the first western country struck from the coronavirus pandemic. The first hotbed of contagion emerged at the end of February 2020 in Codogno, in the province of Lodi, about forty kilometers southeast of Milan, leading quickly to a quarantine setting enforced by law and to the rapid spread of fear. Besides the closure of schools, bars, restaurants and shops, the ED of Codogno was also temporarily closed to new admissions, and most patients were diverted to the neighboring hospitals of Pavia and Lodi. At the beginning of March, as the coronavirus reached the metropolitan area of Milan and started circulating across northern Italy, the entire Lombardy was placed on lockdown.

The main objective of the study was to describe the sociodemographic and clinical features of subjects accessing the psychiatric emergency service for suicidality during the first Sars-CoV-2 epidemic wave in three EDs in Lombardy, and to compare rates and characteristics of accesses between 8 March and 3 June 2020 to those occurring during the same period in 2019. We included accesses prompted by the whole spectrum of suicidal behaviors [4] (i.e., suicidal thoughts, suicidal self-injuring, suicide attempts, completed suicide) hereafter referred to as "suicidality" throughout the manuscript. The three EDs were chosen as differently hit by the epidemic, according to their distance from the first epicenter of the outbreak. In particular, Lodi-Codogno was the first center struck by Sars-CoV-2 epidemic in Italy and very severe restrictions were soon enforced in the attempt to prevent further spreading of the contagion. Pavia was involved in a second time in the epidemic wave, while the overflow of patients from Codogno was diverted to its hospital. Only in a later time the Sars-CoV-2 wave reached the metropolitan area of Milan, as the epidemic was already spreading across the whole of Lombardy.

#### **2. Materials and Methods**

#### *2.1. Materials and Methods*

A retrospective observational study was conducted at three EDs (Lodi-Codogno, San Matteo-Pavia, Fatebenefratelli-Milan) in Lombardy. The ED of Lodi-Codogno, where the first indigenous case of COVID-19 in Italy was confirmed, comprises two emergency rooms located in southern Lombardy, with a catchment area of about 230,000 inhabitants. The ED of San Matteo Hospital in Pavia, located 38 Km west of Lodi-Codogno and usually covering a district of about 550,000 inhabitants, during the first outbreak served to handle the overflow from the neighboring hospitals of Codogno and Lodi, which rapidly became overwhelmed. The ED of Fatebenefratelli Hospital is located in the metropolitan area of Milan (30 and 40 Km north of Lodi and Pavia respectively) serving a district of about 400,000 residents and more than one million professionals commuting daily from suburbs and surrounding areas. All three EDs offer psychiatric emergency service 24/7 and provide treatment for a range of psychiatric conditions.

#### *2.2. Study Population and Data Collection*

A retrospective chart review of medical records was carried out at the three EDs using hospitals' computer databases of emergency rooms reports. All subjects (i) older than 18 years and (ii) accessing the three EDs for suicidality between 8 March and 3 June 2020 were selected for inclusion in the analyses. In addition, subjects meeting the inclusion criteria throughout the same period of 2019 were included as a comparison group. The total number of subjects referring to the EDs and going through a psychiatric evaluation during the two periods was also annotated. The flow-chart illustrating the recruitment process is shown in Figure 1.

Data were extracted anonymously including sex, age, nationality (Italian vs. other), marital, cohabitation and occupational status, usual care provider (private/public Mental Health/Addiction Service), history of alcohol and substance use, phase of access (8 March– 4 May vs. 5 May–3 June), type of suicidality (suicidal thoughts, suicide attempt, selfinjuring, drug ingestion), presence of triggering conflicts, triage priority level (high vs. low), psychopharmacological treatment prescribed before/during/after ED consultation, discharge diagnosis (anxiety/mood/psychotic/personality disorder/no mental disorderharmful substance use), and admission to the inpatient psychiatric service. The period between 8 March and 4 May 2020, when the number of COVID-19 cases rose and the lock-down measures were implemented, was designated as the peak epidemic stage (phase 1), while the period between 5 May and 3 June, as the outbreak began to subside and the measures of lock-down were removed, was defined as the late-epidemic stage (phase 2). The study was performed in accordance with the principles of the Declaration of Helsinki regarding medical research in humans and it satisfied local research ethical requirements. In particular, the privacy of research subjects and the confidentiality of

their personal information were protected by anonymization of all collected data. As a retrospective, non-interventional, low-risk study, the institutional review boards at each participating site approved the study protocol and the local ethic committee was notified before study initiation.

**Figure 1.** Recruitment Flow-chart.

#### *2.3. Statistical Analyses*

Demographic and clinical characteristics of patients accessing the EDs for suicidality in 2019 and 2020, respectively, were compared using a *t*-test for continuous variables and Chi-square test for categorical variables. The number of accesses for suicidality out of the total number of ED visits were compared between the two years using Chi-square test. Additional analyses were conducted within each year group to compare subjects based on sex, phase of the outbreak (phase 1/phase 2) and site of enrollment. Chi-square test with Odd Ratios (OR) values and 95% confidence intervals (CI) were used to find significant predictors of admission to the psychiatric inpatient unit only for the year 2020. A *p* value of

less than 0.05 was considered statistically significant. All statistical analyses were carried out using SPSS, version 26 (IBM, Armonk, NY, USA) [25].

#### **3. Results**

#### *3.1. Characteristics of Patients Accessing the ED for Suicidality during the First Wave of COVID-19 in 2020*

Demographic characteristics of patients referred to the ED for suicidality between March 8th and June 3rd 2020 are displayed in Table 1. Overall, 94 subjects accessed the ED for suicidality (22.3% in Lodi-Codogno, 52.1% in Pavia, 25.5% Milan) with 58.5% accessing in Phase 1 and 41.5% in Phase 2. Most of them (77.7%) were Italian, with no differences in the percentage of foreigners/Italians accessing the ED during Phases 1 and 2. Across the three months, the majority of subjects were unemployed and unmarried. As shown in Table 2, half of the subjects did not usually refer to any mental health/addiction service, and the majority (52.1%) were admitted to the ED after an episode of intentional prescription drug ingestion. Overall, 12.8% accessed the ED for current suicide attempt; the distribution of suicide attempts vs. any other suicidality feature did not significantly differ between phase 1 and phase 2 in the overall sample (9.1% of suicide attempts in phase 1 vs. 17.9 in phase 2; Chi-square = 1.608, *p* = 0.205) nor considering each center separately. At the end of ED consultation, the vast majority, 87.2%, received a mental disorder diagnosis, while the remaining 12.8% were discharged with no psychiatric diagnosis/substance harmful use. Thirty subjects (31.9%) were admitted to the psychiatric inpatient unit. Among a range of possible risk factors (sex, taking antidepressants/anxiolytics/mood stabilizers/antipsychotics, suicide attempt vs. others, having/not having a psychiatric diagnosis, self-referred detrimental impact of COVID-19) only female sex (39.7% vs. 19.4%, OR = 2.7, IC 1.0–7.2) and having a psychiatric diagnosis (36.6% vs. 0%, OR = 0.81, IC 0.72–0.91) were shown to be significant risk for being admitted to the psychiatric inpatient unit. Females were also more likely to present with an episode of intentional prescription drug ingestion (*p* = 0.043), while males were more likely to show acute alcohol/drug intoxication; no differences were found in the prevalence of substance abuse. The majority of males did not usually refer to any mental health/addiction service (63.9%), while the majority of women (58.6%) did (chi = 4.502, *p* = 0.034). A significantly higher percentage of men than women were discharged with antipsychotic (22.2% vs. 6.9%, chi = 4.685, *p* = 0.030) and antidepressant (30.6% vs. 13.8%, chi = 3.870, *p* = 0.49) prescription. At the time of discharge from ED, the majority of females (65.9%) were diagnosed with a psychiatric disorder, while the majority of men (66.7%) were diagnosed with harmful substance use/no psychiatric disorder (4.685, *p* = 0.030).


**Table 1.** Demographic characteristics of the study sample. Values presented in parentheses are per cent, unless otherwise indicated.


**Table 2.** Clinical characteristics of the study sample.


<sup>+</sup> MHS: Mental Helath Service. \* all the features listed relate to the aim of ending own life. 'Suicide attempt' refer to a potentially life-threatening behavior; 'self-injuring' and 'drug ingestion' refer to self-harming acts with a declared suicidal intent but lacking life-threatening potential.

#### *3.2. Comparisons between 2020 and 2019*

A total number of 777 subjects were referred to the ED and went through PES evaluation in the three centers between 1st March and 31st May in 2019. Of those subjects, 101 (13.0%) did so for suicidality. In the same period of 2020, 546 patients overall accessed the

ED and underwent psychiatric consultation, 94 (17.2%) for suicidality, with a statistically significant difference between the two years (Chi-Square: 4.5386; *p* = 0.03). Considering every single center, the difference was not significant for the center of Lodi (14.6% suicide in 2019 vs. 15.4% in 2020, chi: 0.046; *p* = 0.83), nor for Milan (12.8 suicide in 2019 vs. 15.8 in 2020, chi = 0.639, *p* = 0.42), but was so in Pavia (11.6% suicide in 2019 vs. 19.0% in 2020, chi = 5.934; *p* = 0.02). Comparisons of clinical characteristics of patients accessing in 2019 and 2020, respectively, are presented in Table 2. No differences were found for sex, triage priority level and history of substance abuse between the two years. No differences were found about factors triggering suicidality (conflicts with family members vs. anxiety/exacerbation of psychopathology) in the overall sample, nor considering each center separately. However, the difference was significant considering only Phase 2, with 71.8% of subjects accessing in Phase 2 doing so for anxiety/exacerbation of psychopathology, and 42.9% in the same period of the previous year (Chi = 6.345; *p* = 0.012). No differences were found in the percentage of subjects who were admitted to the psychiatric inpatient unit between 2019 and 2020 in the overall sample, nor considering each center or each period separately. No differences were found in the prevalence of each diagnostic group (psychotic disorders, mood disorders, anxiety disorders, personality disorders) as discharge diagnoses between 2019 and 2020. While dichotomizing discharge diagnoses between psychopathological or no mental disorders/harmful substance use, a difference close to significance was found between 2019 and 2020 with 5% of subjects with no mental disorders in 2019 and 12.8% in 2020 (Chi = 3.737, *p* = 0.050).

As shown in Table 3, a significant difference was found between 2019 and 2020 regarding the percentage of subjects treated with any psychotropic drug at the moment of ED consultation, with a minority of patients (26.7%) who were psychotropic drug-free in 2019 compared to 40.4% in 2020 (Chi 4.108, *p* = 0.043). No differences were found between 2019 and 2020 in the type of treatment used before/prescribed after ED consultation, except for patients accessing in 2019 having greater likelihood of being treated with anxiolytic drugs before ED consultation compared to those accessing in 2020 (33.0% vs. 50.5% Chi = 6.130, *p* = 0.013).

**Table 3.** Treatment characteristics of the study sample.


#### **4. Discussion**

The main aim of the study was to compare the characteristics of patients accessing the ED for suicidality during the first wave of COVID-19 in 2020 with those accessing in the

same period of 2019 in three Italian EDs differently affected by the SARS-CoV2 outbreak (Codogno, the first struck by the epidemic wave, Pavia and Milan). First, out of all the people referring to the psychiatric services of the EDs, the proportion of consultations due to suicidality was significantly higher in 2020 that in 2019. This finding is in line with previous data suggesting that massive events may trigger suicidality and contribute to the existing literature about the direct and indirect consequences of the pandemic [12–14]. Analyses separately carried out for each center further indicated that the difference in the rate of psychiatric consultations due to suicidality in the two years was actually significant in the center of Pavia but not in the centers of Lodi-Codogno and Milan. The absence of significance for the center of Lodi-Codogno may appear in contrast with early exposure to the COVID-19 of this area. Codogno was indeed the first epicenter of the outbreak in Italy and its population was subjected for an especially long time to severe restrictions, social isolation, and risk of infection. However, there is the possibility that a number of people needing psychiatric emergency consultation during the first epidemic wave was shifted to the nearby Department of Pavia, in which the number of accesses for psychiatric consultation was in fact especially high compared to the other two centers and significantly higher than in 2019. While no differences were found regarding the reasons triggering suicidality in the phase 1, a greater proportion of suicidal behaviors during the phase 2 was caused by relapsing psychopathology—instead of being triggered by interpersonal problems—compared with the same period of 2019. One hypothesis could be that the stress suffered during the epidemic phase contributed to starting the process of relapse that became fully manifested only during the post-epidemic phase [26]. On the other hand, it is also possible that feelings of uncertainty and the fear of contagion withheld people with relapsing symptoms from seeking for help in the ED during the peak epidemic phase, with some sort of rebound in patients with relapsing psychopathology as soon as the contagion started to subside in the post-epidemic phase [26]. This interpretation could also be in line with the decrease in the overall number of psychiatric emergency consultations from 2019 to 2020, confirming that the epidemic wave led to fear and avoidance of the ED to some extent. This also stands as a caveat to the increased suicidality in 2020, as the total number of accesses for suicidality remained substantially constant from 2019 to 2020, while the increase percentage of accesses for suicidality was mostly due to a drop in the amount of psychiatric emergency consultations for other reasons.

The overall severity of suicidal gestures was not more severe in 2020 than in 2019. In fact, no differences between years were found in the percentage of psychiatric emergency visits leading to inpatient treatment admission nor in the distribution of different features of suicidality. Dichotomizing discharge diagnoses between psychopathological or no psychopathological, a difference very close to significance by year was found, with a higher proportion of subjects with no mental disorders accessing for suicidality in 2020 than in 2019. Although not significant, this result suggests the need of further investigation and might indicate a large impact of COVID-19 on psychological wellbeing and suicidal behaviors, severely involving not only people with preexisting psychiatric disorders but also a broader group of people somehow vulnerable to the multifaced effect of the pandemic [27,28]. Such a hypothesis is corroborated by the finding of a greater proportion of subjects free from psychopharmacological treatments accessed in 2020 compared with 2019 and by the greater likelihood of being already treated with anxyolitics among subjects seeking consultation in 2019. Interestingly, a recent study hypothesized a mediating role of HPA activity and inflammation between social isolation and suicidality, providing a possible neurobiological framework to the increased suicidality observed in our study [29].

Some noteworthy features also emerged from cross-sex comparisons within the 2020 year. While in 2019 sex was not shown to affect the probability of being admitted to a psychiatric inpatient unit, in 2020 women were more likely than men to receive inpatient treatment as a result of psychiatric emergency consultation. Moreover, women were more likely to be already in treatment in outpatient mental health or addiction services at the time of consultation and to receive a mental disorder diagnosis at the time of

discharge. On the other hand, men were mostly not referring to any community-based service and were more likely to seek for help in the ED after suicidal behavior arising from substance use unrelated to any mental disorder. Overall, females looked especially prone to enact suicidal gestures in the context of a preexisting mental disorder, while men appeared likely to show a suicidal behavior mostly independently from mental illness, highlighting the role of environmental risk factors for suicidality in the context of the COVID-19 pandemic. For example, one hypothesis could be that men are more vulnerable to react with externalizing behaviors to psychological stress and less likely to seek psychological and social support [30]. Additionally, the economic and employment strains following the pandemic could especially affect men as far as they are bound to endorse the traditional role of family breadwinners [31]. Indeed, unemployment has been shown to contribute differently to the risk of suicide among men and women [32,33].

We acknowledge some limitations of this study. First, results would be more reliable if comparisons were made not only with 2019 but with multiple years preceding the pandemic. Second, the sample is relatively small, and data were brought from few EDs in Northern Italy. Despite involving three departments differently hit by the pandemic in Lombardy, results cannot be assumed to be representative of the whole region. Third, the study has a retrospective design and data were not collected for the purpose of research. Further, no distinction about the violent/not violent nature of suicidal behaviors was provided. Lastly, as cases were recruited based on ED records, we could not include data about completed suicide, lacking information about the extreme end of suicidal spectrum both in 2019 and 2020.

#### **5. Conclusions**

Our study suggests that the proportion of subjects accessing the ED for suicidalty during the first wave of the COVID-19 epidemic was significantly higher in 2020 compared to the same period of 2019. Although this could be due to an overall drop of ED accesses during the first peak epidemic phase, we also found that a greater percentage of subjects enacting suicidal behaviors during this period was psychotropic drug-free compared to 2019, suggesting that suicidality might not be directly related to a pre-existing treated mental disorder. Our study provides some hints to be used by clinicians managing suicidality during the ongoing emergency and may be of primary ground for further studies on suicidality arising during large-scale health emergencies. Further investigations in later phases of the ongoing pandemic will help to elucidate the overall impact of such emergency on suicidal spectrum behaviors.

**Author Contributions:** Conceptualization, C.G., F.G., F.D., M.V., S.P., P.P., C.M. and G.C.; methodology, F.G., F.D. and P.P.; validation, P.P., C.M. and G.C.; formal analysis, C.G.; investigation, F.G. and F.D.; writing—original draft preparation, C.G. and F.G.; writing—review and editing, C.G., F.G., M.V. and S.P.; supervision, G.C. and C.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of Milano Area 1—ASST FBF SACCO—ASST Fatebenefratelli Sacco Milano. Prot. N 0023880, date of approval 25/05/2021.

**Informed Consent Statement:** Not applicable. Data were collected retrospectively and anonymously from the emergency services database.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to their collection from the hospital database. Data are not available in a publicly accessible repository.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Psychological Interventions for Children with Autism during the COVID-19 Pandemic through a Remote Behavioral Skills Training Program**

**Flavia Marino 1,†, Paola Chilà 1,†, Chiara Failla 1, Roberta Minutoli 1, Noemi Vetrano 1, Claudia Luraschi 1, Cristina Carrozza 1, Elisa Leonardi 1, Mario Busà 1, Sara Genovese 1, Rosa Musotto 1, Alfio Puglisi 1, Antonino Andrea Arnao 1, Giuliana Cardella 1, Francesca Isabella Famà 1, Gaspare Cusimano 1, David Vagni <sup>1</sup> , Pio Martines 2, Giovanna Mendolia 2, Gennaro Tartarisco <sup>1</sup> , Antonio Cerasa 1,3,4,\*, Liliana Ruta 1,\* and Giovanni Pioggia <sup>1</sup>**


† These authors contributed equally to the study.

**Abstract:** COVID-19 has impacted negatively on the mental health of children with autism spectrum disorder (ASD), as well as on their parents. Remote health services are a sustainable approach to behavior management interventions and to giving caregivers emotional support in several clinical domains. During the COVID-19 pandemic, we investigated the feasibility of a web-based behavioral skills training (BST) program for 16 parents and their children with ASD at home. The BST parent training package was tailored to each different specific behavioral disorder that characterizes children with ASD. After training, we found a significant reduction in the frequency of all the targeted behavioral disorders, as well as an improvement in psychological distress and the perception of the severity of ASD-related symptoms in parents. Our data confirm the efficacy of remote health care systems in the management of behavioral disorders of children with ASD, as well as of their parents during the COVID-19 pandemic.

**Keywords:** autism; telehealth; behavioral skills training; parent training

#### **1. Introduction**

The COVID-19 pandemic has had a profound impact on the health of children with autistic spectrum disorders (ASD) and their parents. In fact, these children have been identified as part of a group at higher risk of medical complications and social distress [1] from COVID-19. Moreover, parents of children with ASD often have trouble accessing behavioral services for their children [2]. This common condition has been significantly exacerbated by the COVID-19-related containment measures, with the risk of increasing lifelong impairments and comorbidities related to this disorder. More accessible interventions are urgently needed to support the families of children with ASD by promoting new techniques that can also facilitate clinical and supportive interventions at a distance.

**Citation:** Marino, F.; Chilà, P.; Failla, C.; Minutoli, R.; Vetrano, N.; Luraschi, C.; Carrozza, C.; Leonardi, E.; Busà, M.; Genovese, S.; et al. Psychological Interventions for Children with Autism during the COVID-19 Pandemic through a Remote Behavioral Skills Training Program. *J. Clin. Med.* **2022**, *11*, 1194. https://doi.org/10.3390/jcm11051194

Academic Editor: Nuri B. Farber

Received: 18 December 2021 Accepted: 21 February 2022 Published: 23 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Some studies have discussed the possible consequences the COVID-19 pandemic could have on individuals and their parents. Research suggests that people with autism are particularly vulnerable to conditions of prolonged isolation, as they have to adapt to new routines which can negatively affect their progress [3,4], and an online survey found that people with ASD exhibited an increase in problem behavior during lockdown periods [5].

To manage the spread of COVID-19, since March 2020, an international effort has been made to adapt all healthcare services to work "remotely by default" [6]. Telehealth has therefore become of primary interest, continually evolving to encompass new approaches, new clinical demands, and digital developments.

In several clinical domains, remote digital health services foster patient–clinician relationships, ensure continuity in treatments, and support families by simultaneously reducing burdens on health systems [7]. Technology-mediated care includes live video calls, monitoring health status by medical devices, e-mail, audio, and instant messaging. This service connects clinicians virtually with patients or caregivers by removing any physical distance. Remote health services have also been used to support families of children with ASD. Hyman et al. [8] clarified the definitions of various health services by establishing the validity of this method in a similar way to traditional face-to-face clinical settings, although evidence-based protocols are lacking.

In this neuropsychiatric domain, remote health services may be more effective than inperson meetings. In fact, children with ASD have shown a positive sensitivity to the novelty of this method and the physical separation may allow clinicians to perform more naturalistic observations of the family setting [9]. Technologies related to remote health services also offer a cost-effective solution for extending the reach of behavioral interventions to families who do not live near a qualified provider, thus addressing inequalities in access to health care [10–12]. This method also has the advantage of easily training parents to be effective behavior-analytic teachers of their children [13]. Through technologymediated care systems, it is thus possible to promote new kinds of online parent coaching by providing "anytime, anywhere" assistance to a parent who has access to the Internet during the pandemic era [14].

In this study, our aim was to demonstrate the feasibility and efficacy of a new webbased training approach aimed at reducing the frequency of targeted behavioral disorders in ASD children and improving parents' reported sense of competence related to childrelated behavioral dysfunctions during the lockdown. Our online parent program was focused on behavioral skills training (BST). This is a teaching procedure that involves the use of instructions, feedback, modeling, and rehearsal [15]. This method has been used to develop new ways to support children with ASD with specific problem behaviors. The intervention typically includes modeling and prompting procedures [16] in the context of function-based treatments [17], and functional analyses [18]. The effectiveness of the BST procedure has been widely demonstrated in several clinical contexts [15–18], although its implementation in remote health service programs has been poorly investigated.

#### **2. Materials and Methods**

#### *2.1. Enrollment*

Twenty-eight parents of young children with autism were enrolled in the study. Parents were recruited and tested at the clinical facilities of the Institute for Biomedical Research and Innovation of the National Research Council of Italy (IRIB-CNR) in Messina and at the Centre for Autism Spectrum Disorders, Child Neuropsychiatry Unit, Provincial Health Agency of Trapani, Italy.

Inclusion criteria for the parents were: (1) being a native Italian speaker; (2) being biological parents; (3) having a home internet connection; and (4) being able to use web-based and telehealth tools. The inclusion criteria for ASD children were as follows: (1) being over 3 years of age; (2) clinical diagnosis of ASD based on DSM-5 criteria by a licensed neuropsychiatrist with the support of the Autism Diagnostic Observation Schedule, second edition (ADOS-2, Module 3); (3) a verbal development and performance quotient greater than 45; (4) no hearing, visual or physical disability preventing participation; and (5) not being on psychopharmacological treatment. All participants had had a previous diagnosis which was further confirmed through the evaluation and consent of experienced professionals from the research group (i.e., a child neuropsychiatrist and a clinical psychologist). Data were collected from October 2020 to May 2021 during the second and third waves of the pandemic in Italy.

#### *2.2. Ethics*

All subjects provided informed consent for inclusion prior to their participation in the study. The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Committee of the Research Ethics and Bioethics Committee (http://www.cnr.it/ethics, accessed on 17 December 2021) of the National Research Council of Italy (CNR) (Prot. No. CNR-AMMCEN 54444/2018 01/08/2018) and by the Ethics Committee Palermo 1 (http://www.policlinico.pa.it/, accessed on 17 December 2021) of Azienda Ospedaliera Universitaria Policlinico Paolo Giaccone Palermo (report No. 10/2020– 25/11/2020).

#### *2.3. Study Design*

The objective of the study was to assess the effectiveness of tele-assisted BST for parents in reducing the frequency of specific problem behaviors in children with ASD. To this end, given the wide behavioral variability and the absence of a control group a between-group design was not feasible and a multiple single subject design was also not feasible due to the wide range of techniques used and difficulty in assessing protocol adherence, fidelity and gathering data on a daily basis. Therefore, we applied a repeated measures design with four-time steps, using the frequency of undesired behavior as the outcome variable.

#### *2.4. Treatment*

The protocol was conducted through a web platform [G-Suite; Google LLC; Mountain view, CA, USA)] that gave access to video-conferencing tools. The Teleconsultation Center at IRIB-CNR in Messina had one teleconferencing workstation, with a basic webcam and headset, while parents at home were equipped with a tablet which was used to receive parent training and to record sessions for subsequent data coding and analysis (see Sections 2.5 and 2.7).

Parents and therapists briefly met via videoconferencing or telephone before and after each training session as needed in order to review the procedures, prepare the room and materials, discuss the results obtained, and to plan for the subsequent week's session.

While both parents were expected to participate in the training, video-recording of the target behaviour and behavioural data gathering was only mandatory for one parent.

#### *2.5. Protocol Phases and Parent Training Procedures*

The experimental protocol consisted of a total of eight phases (see Table 1) divided into 13 sessions/meetings lasting 45 min each, in accordance with the ABA procedure, with the participation of both parents and children. The first four sessions were of pure training, eight central sessions were for active treatment, and the last session was for feedback. The final target was to match the treatment to the identified functions of problem behavior.

During Phase 0, an online meeting was arranged with families, and the therapists informed the parents of the study's aims and procedures. Parent-report questionnaires were given to parents and gathered at the beginning of phase 1.

In Phase 1, the first operative meeting was scheduled. The therapist collected information on the individual child's behaviors and discussed them with the parents. At the end of the meeting, the therapist provided them with an individualized frequency checklist in order to have an operationalized definition of the problem behaviors reported by the parents. Identified behaviour must have been observable, measurable, and repeatable.

In the following week, the parents observed and monitored their child's problem behaviors whilst completing the behavior frequency checklist.

**Table 1.** Protocol structure.


In Phase 2, the second operative meeting was scheduled. The therapist and the parents discussed the results reported in the checklist and selected the target behaviour. During the online meeting, the therapist established a first baseline related to the problem behavior. The behavior selected for treatment was the more frequently observed behavior that reduced learning opportunities or social inclusion or was either physically or emotionally harmful for the child or their family. At the end of the meeting, the parents were instructed to record videos of the target behavior during the following week.

In Phase 3, the therapist, during the third meeting, compiled the Antecedent-Behavior-Consequence (ABC) worksheet using information collected through the recorded videos. Parents received live coaching on functional analysis procedures in order to understand and identify the antecedents [events that precede] and consequences [events that follow] of the problem behavior using the recorded videos as examples. The therapist provided instructions on filling out the Antecedent-Behavior-Consequence (ABC) worksheet. At the end of the meeting, the use of short videos helped parents to interpret the problem behavior and to complete the ABC worksheet with the assistance of the therapist. The therapist then asked the parents to record the data on the ABC worksheet during the following week while continuing to record the videos.

In Phase 4 (fourth meeting), the therapist focused on analyzing the ABC worksheet compiled by the parents in order to identify the function of the monitored behavior through the recorded videos. Through a careful analysis of the antecedents and the consequences of the problem behavior over time, the therapist explained to parents why and how to follow the correct behavioral procedures that would favor a positive change. At the end of the meeting, parents were instructed to continue monitoring the behavior, recording video data, and filling out the ABC worksheet. Parents were now instructed to record videos with the same duration (45 min), in the same home location, and in the same context (i.e., presence/absence of parents) in order to gather the baseline (T0) for the BST.

In Phase 5, the fifth online meeting was held. The BST approach was integrated into the meeting sessions. This training utilizes instructions, modeling, rehearsal, and feedback in order to teach a new skill. The therapist first explains the skill to the parent, then models his/her behavior, who in turn models that of their child. The rehearsal phase is associated with role-playing used to train parents and model their behaviour during online meetings. At the end of each instance of the BST, the operator provided feedback on the performance. Feedback is positive when parents perform in accordance with the

BST procedure, or corrective when the parents have difficulty in following the instructions correctly. In our study, we used both types of feedback to aid parents in following the BST instructions. Online coaching was complemented by homework with written instructions on the procedure, which was sent to parents at the end of meetings.

In Phase 6, a debriefing session was held between the therapist and the parents regarding the tasks that had been previously assigned. The procedural fidelity was evaluated in order to assess the adherence to the treatment and to maintain it if the behavioral change was moving towards the expected goal. In the case of lapses in the procedures, phase 5 was briefly resumed and the fidelity reestablished.

During the 6th to 12th online meetings, parents and therapists moved between phases 5 and 6 according to the training needs and continued to record videos.

Parents were instructed to continue to apply the procedures explained during BST and record a new session with the same duration (45 min), in the same home location, and in the same context (i.e., presence/absence of parents) every day. The idea was to reproduce the same procedures presented by the therapist during the sessions, leading to comparable outcome measurements over time. These sessions at home without a consultant's input were recorded digitally using video-conferencing software for data collection. Specific reminders (by email or phone calls) were delivered to the parents in order to ensure that the videos were recorded during T1, T2 and T3.

Phase 7 was dedicated to monitoring fidelity and analyzing the progress of tasks. The results were analysed through a visual inspection of recorded materials during T0, T1, T2 and T3. Two additional experienced behavioral consultants, blind to any other result, evaluated the frequency of the problem behavior and the fidelity of the parental behavioral procedure. During the video analysis, the behavioral consultant marked the presence/absence (using "+" or "−" signs on a datasheet) of the problem behavior, obtaining a trend of the behavior over time, and marked the correct/incorrect procedure performed by the parents according to the schedule given by the therapist. Fidelity was dichotomized using "p" or "n" letters for each instance (pass or no-pass).

In the final phase (Phase 8), the feedback from the participants and parent-reported questionnaires was gathered. A timetable of the protocol, treatment and phases is reported in Table 2.

#### *2.6. Training Experience*

The therapists who delivered the interventions were all chartered psychologists, or psychotherapists, with behavioral analyst training and at least 5 years' experience in working with children on the autism spectrum.

#### *2.7. Outcome Measurements*

Outcome measurements were divided into: (a) objective measurement of problem behavior in children occurring on the day immediately after each session; and (b) psychological assessment of parents recorded before and after treatment.

For children with ASD, the main outcome was the frequency of the problem behavior which was recorded for statistical purposes at four different timepoints: Baseline (T0), and 20 days (T1), 40 days (T2), and 60 days (T3) after training (phase 5).

In terms of the parents, we collected outcome measurements on parenting stress together with a scale for assessing the parent's perception of the child's behavioral manifestations. This evaluation was carried out twice, at the baseline and at the end of treatment. The main outcome measures were the Parenting Stress Index (PSI) to assess the level of stress before and after treatment and the Home Situation Questionnaire (HSQ-ASD), which provides objective measures of the perception and influence of children's behavior on the parents' lives.


**Table 2.** Timetable of the protocol and data acquisition.

#### 2.7.1. Parenting Stress Index/Short Form (PSI-SF)

The PSI-SF is a self-assessment questionnaire [19]. It takes about 10–15 min to complete the questionnaire. Parenting stress levels are assessed by analyzing three different factors: 1. characteristics of the children, 2. characteristics of the parent, and 3. aspects related to the parental situation. The short module is made up of 36 items, divided into three subscales: (1) Parenting Distress (PD), referring to the feelings of the parents; (2) dysfunctional parent–child interaction (P—CDI), which focuses on the child's perception as unresponsive to parental expectations; and (3) Difficult Child (DC), which focuses on some of the characteristics of the child that make him/her easy or difficult to manage.

#### 2.7.2. Home Situation Questionnaire (HSQ-ASD)

The HSQ-ASD [20] is a caregiver-rated scale designed to assess the severity of disruptive and non-compliant behaviors in children. The scores obtained with this scale refer to the parent's perception of their child's behavioral manifestations. Within the scale, data are collected on the inflexibility (HSQ-I) and avoidance (HSQ-A) manifested by the child. This modified and revised version for ASD consists of 27 elements. Parents were asked to indicate whether their children had problems with compliance in these situations and, if so, to rate the severity on a Likert scale of 0 to 9, with higher scores indicating greater non-compliance.

#### *2.8. Statistical Analysis*

Statistical analyses were performed using SPSS v. 23.0 (IBM, Armonk, NY, USA). The Kolmogorov–Smirnov test was carried out which confirmed the assumptions of normality only for psychological outcome measures.

Families who dropped out and families who continued with treatment were compared using a *t*-test for continuous variables and chi-square test for categorical variables.

For children, although the sample size was small, based on our experience in treating behavioral disorders in young children with ASD, we expected large effects. Therefore, we applied repeated measures ANOVA, reporting the post hoc power observed. The Greenhouse–Geisser correction was used if conditions of sphericity were not met.

The initial frequency of behavior can differ considerably for different children and different behaviors. For each participant, we therefore used the ratio of frequencies measured at different time steps with the initial frequency.

Inter-rater agreement was computed using Cohen's kappa (*k*).

For the parents' measures (PSI-SF; HSQ-ASD), we expected a smaller effect size, and therefore used non-parametric (Wilcoxon signed-rank test) and parametric statistics (paired *t*-test) aimed at analyzing the effects of the treatment on parents' outcome measures. We adjusted the alpha level using a Šidák correction for hierarchical multiple comparisons, with alpha = 0.025 for the primary measures (SI/SF and HSQ-ASD scores).

#### **3. Results**

The attrition rate was 42%. Indeed, six families dropped out before the beginning of phase 1, due to difficulties in managing weekly online connections, work commitments and/or family management of other children. Eight children with ASD and their relative parents (n◦16), completed the treatment and were finally analyzed. Table 3 shows the demographic and psychological characteristics of children with ASD and their parents. The between-groups comparison showed no significant differences on any descriptive and psychological characteristic (*p* > 0.05).

Table 4 shows each individual behavioral symptom which was intended to be eliminated or reduced and its function. In Table 5, the behavioral treatment for each case is presented through the web-based training protocol with a brief explanation. The different procedures were performed as reported by Cooper and colleagues [18]. As expected, children with ASD showed heterogeneous behavioral spectrum disorders. Treatments were tailored to each specific behavioral symptom with the aim of reducing its frequency.

#### *3.1. Fidelity*

No parent reported with videos every day. On average, 2.93 (0.73) [2.10–4.32] videos were reported per week. There were no weeks with less than one video for each participant. No video was missing at T0, T1, T2 and T3. For all the families, behavioural data were gathered only by the main caregiver (in all families, it was the mother).

Procedural fidelity of the parent was 0.81 (0.13) [0.75–1.00] at T1, 0.97 (0.07) [0.80–1.00] at T1, 1.00 (0.00) [1.00–1.00] at T3. It should be noted that the number of instances on which fidelity was based decreased with the increase in time-steps.

#### *3.2. Inter-Rater Agreement*

Inter-rater agreement was excellent. Raters agreed in 94% of 224 instances for behavioral frequency, *k* = 0.883, and 92% of 119 instances for parents' procedural fidelity, *k* = 0.849. No disagreement was present in instances at T3. For the other time-steps, a final agreement was reached in all cases through discussion between the raters.

#### *3.3. Behavioural Results*

During active treatment, there was an immediate trend towards a better clinical outcome (Table 6), which at the end of the treatment was significantly improved (Figure 1).


**Table 3.** Demographic and psychological characteristics of children with ASD and their parents.

Data are given as mean values (SD), and median (range). ASD: autism spectrum disorders; DQ: Developmental Quotient. Data are expressed as mean ± SD or median (range) values if assumptions of normality are proved or otherwise.

**Table 4.** Problem behaviors and relative function in the ASD children enrolled.



**Table 5.** Problem procedures and relative explanation in the ASD children enrolled [21].

Mauchly's test of sphericity was not significant, Mauchly's W = 0.440, *X*<sup>2</sup> = 4.70, *df* = 5, *p* = 0.460.

The multivariate test was significant, *F* (3, 5) = 247, *p* < 0.001, n2 = 0.993 with an observed power (OP) of 1.00. The within-subject effect was also significant *F* (3, 21) = 92.3, *p* < 0.001, n<sup>2</sup> = 0.930, OP = 1.00. However, the between-subject effects were also significant, *F* (1, 7) = 398, *p* < 0.001, n<sup>2</sup> = 0.983, OP = 1.00.

In order to better understand the structure of change, we tried different polynomial models of frequency-change in time, and the best fit was for a linear model with *F* (1, 7) = 770, *p* < 0.001, n<sup>2</sup> = 0.991, OP = 1.00.

A decrease in behavioral frequency was present in all children in all the time steps, and a pairwise comparison showed that differences were significant between T1 and T0 (M = 38% [17%; 58%], *p* = 0.002), T2 and T1 (M = 26% [5%; 47%], *p* = 0.016), and T3 and T2 (M = 26% [2%; 50%], *p* = 0.032).


**Table 6.** Frequency of behavioral symptoms during active treatment.

\* Frequency is expressed as the number of events during the session.

**Figure 1.** Significant reduction in the frequency of behavioral symptoms during web-based treatment with the BST approach.

#### *3.4. Parental Wellbeing*

We also evaluated the effects of our web-based training on the psychological wellbeing and perception of severity in parents of ASD children, measured before and after the treatment (Table 7). As concerns the PSI-SF scale, we found a significant reduction in total psychological stress and PD and P-CDI subscales after treatment. A similar beneficial effect was also detected in the HSQ-ASD scale, where parents showed a significant reduction in the perception of the severity of inflexibility (HSQ-I) and avoidance behaviors manifested by their children after treatment (HSQ-A).


**Table 7.** Psychological effects on parents before and after BST treatment.

Data are given as mean values (SD), and median (range). PSI-SF: Parenting Stress Index/Short Form; HSQ-ASD: Home Situation Questionnaire; PD: Parenting Distress; P-CDI: Dysfunctional parent–child interaction; DC: Difficult Child.

#### **4. Discussion**

In this study, we demonstrated that a well-known behavioral approach, that is, BST, could be efficiently offered to children with ASD and their parents using a low-cost and commercial web-based training approach. We found a significant reduction in the frequency of behavioral disorders detected in children with ASD after treatment, as well as a general reduction in psychological distress and the perception of the severity of ASD-related symptoms in parents.

This study demonstrated the feasibility of live coaching on BST procedures as a valid approach to managing behavioral disorders in children with ASD and for helping their parents deal with such disorders during COVID-19 pandemic. Distance behavioral training programs should be encouraged, and public awareness should be raised among parents and clinicians aimed at regarding telehealth as an alternative and valid means of providing treatment [22,23]. In the future, such programs could help tackle the lack of therapists available to support the growing demand for their services and the role of primary caregivers as a critical component in the success of treatments [24,25].

In our study, all the children showed a marked progressive reduction in the frequency of their targeted behavioral symptoms during the follow-up period. In fact, after 20 days (T1), the lowest reduction was 17% and the median was 35%; after 40 days, the lowest was 38% and the median 65%; and finally, at the end of the protocol (60 days), the lowest was 75% (ASD S2) with a median reduction of 90%, with two out of eight children no longer showing the target inappropriate behavior.

We also found a reduction in parent-reported stress, a decrease in child inflexibility and avoidance, and a more functional parent–child interaction. The only scale that showed no significant reduction (and showed a slight increase) was PSI-DC. This scale is designed to assess the parent's perception of child-related disorders. To explain this finding we propose that, although our remote BST parent training might induce a better understanding of the difficulties they are facing, the adaptation to stressful events associated with caring for an autistic depends on several factors (family demands; family adaptive resources, the family's definition of the stressful situation and family adaptive coping mechanisms) [23].

There are several studies demonstrating the effectiveness of parental training on the behavioral patterns of children with ASD [26,27]. In fact, parent training can effectively help in improving parent–child interactions and social communication [28]. A study by Tonge et al. [29] demonstrated that both parental education and training in behavioral procedures for children with autism can be beneficial for parental mental health. In Tonge's study [29], parents received a manual-based education and behavior management skills training package. Sessions were skills-based and action-oriented through the provision of workbooks, modeling, videos, rehearsal, homework tasks, and feedback. At a 6-month follow-up evaluation, the combination of parent education and behavioral teaching led to a significant reduction in anxiety, insomnia, and somatic symptoms. Hassan et al. [30] demonstrated the effectiveness of BST in developing parental skills and treatment strategies to support specific social behaviors in their children. In Hassan's study, free-play sessions with parents and their children were structured. These authors demonstrated that treatment carried out according to the BST procedure improved functional skills and communication by decreasing the manifestation of destructive behaviors.

A recent systematic review provided evidence of the utility of telehealth as a service delivery model for providing analytic-based services and for training caregivers to implement behavioral assessments and procedures [31]. Despite this evidence, there are a few studies which evaluated the feasibility of telehealth for families with ASD. Boutain et al. [32] evaluated the success of a remote web-based BST training package to teach parents to implement new treatments with their children for the independent completion of three self-care skills (washing hands, washing faces, and applying lotions). Another study on online coaching in daily living skills, with four children ranging in age from 5 to 9 years, showed that parents faithfully implemented treatment which led to increases in independent daily living skills for all the participants [33]. In line with all this evidence, we demonstrated that a remote BST service is useful in managing behavioral disorders and improving parent awareness by reducing the frequency of various behavioral disorders in children, as well as lowering psychological distress in parents.

#### *Limitations*

The first limitation of this study was the high attrition rate (42%). Nevertheless, it should be noted that all the drop-outs happened at the beginning of the study and none dropped after the initial assessment. Even if we found no significant difference between drop-outs and families who continued till the end of the protocol, it could be noted that the families who dropped out tended to have working mothers with more children and lower educational levels. This agrees with the concerns expressed during the interviews about the difficulties in reconciling BST with family and work schedules. This is further reinforced by the fact that in all families that continued with the protocol, even if both parents participated in the training, only the mothers actively reported behavioral data. As a consequence, we cannot account for the effect of the involvement and support of the fathers in the therapeutic outcome. Furthermore, we did not record other undesired behaviors, and we therefore do not know whether parents independently applied the learned procedures to other behaviours and if that could partly explain the decrease observed in parental psychological measures. Future studies should improve the feasibility of this procedure and study its contextual needs and accommodations to remotely intercept different populations and family organizations.

Another limitation of the current study was the limited number of participants. A larger sample could provide a more accurate analysis of the effects of treatment. One internal limitation was the lack of a control group which prevented a direct comparison with another type of intervention. Furthermore, the analysis was performed on a group of heterogeneous behavioral disorders and procedures. This limitation is evident due to the large between-subject effect size of the study. Nevertheless, this limitation is also a strength, given the lack of online behavioral studies on the topic. In fact, we believe we have shown

the feasibility of the training for a wide set of procedures and hope that this will foster a larger variety of new studies that no longer focus exclusively on a single procedure.

Finally, we gathered only the frequency of undesired behaviours to avoid overwhelming the parents with too many tasks. Therefore, we did not know of the occurrence of alternative behaviours or the emergence of new undesired behaviours. We recommend that future studies include a larger number of participants and a parallel control group receiving a traditional behavioral treatment. In addition, future research could include an analysis of treatments with homogeneous behaviors. Another important topic would be to assist practicing clinicians in determining an appropriate protocol for the delivery of a 1:1 telehealth service. Some proposals are already in place, but still lack experimental support [31].

#### **5. Conclusions**

The COVID-19 pandemic has created an enormous amount of suffering for all levels of society, but especially for more vulnerable people. At the same time, it has also led researchers to search for new solutions and to challenge long-held assumptions regarding therapy. Remote health services can provide a sustainable model for both conducting assessments and training healthcare professionals and staff in order to implement several behavioral strategies, as proposed by Ferguson et al. [34]. In this study, we confirm that parental coaching can also be carried out through an easily accessible remote service aimed at making the parents of children with ASD increasingly competent in the daily management of their children and being more empowering in their role. Such remote services can provide support through skills and behavior management treatments not only during global health crises, but also as a sustainable and standard model for the future.

**Author Contributions:** Conceptualization, F.M., P.C., A.C. and G.P.; methodology, F.M., P.C., C.F., R.M. (Roberta Minutoli), N.V., D.V. and A.C.; software, M.B., A.P., G.T. and D.V.; validation, F.M., P.C., R.M. (Roberta Minutoli), N.V. and L.R.; formal analysis, A.A.A., A.P., R.M. (Rosa Musotto), G.T., D.V., A.C. and G.P.; investigation, F.M., C.F., N.V., C.L., C.C., E.L., G.C. (Giuliana Cardella), F.I.F. and G.C. (Gaspare Cusimano); resources, F.M., C.F., R.M. (Rosa Musotto), S.G., P.M., G.M., D.V., L.R. and G.P.; data curation, F.M., C.F., N.V., C.L., C.C., E.L., S.G., G.C. (Giuliana Cardella), F.I.F., G.C. (Gaspare Cusimano), A.A.A., G.T. and D.V.; writing—original draft prepa-ration, F.M., C.F., D.V., A.C. and G.P.; writing—review and editing, F.M., D.V., A.C. and G.P.; vis-ualization, F.M., D.V. and G.P.; supervision, F.M., D.V., A.C., L.R. and G.P.; project administration, G.P.; funding acquisition, P.M., G.M., G.C. (Gaspare Cusimano) and G.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Assessorato Regionale delle Attività Produttive, Sicily, grant number n. 08SR2620000204, entitled LAB@HOME—Una Casa Intelligente per l'Autismo, P.O. F.E.S.R. Sicilia 2014/2020, Azione 1.1.5, and funded by the Azienda Sanitaria Provinciale—ASP-di Trapani: Grant number n. 08SR2620000204 Assessorato Regionale delle Attività Produttive Sicilia and Deliberation n. 20190003196-10/12/2019 ASP Trapani.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Research Ethics and Bioethics Committee (http://www.cnr.it/ethics, accessed on 17 December 2021) of the National Research Council of Italy (CNR) (Prot. N. CNR-AMMCEN 54444/2018 01/08/2018) and by the Ethics Committee Palermo 1 (http://www.policlinico.pa.it/, accessed on 17 December 2021) of Azienda Ospedaliera Universitaria Policlinico Paolo Giaccone Palermo (report n. 10/2020–25/11/2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The datasets generated during the current study are available from the corresponding author on reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **PTSD Symptoms and Coping with COVID-19 Pandemic among Treatment-Seeking Veterans: Prospective Cohort Study**

**Marina Letica-Crepulja 1,2,\* , Aleksandra Stevanovi´c 1,2,3 , Diana Palai´c 2, Iva Vidovi´c <sup>1</sup> and Tanja Franˇciškovi´c 1,2,3**


**Abstract:** Background: The aim of this study was to examine post-traumatic stress disorder (PTSD) symptom levels and coping strategies during the COVID-19 pandemic among treatment-seeking veterans with pre-existing PTSD. Method: A cohort of 176 male treatment-seeking veterans with pre-existing PTSD during the first COVID-19 pandemic lockdown (T1) and 132 participants from the same cohort one year after the onset of the pandemic (T2) participated in a longitudinal study. All participants responded to a COVID-19-related questionnaire and the following measures: the Life Events Checklist for DSM-5 (LEC-5), PTSD Checklist for DSM-5 (PCL-5) and the Brief COPE. Results: The intensity of overall PTSD symptoms, avoidance symptoms and negative alterations in cognitions and mood was lower at T2. PTSD symptoms were not significantly correlated with SARS-CoV-2 potentially traumatic events (PTE) at T2. Veterans scored higher on emotion-focused and problem-focused coping than on dysfunctional coping. Conclusions: Veterans with pre-existing PTSD who were receiving long-term treatment coped with COVID-19 stressors without the effects of retraumatization and a consequent worsening of PTSD symptoms.

**Keywords:** COVID-19; PTSD; war-related stress; pandemic-related stress; treatment-seeking

#### **1. Introduction**

The coronavirus disease-19 (COVID-19) pandemic has caused multiple health, social, and economic stressors and presents a progressively emerging and potentially long-lasting life threat [1]. Pandemic stressors have a broad spectrum of impacts including mental health risk for individuals around the world. Extensive research has documented the negative psychiatric consequences of the COVID-19 pandemic [2–4]. A study that examined the global impacts of the pandemic on major depressive and anxiety disorders showed that prevalence of major depressive disorder and anxiety disorders had increased by more than 25 percent worldwide [3].

Patients with pre-existing mental disorders appeared to be at higher risk for wideranging mental health effects [5–7]. Regarding the veteran population, the results from recent studies revealed resilience to mental health problems and lower rates of suicidal ideation among US military veterans nearly 10 months into the pandemic [8,9]. However, these studies also showed that the prevalence of generalized anxiety disorder increased and the prevalence of major depressive disorder and post-traumatic stress disorder (PTSD) remained stable during the pandemic [10]. A recent longitudinal study, conducted during the period between two lockdowns among treatment-seeking veterans with pre-existing mental health difficulties in the UK, did not find any significant changes in symptoms of PTSD, common mental health disorders, anger, or alcohol use between the lockdowns [11].

**Citation:** Letica-Crepulja, M.; Stevanovi´c, A.; Palai´c, D.; Vidovi´c, I.; Franˇciškovi´c, T. PTSD Symptoms and Coping with COVID-19 Pandemic among Treatment-Seeking Veterans: Prospective Cohort Study. *J. Clin. Med.* **2022**, *11*, 2715. https://doi.org/ 10.3390/jcm11102715

Academic Editors: Alfonso Troisi and Ursula Werneke

Received: 20 February 2022 Accepted: 9 May 2022 Published: 11 May 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

A study that compared prepandemic and peripandemic levels of PTSD symptoms among Croatian treatment-seeking veterans with pre-existing PTSD demonstrated a reduction in PTSD symptom levels during the onset of the COVID-19 pandemic [12]. As the COVID-19 pandemic is often referred to as a "marathon, not a sprint," and is spread around the world, there remains a need for further longitudinal studies to examine long-term changes in mental health of veterans with pre-existing difficulties in different settings.

Coping reflects a form of adaptation elicited by stressful circumstances. The previous research has repeatedly shown that combat veterans with PTSD use a non-adaptive and avoidance coping style [13–15]. The coping strategies veterans use when facing the pandemic stressors are important, because they can either alleviate or cause additional stress. There is a lack of systematic research on coping with pandemic stressors among veterans with pre-existing mental health difficulties including PTSD.

Almost thirty years after the Homeland War in Croatia (1991–1995), veterans still suffer from numerous health problems. In 2020, there were 33,089 treatment-seeking veterans with PTSD, which is 7.76% of the overall veteran population in Croatia [16]. A recent study revealed high rates of overall symptoms and greater severity of post-traumatic symptoms (i.e., complex PTSD) among treatment-seeking veterans years after the war ended [17]. The aim of this study was to follow the trajectories of PTSD symptom levels and examine coping strategies during the onset and one year after the onset of the COVID-19 pandemic among treatment-seeking veterans with pre-existing PTSD.

#### **2. Materials and Methods**

#### *2.1. Participants and Procedure*

All participants were male veterans from the Homeland War in Croatia (1991–1995) and had been in treatment for war-related PTSD for an average of 18.8 years (range 3 to 30 years at the second measurement) at the Referral Center of the Ministry of Health of the Republic of Croatia (RCPTSD) at the Clinical Hospital Center (CHC) Rijeka. Of 250 treatment-seeking veterans whom we approached, 176 (70.4%) participated at the first measurement (T1), which took place from April 15 to the end of May 2020 (i.e., during the onset of the COVID-19 pandemic). Sixty-four veterans could not be reached, five refused to participate, and five were excluded from further analysis due to incomplete data. They were recontacted during April and May 2021 (T2) when the third wave of the COVID-19 pandemic reached its peak. One hundred thirty-two (132) responded and consented to participate in the second measurement. Of the 44 who did not take part in the second measurement, five refused to participate, one had died, and the remaining 39 could not be reached even after three phone calls. Since their first referral, and before T1, participants had been involved in one or more outpatient treatment options: intensive PTSD program for day-care hospital (78.2%), long-term psychotherapy (26.3%), low-level treatment groups such as PTSD Club (11.3%), and regular outpatient psychiatric appointments (89.5%). Every other participant had received inpatient care (49.6%) at some point during their treatment. At T1 and T2, and in between, participants had been treated in regular outpatient psychiatric appointments with no changes in the administration of regular medication.

Due to the ongoing COVID-19 restrictions, the assessments were conducted by telephone or face-to-face with participants who attended the check-up. After the participants had been given detailed information about the study, all participants provided their written informed consent. In cases when the assessments were made by telephone, the written informed consent was provided at the next regular onsite appointment. The two parts of the evaluation consisted of a structured clinical interview and self-report questionnaires. Besides sociodemographic items, the structured interview at T2 included additional questions on difficulties with specific aspects of the COVID-19 pandemic. The study was approved by the Ethical Committee of CHC Rijeka, ethical approval code 003-05/20-1/85.

#### *2.2. Measures*

The measures used in the study were the Life Events Checklist for DSM-5 (LEC-5), PTSD Checklist for DSM-5 (PCL-5) and the Brief COPE. At the second assessment, the participants were asked whether they had experienced any traumatic life event in the past twelve months since the lifetime traumatization was assessed at T1.

The Life Events Checklist for DSM-5 (LEC-5) was used to assess possible traumatic events experienced by participants in the past twelve months [18]. The self-report measure lists 16 traumatic events and an additional item indicating any other stressful event. For the study, the total score of lifetime trauma, calculated as the sum of traumatic events, ranged from 0 to 17. The checklist was reported to have good psychometric properties [18,19].

The PTSD Checklist for DSM-5 (PCL-5) with Criterion A is a self-report measure revised to match the adapted DSM-5 criteria for PTSD [20]. A provisional PTSD diagnosis can be made considering items rated 2 = moderately or higher according to the DSM-5 diagnostic rule (at least one B, one C, two D, and two E symptoms present). Symptom severity was calculated as the sum of all items (0–80) or as the sum within a specific cluster of symptoms. Authors reported the score of 33 or above to be indicative of probable PTSD diagnosis and was thus used as the cut-off score in this study. Validation studies showed excellent psychometric properties for evaluating PTSD [20–23]. PCL-5 showed good internal consistency in our study, with Cronbach alphas ranging from 0.72 to 0.85 for clusters and 0.90 for total PCL-5.

The Brief COPE is a 28-item multidimensional measure of coping strategies used for regulating cognitions and behaviors in response to stressors [24]. Fourteen two-item scales are rated on the four-point rating scales (1 = I have not been doing this at all to 4 = I have been doing this a lot). The score of each scale is the sum of two items, with the possible range of 0–8. Each scale can be viewed independently or as part of emotion-focused, problem-focused, or dysfunctional coping strategies [24,25]. The Brief COPE is reported to have good psychometric properties [24]. Good psychometric properties were also reported for the Croatian version of Brief COPE in the study with a large adult online sample [26]. The Cronbach alpha in our study was 0.81.

The COVID-19-related questionnaire was created for specifically this study, and consisted of three parts. The first part was related to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection of the participants and their close relatives, severity of the COVID-19 symptoms, the need for hospitalization, the level of recovery and death of a close family member. The second part was related to the level of compliance to the government COVID-19 measures. The third part was related to COVID-related stressors: pandemic duration, pandemic uncertainty, media coverage/exposure, restricted family gatherings, restricted access to medical services, fear of COVID-19, physical distancing, financial burden, self-isolation, restricted access to domestic supplies, and other.

#### *2.3. Data Analysis*

The existing dataset from T1 and the data collected at T2 were used for the analysis. Descriptive statistics were used to present frequencies/percentages or means and standard deviations for parametric measures. Pearson's chi-square test was used to test for differences between groups on categorical variables. In cases where the cells' frequency was below five, the Yates correction was applied. Group differences on continuous variables were tested with *t*-tests or Mann–Whitney U tests when number of cases was <20. Differences between measurements on continuous variables were tested with t-tests for repeated measurement. Spearman's correlation coefficient was applied to test for significant correlations. Statistical significance was set as *p* < 0.05. Statistical analysis was performed with Statistica software, version 12 (Dell Inc., Tulsa, OK, USA).

#### **3. Results**

#### *3.1. Demographics and War-Related Characteristics*

Of 176 male participants in the first measurement, 132 (75%) war veterans participated in the second measurement. The sociodemographic characteristics are presented in Table 1. The missing 44 participants and those who participated at T2 did not differ significantly on any of the sociodemographic characteristics or war-related measures. Therefore, the functional equivalence and representativeness of the sample allowed for further analysis.

**Table 1.** Sociodemographic characteristics of study participants at the first assessment (T1). Data are presented as count (percentage) unless otherwise indicated.


Since the previous assessment, none of the participants changed their educational level, four (3%) relocated, and six (4.5%) had their marital status changed. In the last year, 16 (12.1%) participants had their employment status changed as 4 participants found employment, 1 lost his job, and 11 retired. Eighteen participants (13.6%) reported a change in their economic status with 16 (12.1%) reporting lower economic status and 2 (1.5%) better economic status. The majority of participants (*n* = 101, 76.5%) did not experience a potentially traumatic event (PTE) over the past twelve months, 28 (21.2%) reported one PTE, and 3 (2.3%) participants reported from two to six PTE in the last year (assessed by LEC).

#### *3.2. COVID-19 Related Characteristics*

Since the first assessment when only one participant had been infected, 14 (10.6%) participants were infected with SARS-CoV-2 between T1 and T2. Five participants had mild, four had moderate and four had severe COVID-19 symptoms. One participant with severe COVID-19 symptoms had been treated in the intensive care unit. Eight had wholly recovered, and six reported they had partially recovered from COVID-19. Compared to the previous year when only 2 participants had a close family member who tested positive, at the second measurement, 46 (34.8%) reported a family member or a close person with COVID-19, and 5 participants had lost a close relative due to the illness.

The majority of the participants said that they had entirely followed (*n* = 71, 53.8%) or mostly followed (*n* = 41, 31.1%) the precautionary measures introduced by the government during the past year.

The participants were asked to assess if they were affected by COVID-19-related non-PTE stressors. The most challenging COVID-19-related non-PTE stressful aspect was pandemic duration, since 99 (75%) of participants reported to be affected by it. Other stressors were pandemic uncertainty (69.7%), media coverage/exposure to pandemic (67.4%) and restriction of family gatherings (65.9%). The least frequent COVID-19-related non-PTE stressor was self-isolation (35.6%) (Figure 1). The increase in number of experienced COVID-19-related non-PTE stressors was not significantly related to the overall PTSD symptom intensity at T2 (r = 0.15, *p* = 0.08).

**Figure 1.** The proportion of the participants affected by COVID-19-related non-PTE stressors.

#### *3.3. PTSD Symptom Severity*

Most of the participants scored 33 or higher on total symptom severity, indicative of probable PTSD diagnosis (T1 83.8%, T2 80.6%). The intensity of the overall PTSD symptoms, avoidance symptoms and negative alterations in cognitions and mood was significantly lower at T2. The change in intrusion symptoms and alterations in arousal and reactivity was not statistically significant (Table 2).

The PTSD symptoms were neither significantly correlated with SARS-CoV-2 (personally or a close one's) infection nor the intensity of the COVID-19 symptoms (personally or of a close one) at T2. Of 37 participants with at least one PTE in the last year assessed by LEC, 9 experienced COVID-19-related PTE. A comparison between participants with COVID-19-related PTE (severe personal COVID-19 illness and death of a close one due to COVID-19) and participants with at least one other PTE in the last 12 months was made. The participants with COVID-19-related PTE (*n* = 9) did not differ in the intensity

of overall PTSD symptoms at T2 compared to participants with other PTE (*n* = 28) in the last 12 months (U = 101.5, *p* = 0.39). In the overall sample, the number of experienced PTEs in the last year was not significantly related to overall PTSD symptom intensity (r = 0.12, *p* = 0.19).

**Table 2.** Overall PTSD symptom severity and PTSD symptom clusters severity on two measurements during the COVID-19 pandemic \*.


\* Abbreviations: PTSD—post-traumatic stress disorder; SD—standard deviation; PCL-5—PTSD Checklist for DSM-5 (PCL-5) with Criterion A.

The overall PTSD symptoms and cluster D symptoms were significantly correlated with perceived difficulties in dealing with media coverage (overall symptoms: r = 0.19, *p* < 0.05; cluster D symptoms: r = 0.23, *p* < 0.05) and with restricted access to domestic supplies (overall symptoms: r = 0.22, *p* < 0.05; cluster D symptoms: r = 0.24, *p* < 0.05). The increase in number of experienced COVID-19-related non-PTE was not significantly related to the overall PTSD symptom intensity at T2 (r = 0.15, *p* = 0.08).

#### *3.4. Coping with COVID-19-Related Issues*

In general, the average scores of individual coping strategies had diminished over a year. The average means for individual coping strategies in two measurements are presented in Figure 2. Acceptance and self-distraction continued to be the most frequently used coping strategies one year into the pandemic. The participants reported a significant decrease in self-distraction (t = 6.402, *p* < 0.001), active coping (t = 3.788, *p* < 0.001), denial (t = 2.825, *p* = 0.005), emotional support (t = 5.368, *p* < 0.001), venting (t = 3.489, *p* = 0.001), positive reframing (t = 3.821, *p* < 0.001), planning (t = 2.815, *p* = 0.006), and religion (t = 2.55, *p* = 0.012).

In an alternative way of grouping coping strategies, in the second measurement, veterans scored higher on emotion-focused coping (Mean = 4.07, SD = 1.01) and problemfocused coping (Mean = 3.71, SD = 1.57) than on dysfunctional coping (Mean = 3.13, SD = 0.96). Compared to the first measurement, there was a significant decrease in all three types of coping (emotion-focused coping: t = 4.72, *p* < 0.001; problem-focused t = 3.589, *p* < 0.001; dysfunctional coping: t = 3.45, *p* < 0.001) (Table 3).

**Table 3.** Average scores for alternative grouping of the coping strategies on two measurements during the COVID-19 pandemic.


**Figure 2.** The average scores on individual coping strategies (the Brief COPE) in overall sample (range of 0–8).

#### **4. Discussion**

The aim of this study was to compare PTSD symptom levels and examine coping strategies at the onset and one year after the onset of the COVID-19 pandemic among treatment-seeking male veterans with pre-existing PTSD. The main finding was that the intensity of the overall PTSD symptoms (T1= 47.28, SD = 12.81; T2= 43.82, SD = 14.46; *p* = 0.016), avoidance symptoms and negative alterations in cognitions and mood decreased despite the greater exposure to COVID-19-related PTEs and the twelve-month longer exposure to COVID-19-related non-PTE stressors. The findings suggested that veterans with PTSD receiving a long-term treatment experienced COVID-19 stressors without the effects of retraumatization and a consequent worsening of PTSD symptoms. Stressful experiences to which the participants were exposed during the lockdown differ from the initial traumatic war-related stressors. Therefore, they had a lower potential for retraumatization (i.e., PTSD exacerbation after experiencing a new traumatic event). War-related stressors are central in the personal life experience of the participants and are the most substantial factors associated with PTSD [27–29]. The stressful experiences related to the COVID-19 pandemic appeared to be far from the central negative event to the participants' life and identity. The findings of this study are consistent with emerging results regarding the mental health of veteran populations during the pandemic. In a nationally representative sample of US military veterans, the prevalence of major depressive disorder and PTSD positive screens remained stable while the prevalence of generalized anxiety disorder positive screens increased [10] and the rate of suicidal ideation decreased nearly 10 months into the pandemic [9]. Regarding the mental health among treatment-seeking veterans with pre-existing mental health difficulties, the UK study revealed no significant changes in symptoms of PTSD during the pandemic [11] and a study conducted in Croatia revealed a decrease in PTSD symptoms during the onset of the pandemic as compared to the measurement a year before [12]. PTSD symptoms and particularly avoidance symptoms may have reduced because of the restrictive measures that prevented greater extent of exposure to reminders of trauma in everyday life. Adherence to treatment, regularity of appointments, and stability of treatment as all participants continued to be treated as usual (TAU) may also be viewed as a functional coping strategy that prevented considerable risk of worsening PTSD symptoms [12]. The results emphasize the importance of treatment engagement of veterans who meet criteria for mental disorder, since approximately 60% of them do not seek help because of concerns about stigma, with many expecting to face

prejudice and discrimination [30]. Appropriate treatment options during the COVID-19 pandemic to prevent, treat, and mitigate the effects of COVID-19 are likely to promote mental health and prevent its deterioration [5].

PTSD symptoms were neither significantly correlated with exposure to the COVID-19 related PTEs nor were they significantly correlated with the number of experienced PTEs between the two pandemic timepoints. Regarding COVID-19-related non-PTEs in general, the higher number of experienced stressors was not significantly related to the overall PTSD symptom intensity at the second measurement (r = 0.15, *p* = 0.08). Recent research on the correlation between pandemic-related stressors and mental health problems in the general population revealed higher levels of symptoms of adjustment disorder in those with a variety of COVID-19-related stressors [31]. Among veterans with an increased level of mental health suffering, different COVID-related stressors appeared to be the strongest risk factors for increased suicidality [9], increased distress [10], common mental disorders, and hazardous drinking [32]. In treatment-seeking veterans, those who reported more COVIDrelated stressors and lower levels of social support may have been particularly vulnerable to an increasing severity of a range of mental health difficulties [11,33]. The results of this study indicating no correlation of PTSD symptom level with COVID-related PTE and non-PTE can be explained by a lower potential of COVID-related stressful experiences worsening PTSD symptoms because they differ from the central negative traumatic events (war-related criterion A) [12,29]. As the veteran cohort in this study was aged 54 (SD = 5.97), the results could be explained in light of the recent findings indicating that older adults may be more resilient and less affected by the mental health effects of the pandemic than younger age groups [3].

The veterans used coping strategies to a lesser extent one year after than during the onset of the pandemic, which points to a certain level of adjustment to "the new normal" circumstances. Moreover, they used emotion-focused and problem-focused strategies, which prevented worsening of their PTSD symptoms and enabled better adjustment to the pandemic circumstances. Current literature suggests that certain coping strategies, such as avoidance coping, are associated with a greater PTSD severity [34,35] and others, such as problem- or emotion-focused coping strategies without the aid of social support, are protective and associated with lower levels of PTSD [36]. These findings are in line with recent reports from Canada, where veterans reported predominantly adaptive coping strategies in dealing with COVID-related stressors and dysfunctional strategies such as increased alcohol intake the least [37]. US veterans reported greater appreciation of life, closer interpersonal relationships, and an increased sense of personal strength due to coping with COVID-related stressors [8]. Most of the participants (*n* = 111, 84.8%) followed the protective measures introduced by the government during the pandemic. Compliance with the measures may be viewed as an adaptive coping strategy of acceptance. A possible explanation for compliance with the measures may be returning to "combat mode" and military subordination [12].

Despite the decreased intensity between the two measurements, the PTSD symptoms and symptoms of negative alterations in cognitions and mood were significantly correlated with perceived difficulties in dealing with media coverage (overall symptoms: r = 0.19, *p* < 0.05; negative alterations in cognitions and mood: r = 0.19, *p* < 0.05) and with restricted access to domestic supplies (overall symptoms: r = 0.22, *p* < 0.05; negative alterations in cognitions and mood r = 0.22, *p* < 0.05). It has been repeatedly emphasized after the exploration of the psychological, social, and neuroscientific effects of COVID-19 that one of the immediate mental health priorities is to establish longer-term strategies for mental health since intensive media consumption may amplify distress and anxiety, and optimal patterns of consumption may enhance wellbeing [38,39]. In research carried out on the general population, both participants who had been directly exposed to COVID-19 and participants who had been indirectly exposed to COVID-19 (e.g., via media) experienced PTSD-like symptoms [40], which points to a recommendation of minimizing COVID-related media consumption.

The current findings should be considered within the context of several limitations. The sample size was relatively small, which increases the risk of type II errors. The present findings may not be generalized to the wider population of treatment-seeking veterans with PTSD as the context of traumatization and the context of the pandemic were related to Croatia. Therefore, further research is needed to evaluate the generalizability of the current findings to female, younger or more diverse veteran samples.

**Author Contributions:** Conceptualization, M.L.-C.; methodology, M.L.-C., A.S., D.P., I.V. and T.F.; software, A.S.; validation, M.L.-C. and A.S.; formal analysis, A.S.; investigation, M.L.-C., A.S., I.V. and D.P.; resources, M.L.-C.; data curation, M.L.-C., A.S., I.V. and D.P.; writing—original draft preparation, M.L.-C. and A.S.; writing—review and editing, M.L.-C., A.S., D.P. and T.F.; visualization, M.L.-C. and A.S.; project administration, M.L.-C. and A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded in part by the University of Rijeka, grant number uniri-prbiomed-19-3.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Clinical Hospital Center Rijeka (003-05/20-1/85; 12 June 2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available on doi: 10.5281/zenodo.5801404.

**Acknowledgments:** The authors would like to thank the participants of the study, without whom this research could not have been undertaken. The authors would also like to thank their colleagues Marica Cargonja, Karlo Damiš, Vlatka Fanjkuti´ ˇ c, Irena Jedriško, Paola Jovanovi´c, Ingrid Modrˇcin, Ivana Popovi´c, and Ornela Šebeli´c for their cooperation and support.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Adverse Mental Health Sequelae of COVID-19 Pandemic in the Pregnant Population and Useful Implications for Clinical Practice**

**Dariusz Wojciech Mazurkiewicz 1,\* , Jolanta Strzelecka <sup>2</sup> and Dorota Izabela Piechocka <sup>3</sup>**


**Abstract:** The COVID-19 pandemic has increased risk of disturbances in the functioning of everyday life, directly or indirectly has influenced the risk of mental disorders in the most vulnerable populations, including pregnant women. The aim of this study was to analyze adverse mental health effects in the pregnant population during the COVID-19 pandemic, investigate risk factors for adverse mental health outcomes, identify protective factors, and create practical implications for clinical practice, bearing in mind the need to improve perinatal mental healthcare during such pandemics. Qualitative research was conducted in the electronic databases PubMed and Web of Sciences for the keywords COVID-19, pregnancy, depression, anxiety, and telemedicine for relevant critical articles (*n* = 3280) published from 2020 until October 2021, outlining the outcomes of control studies, meta-analysis, cross-sectional studies, face-to-face evaluation survey studies, remotely administered survey studies, and observational studies regarding the main topic; all were evaluated. Mental health problems among pregnant women linked to the COVID-19 pandemic, in most cases, show symptoms of depression, anxiety, insomnia, and PTSD and may cause adverse outcomes in pregnancy and fetus and newborn development, even at later stages of life. Therefore, useful implications for clinical practice for improving the adverse mental health outcomes of pregnant women associated with the COVID-19 pandemic are highly desirable. Our research findings support and advocate the need to modify the scope of healthcare provider practice in the event of a disaster, including the COVID-19 pandemic, and may be implemented and adopted by healthcare providers as useful implications for clinical practice.

**Keywords:** COVID-19; psychological distress; anxiety; depression; pregnancy; disaster; telemedicine; theory–practice gap; stress; trauma

#### **1. Introduction**

The COVID-19 pandemic, as a public health emergency of international concern, constitutes a challenge to psychological resilience [1], heavily impacting global mental health [2] by causing acute respiratory coronavirus 2 (SARS-CoV-2) [3], with a rapid increase in the number of cases and deaths since its first identification in Wuhan, China, in December 2019 [4]. The vulnerable population of women who became mothers during the COVID-19 emergency appear to be at high risk for developing mental health problems [5]. Respiratory fluid droplets containing the SARS-CoV-2 virus may spread from person to person—in some cases, through contact with an infected surface when the person touches their eyes, nose, or mouth—and the average SARS-CoV-2 incubation period is 3–7 days [6]. The symptoms of COVID-19 thus may or may not be observed, and this disease can be confirmed by laboratory test results. Pregnant women may suffer from asymptomatic

**Citation:** Mazurkiewicz, D.W.; Strzelecka, J.; Piechocka, D.I. Adverse Mental Health Sequelae of COVID-19 Pandemic in the Pregnant Population and Useful Implications for Clinical Practice. *J. Clin. Med.* **2022**, *11*, 2072. https://doi.org/10.3390/ jcm11082072

Academic Editor: Alfonso Troisi

Received: 9 February 2022 Accepted: 3 April 2022 Published: 7 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

COVID-19 and an increased leukocyte count, raised lymphopenia, and higher C-reactive protein (CRP) levels, requiring extracorporeal membranous oxygenation (ECMO) and invasive ventilation more than nonpregnant women do [7]. The risk factors of severe COVID-19 in pregnancy are directly linked to preexisting diabetes, chronic hypertension, pre-eclampsia, and high body mass index (BMI); in the long term, outcomes such as higher risk of preterm birth (PTB) and/or pre-eclampsia may occur in pregnant women who have suffered from COVID-19 [8]. The risk of vertical SARS-CoV-2 infection in pregnancy, and its related fetal growth restriction, miscarriage, and preterm births are still quite unclear and under deliberation [9], and neonates born to mothers infected with SARS-CoV-2 have an overall favorable prognosis [6]. However, there is a clear link between poor mental health in pregnant women and pregnancy complications [10]. Mental health problems have been observed and reported to increase in the pregnant population in general, including an increase in the incidence of depression, antenatal, postpartum depression, and anxiety, all associated with adverse effects on intrauterine growth, birth weight, prematurity, behavioral, and/or mood in offspring, increasing the risk of depression during adolescence and adulthood, and linked to perinatal suicides and maternal mortality in the first 12 months after delivery [11–13]. The COVID-19 pandemic changed people's health behavior and sparked a psychological response reaction in society [14], especially in the pregnant population, currently suffering from an overall increased severity of anxiety [15] and an increased incidence of anxiety and depression [16]. There has been a noted sensitivity to social risks, an increase in negative emotions linked to anxiety and depression, indignation, and a decrease in positive emotions and life satisfaction after the declaration of COVID-19 in China related to the general human population [17].

The aim of this study was to analyze adverse mental health effects in the pregnant population during the COVID-19 pandemic, investigate risk factors for adverse mental health outcomes, identify protective factors, and create practical implications of mental health prophylaxis for clinical practice, bearing in mind the need to increase efforts in perinatal mental healthcare during such a pandemic.

#### **2. Materials and Methods**

A systematic scoping review [18] was conducted in electronic databases PubMed and Web of Sciences for keywords COVID-19, pregnancy, depression, anxiety, telemedicine for relevant critical articles (*n* = 3280) published from 2020 until October 2021 outlining the outcomes of control studies, meta-analysis, cross-sectional studies, face-to-face evaluation survey studies, remotely administered survey studies, and observational studies regarding the main topic, which were then evaluated. The equator checklist document used in this systematic scoping review was PRISMA Extension for Scoping Reviews [19] to analyze the presence of adverse mental health symptoms in pregnant women linked to the COVID-19 pandemic in domains as follows: social and medical consequences of COVID-19, psychological factors responsible for adverse COVID-19 mental health outcomes in pregnant women, the determination of COVID-19 mental health problems, including symptoms and diagnosis among pregnant women in different countries of the world, the influence of COVID-19 on psychological and medical factors related to adverse pregnancy and offspring development outcomes, and dilemmas and hopes in ways to improve the provision of services to pregnant women during disasters, including the COVID-19 pandemic.

#### *2.1. Literature Search and Study Selection*

A search was conducted on the PubMed and Web of Science databases. The search strategy included keywords linked to coronavirus, psychological symptoms, depression, and pregnancy. MESH terms (e.g., "pregnancy" (Mesh) AND "coronavirus" (Mesh) AND ("depression" (Mesh) OR "depressive disorder" (Mesh) OR "anxiety" (Mesh) OR "telemedicine" (Mesh) and text word search terms ("pregnancy" AND "coronavirus" AND ("mental health" OR "depression" OR "anxiety" OR "telemedicine") were used.

#### *2.2. Inclusion and Exclusion Criteria*

The inclusion criteria were as follows: (1) any study including outcomes of control studies, meta-analysis, cross-sectional studies, face-to-face evaluation survey studies, remotely administered survey studies, and observational studies that recruited pregnant women with adverse mental health outcomes that resulted from the pregnant women's experience with, exposure to, or infection with COVID-19; (2) studies written in English; (3) articles published from 2020 until October 2021; and (4) any original paper appearing in a peer-reviewed journal.

#### *2.3. Data Collection*

Two researchers used a multistep approach to select eligible studies. In total, 3280 publications were retrieved in our study; duplicates, publications, upon title and abstract review, not meeting the research criteria, publications not linked to the research topic, or those not meeting the inclusion criteria after full-text evaluation were removed. In the case of disagreement among researchers, it was resolved during a consensus session with a third researcher. After a full-text review, 16 articles met the inclusion criteria. Figure 1 represents the PRISMA flow diagram summarizing the screening process.

**Figure 1.** PRISMA flow diagram of the screening process.

#### *2.4. Data Extraction and Investigated Variables*

Two researchers independently extracted the following data: the aim of the study, country of the study, sample size, study design and assessment tool, and summary of study benefits and limitations. The publications strictly focused on the adverse mental health outcomes of the COVID-19 pandemic in pregnant women population, and research

topic publications that met the criteria for inclusion were linked. Data were processed independently by two researchers and are summarized in Table 1.

**Table 1.** Qualitative synthesis findings.





#### *2.5. Quality Assessment Tool*

The qualitative Newcastle–Ottawa Quality Assessment Scale (NOQAS) [36] and the adapted version for cross-sectional studies to score each study (Supplementary Materials) were used. This tool on the basis of the criteria included 3 categories ("selection", "comparability", and "outcome"), with a maximal score of 9 and 10 points for cohort and cross-sectional studies, respectively. The "selection" category, which accounted for a maximum of 4 points (5 points for cross-sectional studies), the "comparability" category, which accounted for a maximum of 2 points, and "outcome," which accounted for a maximum of 3 points. The quality of each study is shown in Table 1.

#### **3. Results**

#### *3.1. Characteristics of Included Studies*

Qualitative synthesis findings were concluded on the basis of 16 articles out of 3280 researched publications. A total of sixteen studies included in the final analysis: one study was conducted in China, Canada, Iran, Colombia, Ireland, Denmark, and Israel; two studies were done in the U.S., Poland, and Italy; three studies were conducted in Turkey. These publications described research into adverse mental health outcomes that resulted from a total number of 515,803 pregnant women's experience with, exposure to, infection with COVID-19 in the following studies (Figure 2): China (N = 544) [20]; Poland, the U.S., Germany, and Israel (N = 4451) [21]; the U.S. (N = 788) [23]; Israel (N = 336) [30]; Turkey (N = 260) [24], (N = 63) [25], and (N = 223) [31]; Canada (N = 1987) [26]; Italy (N = 100) [27] and (N = 101) [34]; Iran (N = 205) [28]; Colombia (N = 946) [29]; Ireland (N = 473,000) [32]; Denmark (N = 31,180) [33]; Poland (N = 500) [22] and (N = 1119) [35].

#### *3.2. Quality of Included Studies*

A qualitative analysis was conducted with necessary reciprocal translations for interpreting the evaluation score of 16 articles out of 3280 published studies, outlining the outcomes of control studies, meta-analysis, cross-sectional studies, face-to-face evaluation survey studies, remotely administered survey studies, and observational studies regarding the main topic, which were then evaluated. Overall, 75% (12/16) of the studies evaluated using the Newcastle–Ottawa scale had an overall low risk of bias. The risk of bias for medium in three of the 16 studies (18.75%), and high in one (6.25%). Our study analyzes the presence of adverse mental health symptoms in pregnant women linked to the COVID-19 pandemic, and allows for us to divide and describe findings in five domains as follows.

**Figure 2.** A total number of 515, 803 pregnant participants by the included sixteen studies.

#### *3.3. Domain 1: Social and Medical Consequences of COVID-19 in Pregnant Women Population*

Pregnant women are worried about the COVID-19 pandemic and have difficulty in accessing professional medical help; feel insecure about exposure risk to the coronavirus when accessing medical facilities; infection of the infant in the peripartum period; financial problems lead to conflict in the family; a single mother may be more prone to anxiety isolation, fear of being trapped and rumors spreading on social media, growing anxiety and social panic; and fear of blame, guilt and stigmatization related to being infected with COVID-19 [22]. A fear of infection, anger, and confusion related to financial status, ennui, inadequate information and supplies, and quarantine duration increased posttraumatic stress disorder (PTSD) symptoms and feelings of vulnerability [26,37–41], and all undeniably badly affect the emotional and mental statuses of pregnant women. Pregnant women are more likely to develop mental health problems than nonpregnant women of childbearing age are, and this was confirmed on the basis of a cross-sectional study of pregnant (*n* = 544) and nonpregnant (*n* = 315) women in Beijing, China [20]. The findings about the suppression of immune system function in pregnant women allow for us to classify the population of pregnant women into a group at high risk of SARS-CoV-2 infection [42]; on other hand, this population does not appear to be at increased risk of contracting SARS-CoV-2, since there is scientific confirmation that their rates of infection seem to parallel the rates of infection in their surrounding communities [43]. Women are more vulnerable and arguably less resistant to viral infection during the SARS-CoV-2 pandemic, as women and younger convalescent COVID-19 plasma donors are more likely to lack measurable neutralizing antibodies, while higher antibody levels were observed in men, older donors, and those who had been hospitalized [44]. Maternal risk factors associated with severe COVID-19 and admission to an intensive care unit are increasing age, high body mass, any preexisting maternal comorbidity, chronic hypertension, preeclampsia, preexisting diabetes, and both non-White ethnicity and high body mass index, are maternal risk factors for death and the need for invasive ventilation [8]. Unfortunately,

evidence now exists for the maternal–fetal transmission of SARS-CoV-2, and N proteins were strongly expressed in the placenta of a COVID-19-positive pregnant woman whose newborn tested positive for viral RNA and developed COVID-19 pneumonia soon after birth [34]. The intrauterine transmission of SARS-CoV-2 appears to be rare, and this is possibly related to low levels of SARS-CoV-2 viremia and a decreased co-expression of ACE2 and TMPRSS2 needed for SARS-CoV-2 entry into the cells in the placenta [45]. The mode and timing of delivery should be individualized based on the severity of disease, existing comorbidities, and obstetric indications [46]. However, early cord clamping may minimize the risk of viral transmission by avoiding longer, close contact with the infected mother [46], and in turn, it may increase the mother's good feelings about the safety of the newborn.

#### *3.4. Domain 2: Main Psychological Factors Responsible for Adverse COVID-19 Mental Health Outcomes in Pregnant Women*

Scientific publications reported that it is unclear whether and how SARS-CoV-2 can be transmitted from the mother to the fetus [47] and whether vertical transmission or placental pathology might occur following maternal infection during pregnancy remains unknown [48]. Women experience elevated levels of stress related to being worried about perinatal infection [35]. Fears of perinatal COVID-19 infection (29.1% of 4451 pregnant women in the U.S.) and feeling unprepared for birth due to the COVID-19 pandemic (nearly 30% of 4451 pregnant women in the U.S.) are noted as two major pandemic-related stressor domains for pregnant women in Poland, the U.S., Germany, and Israel [13]. We are faced with a paradox, because quarantine in the COVID-19 pandemic exacerbates the stress of isolation in pregnant women when a social connection helps us to cope with stress and maintain resilience [49].

#### *3.5. Domain 3: Determination of COVID-19 Mental Health Problems Including Symptoms and Diagnosis among Pregnant Women in Different Countries*

Even though 3–5% of the general population is affected by anxiety symptoms, a severe stressor such as the COVID-19 pandemic can increase such a rate in the general population of pregnant women if the prevalence of gestational anxiety is between 15% and 23%, as confirmed by scientific data [22]. COVID-19 pandemic-related stress predicts heightened anxiety in pregnant women during this crisis [23]. A preliminary study in Turkey on the effects of the COVID-19 pandemic on anxiety and depressive symptoms in pregnant women confirmed such a fact in 35.4% of the population; the obtained scores were higher than 13 on the Edinburgh Postpartum Depression Scale (EPDS) among respondents (*n* = 260) [24]; a score of more than 10 on the EPDS suggests that minor or major depression may be present [50]. The level of anxiety and depressive symptoms in pregnant women during COVID-19 infection significantly increased before such an event, and the pandemic outbreak leads to adverse birth outcomes [25]. However, a Canadian study indicated that pregnant women did not have higher levels of depression, stress, and anxiety prior the COVID-19 outbreak when compared with the time after the outbreak outcomes [51]. A study on elevated depressive and anxiety symptoms among pregnant individuals during the COVID-19 pandemic in Canada were surveyed, and its results provided information about substantially elevated anxiety and depressive symptoms compared to those in similar pre-pandemic pregnancy cohorts, with 37% reporting clinically relevant symptoms of depression and 57% reporting clinically relevant symptoms of anxiety; it also indicated stressor concerns about not receiving the necessary prenatal care, having relationship strain and social isolation due to the COVID-19 pandemic; more physical activity was associated with lower psychological symptoms [26]. A study conducted in Naples, Italy highlighted that participants who had never had depression or anxiety in previous pregnancy rated the psychological impact of the COVID-19 outbreak as severe (over 50%), especially in the first trimester of pregnancy during the outbreak, reported higher than normal anxiety in general (66%), and high anxiety regarding the vertical transmission of the disease was reported by almost half of the respondents [27].

An online descriptive–analytical cross-sectional study during the outbreak of COVID-19 with the DASS-21 tool (Depression, Anxiety and Stress Scale 21, which includes 21 questions and three subscales of stress, depression, and anxiety, and contains seven questions for each subscale, scored from not at all (0) to very high (3) for each question with 205 pregnant women in Iran, produced mean (SD) scores of depression, stress, and anxiety of 3.91 (3.9), 6.22 (4.25), and 3.79 (3.39); where depression, stress, and anxiety symptoms were observed in 32.7%, 32.7%, and 43.9% of the participants, from a mild to very severe degree [28]. In 2020, the clinical impact, psychological effects, and knowledge of pregnant women during the COVID-19 outbreak in seven cities in Colombia were evaluated, where 49.1% of women reported suffering from insomnia [29]. In comparison with the Turkish study [24], findings show that a lower rate of depressive symptoms (25%) was reported in Colombian work [29], while the same Colombian study confirmed a similar anxiety symptom rate (50.4%) with findings by Italian scientists [27], and higher than that in Iran's study results [28]. A study on distress and anxiety among Jewish and Arab pregnant women in Israel in three domains on levels of all COVID-19-related anxieties were quite high, especially in Arab women, with concern over the health of the fetus, public transportation and place, being infected themselves, and the delivery of the baby [30]. In face-to-face evaluation surveys, EPDS and the Maternal Attachment Inventory were adopted in a study (conducted in COVID-19 pandemic referral hospital in Ankara, Turkey) and applied within 48 h after birth of the patients regarding the effect of the COVID-19 pandemic and social restrictions on depression rates and maternal attachment in immediate postpartum women; the study result highlighted that the rate of depression was twice lower in comparison with the reported findings of a preliminary study on depression and anxiety symptoms in pregnant Turkish women (35.4% rate of women had EPDS scores higher than 13) by Durankus and Aksu [24]; depressive symptoms of new mothers may have been positively impacted by appropriate isolation in hospitals for pregnant women, so the feeling of being safe and isolated may be the cause of better EPDS scores [31].

#### *3.6. Domain 4: Influence of COVID-19 on Psychological and Medical Factors Related to Adverse Pregnancy and Offspring Development Outcomes*

Influence of COVID-19 on medical factors may be linked to adverse pregnancy and offspring development outcomes. Hyperinflammation as a symptom of SARS-CoV-2 reported in pregnant women suffering from SARS-CoV-2 infection, and cytokine storms could hypothetically adversely increase the risk for neurodevelopmental disorders in the neonates [3]. In cases of acute respiratory syndrome coronavirus (SARS-CoV) and MERS-CoV, high incidences of spontaneous miscarriage, preterm birth, and neonatal death have been reported [52,53]. However, a study was performed in China on the effects of SARS-CoV-2 infection during pregnancy (*n* = 35) versus nonpregnant patients, and its findings on the clinical symptoms of COVID-19 in pregnant women were misleading and atypical (compared with those in 31 nonpregnant patients), with a significantly lower proportion of fever (54.8% vs. 87.5%, *p* = 0.006), a shorter average interval from onset to hospitalization, and a higher proportion of severe or critical COVID-19 (32.3% vs. 11.4%, *p* = 0.039) [54].

The influence of COVID-19 on psychological factors may be related to adverse pregnancy and offspring development outcomes, too. There is a clear link between poor mental health in pregnant women and pregnancy complications [10]. The exposure of pregnant women to psychological distress, even linked with depression and stress in general or anxiety, is a risk factor of schizophrenia spectrum disorders, autism spectrum disorder (ASD), antisocial behavior, and attention deficit hyperactivity disorder (ADHD) [55,56]; thus, the psychological stress of the COVID-19 pandemic during pregnancy can increase the risk of neurodevelopmental disorders in offspring [57], and the risk preterm delivery, low birth weight, and postnatal complications [58]. The COVID-19 lockdown may have adverse birth weight outcomes. An unprecedented reduction in births of very low birth weights (VLBW) and extremely low birth weights (ELBW) infants was reported in the general population during the COVID-19 lockdown in Ireland [32] and Denmark [33].

#### *3.7. Domain 5: Dilemmas and Hopes in Ways to Improve the Provision of Services to Pregnant Women during Disasters, Including the COVID-19 Pandemic*

Providing support and care in psychopsychiatric services for pregnant women is a priority. It may be difficult during the COVID-19 pandemic if a mental health provider system is lacking and healthcare professionals are not trained in the field of mental health, as was confirmed in the case of Kashmir, India [59]. In survey research conducted in Belgium, only 38.2% of women received, and 61.8% of respondents did not receive, medical help from their obstetrician because of the COVID-19 pandemic [60]. The scope of psychological and psychiatric services for pregnant women is also changing in favor of telemedicine [23] to have remote access to a psychiatrist and provide psychological therapy for pregnant women and even continue such service after the delivery of a baby. This option is recommended, especially during the COVID-19 pandemic, by the American College of Obstetricians and Gynecologists (ACOG) and the Society for Maternal and Fetal Medicine (SMFM) [61,62]. Research conducted in Poland demonstrated that 47.41% of women had at least one telehealth appointment during their pregnancy, which is a much higher percentage rate than 31.8% of telehealth appointments in New York during the COVID-19 pandemic [63]. The same study concluded that it is preferable to recommend a hybrid healthcare model over a traditional in-person care model, and patient testing and screening should be performed in person, while follow-up visits can be carried out via telehealth, and the recommended model allows for providers to lower the risk of COVID-19 infection while maintaining a high standard of prenatal care [63]. The education of midwives is a high priority to address the effects of a life-threatening mass disaster event, the health of a pregnant woman and her fetus, the course of her pregnancy and delivery, the methods of prevention and treatment, and extending professional authorizations under the midwife license in the face of terrorism and/or mass disaster [64].

#### **4. Discussion**

The COVID-19 pandemic formed a serious multi-etiological global mental health challenge influencing every aspect of life and disrupting the social fabric [65]. The COVID-19 pandemic has caused general anxiety worldwide [66] and created numerous stressful conditions, especially for vulnerable populations such as pregnant women [35]; in some cases, the increased the risk of psychological imbalance associated with changes in everyday life [67] and increased the risk of the most common mental disorders among pregnant women during this a worldwide disaster [68], including anxiety, depression, insomnia, and PTSD [20,22–24,29]. The aim of this study was to analyze adverse mental health effects in the pregnant population during the COVID-19 pandemic, investigate risk factors for adverse mental health outcomes, identify protective factors, and create practical implications for clinical practice, bearing in mind the need to improve perinatal mental healthcare during such pandemics. Qualitative research was conducted in the electronic databases PubMed and Web of Sciences for the keywords COVID-19, pregnancy, depression, anxiety, and telemedicine for relevant critical articles (*n* = 3280) published from 2020 until October 2021, outlining the outcomes of control studies, meta-analysis, cross-sectional studies, faceto-face evaluation survey studies, remotely administered survey studies, and observational studies regarding the main topic; all were evaluated. The qualitative synthesis findings were concluded on the basis of 16 articles out of 3280 researched publications. Our findings on a total number of 515,803 pregnant participants are linked to adverse mental health sequelae of the COVID-19 pandemic in the pregnant population. Our study analyzes the presence of adverse mental health symptoms in pregnant women associated with the COVID-19 pandemic and allows us to divide and describe findings in five domains: Domain 1: Social and medical consequences of COVID-19 in the pregnant women population. Domain 2: Main psychological factors responsible for adverse COVID-19 mental health outcomes in pregnant women. Domain 3: Determination of COVID-19 mental health problems, including symptoms and diagnosis among pregnant women in different countries. Domain 4: Influence of COVID-19 on the psychological and medical factors related

to adverse pregnancy and offspring development outcomes. Domain 5: Dilemmas and hopes in ways to improve the provision of services to pregnant women during disasters, including the COVID-19 pandemic. Pregnant women are more likely to develop mental health problems than nonpregnant women of childbearing age are, and this was confirmed on the basis of a cross-sectional study of pregnant (*n* = 544) and nonpregnant (*n* = 315) women in Beijing, China [20]. Elevated anxiety and depression symptoms that may have a long-term impact on offspring related to COVID-19 worries about threats to their own lives and their baby's health [23], not getting enough prenatal care, and social isolation were the most frequently reported factors in the population of 1987 women <35 weeks gestation [26]. Evidence from 38.5% of participants reported high preparedness stress; 26% reported high perinatal infection stress, and pregnant women are most vulnerable to pandemic-related stress [35]. Pregnant women have an advantage of facing mental problems related to COVID-19, showing fewer depression, anxiety, insomnia, and PTSD symptoms than nonpregnant women do [20]. However, the effects of the COVID-19 pandemic on mental health are still not fully understood; thus, research in this area is still being conducted in many countries—among others, in Turkey, Colombia, and Italy. In comparison with the Turkish study [24], findings show that a lower rate of depressive symptoms (25%) was reported in Colombia [29], while the same Colombian study confirmed similar anxiety symptom rate (50.4%) with findings pointed out by Italian scientists [27] and higher than that in Iran's study results [28]. We noticed two major pandemic-related stress domains for pregnant women: fear of perinatal COVID-19 infection and being unprepared for birth were reported in four countries [21]. Pregnant women are worried about the COVID-19 pandemic and have difficulty in accessing professional medical help, feel insecure about the exposure risk to the coronavirus when accessing medical facilities, fear infection of the infant in the peripartum period, and have financial problems that lead to conflict in the family; additionally, a single mother may be more prone to anxiety isolation; fear of being trapped and rumors spreading on social media; growing anxiety and social panic; and fear of blame, guilt, and stigmatization related to being infected with COVID-19 [22]. Psychological impact and anxiety of the COVID-19 epidemic were found be more severe in women who were in the first trimester of pregnancy during the outbreak; high anxiety regarding the vertical transmission of the disease was reported by almost half of the respondents [27]. Notable, the rate of psychological consequences of the pandemic was much larger than the number of patients clinically affected by the virus, with symptoms of anxiety, insomnia, and depression [29]. Depressive and anxiety symptoms were significantly increased during the SARS-CoV-2 pandemic compared with pre-pandemic surveys, and effective screening strategies for depression and anxiety symptoms during the pandemic should be prioritized to allow for timely treatment [25]. Lockdowns (e.g., reduced infection load and reduced physical activity) are possibly beneficial for reducing extreme prematurity and potentially reducing infant mortality, a nonsignificant but slightly increased number of very premature births [33]. Evidence from pregnant women in China found that low levels of physical activity were associated with increased symptoms of depression during COVID-19, and in Canada, pregnant women who engaged in higher levels of physical activity tended to show lower symptoms of depression and anxiety [69]. The pregnant woman population is struggling with the COVID-19 pandemic, experiencing stressful factors such as adverse fetal outcomes and the increase in ruptured ectopic pregnancies, causing maternal deaths and stillbirths, concerns about the spread or infection of SARS-CoV-2, financial instability, limited scientific knowledge about the effects of COVID-19 on fetal wellbeing, and interruptions of prenatal care [35]. Urgent need to provide psychosocial support to the pregnant women population during the COVID-19 crisis was highlighted. Adverse events may otherwise occur during pregnancy and thus affect both mother and fetus [24].

Our study produced five overarching domains by analysis and synthesis of the current knowledge on the main topic related to adverse consequences of the COVID-19 on the mental health of pregnant women and the possibility of improving these outcomes in a medical provider practice in a way to reach the preparation of useful implications for clinical practice during the COVID-19 pandemic.

#### *4.1. Useful Implications for Clinical Practice*

Research findings support and advocate the need to modify the scope of healthcare provider practice in the COVID-19 pandemic disaster and could be implemented and adopted by healthcare providers, as useful implications for clinical practice on the following discoveries:


#### *4.2. Strengths and Limitations*

The strength of this study is that it was conducted with the use of and following up the PRISMA extension for scoping review guidelines. The limitations of this study may have resulted from the inclusion criterion of researching articles only written in English language, which could have excluded valuable scientific publications in other languages.

#### **5. Conclusions**

Our study analyzed the presence of adverse mental health symptoms in a total number of 515,803 pregnant women's experience with exposure to or infection with COVID-19. The psychological impact and anxiety of the COVID-19 crisis were found be more severe in women who were in the first trimester of pregnancy during the disaster. Depression symptoms worries about threats to their own lives, their baby's health, not getting enough prenatal care, and social isolation are some of the most important factors in group of <35 weeks gestation age; 26% pregnant women reported perinatal COVID-19 infection stress and being unprepared for birth. Depressive and anxiety symptoms were significantly increased during the SARS-CoV-2 pandemic compared with pre-pandemic surveys.

Urgent need to provide psychosocial support to this population during the crisis was highlighted and allowed us to produce the useful implications for clinical practice bearing in mind adverse mental health effects in the pregnant population during the COVID-19 pandemic and following our efforts to investigate risk factors for adverse mental health outcomes, identify protective factors and the need to improve perinatal mental healthcare during such pandemics. The useful implications for clinical practice to improve the adverse mental health outcomes of pregnant women associated with the COVID-19 pandemic are highly desirable. This is necessary to not only eliminate the long-term consequences of different forms of severe stressors' outcomes but also to reduce the economic costs of medically treating adverse severe mental problems. Improving the effectiveness of midwifery and nursing practice for such disaster responses is pivotal. To address adverse mental health issues during the pandemic, it is necessary to adopt several psychosocial interventions to reduce anxiety and other destructive mental health outcomes among the population of pregnant women with methods for controlling outbreaks like COVID-19. It is urgent to conduct cohort studies of the impact of the COVID-19 pandemic on the mental state of pregnant women, fetuses, and child development outside the mother's womb.

**Supplementary Materials:** The following supporting information can be downloaded at https: //www.mdpi.com/article/10.3390/jcm11082072/s1: NEWCASTLE—OTTAWA QUALITY ASSESS-MENT SCALE.

**Author Contributions:** Conceptualization, D.W.M.; methodology, D.W.M. and J.S.; formal analysis, D.W.M. and D.I.P.; resources, D.W.M., J.S. and D.I.P.; writing—original draft preparation, D.W.M. and J.S.; writing—review and editing, D.W.M. and D.I.P.; visualization, D.W.M., J.S. and D.I.P.; and supervision, D.W.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Impact of COVID-19 Related Maternal Stress on Fetal Brain Development: A Multimodal MRI Study**

**Vidya Rajagopalan 1,\*, William T. Reynolds <sup>2</sup> , Jeremy Zepeda <sup>3</sup> , Jeraldine Lopez 4, Skorn Ponrartana 5, John Wood 6, Rafael Ceschin <sup>7</sup> and Ashok Panigrahy <sup>8</sup>**


**Abstract:** Background: Disruptions in perinatal care and support due to the COVID-19 pandemic was an unprecedented but significant stressor among pregnant women. Various neurostructural differences have been re-ported among fetuses and infants born during the pandemic compared to prepandemic counterparts. The relationship between maternal stress due to pandemic related disruptions and fetal brain is yet unexamined. Methods: Pregnant participants with healthy pregnancies were prospectively recruited in 2020–2022 in the greater Los Angeles Area. Participants completed multiple self-report assessments for experiences of pandemic related disruptions, perceived stress, and coping behaviors and underwent fetal MRI. Maternal perceived stress exposures were correlated with quantitative multimodal MRI measures of fetal brain development using multivariate models. Results: Increased maternal perception of pandemic related stress positively correlated with normalized fetal brainstem volume (suggesting accelerated brainstem maturation). In contrast, increased maternal perception of pandemic related stress correlated with reduced global fetal brain temporal functional variance (suggesting reduced functional connectivity). Conclusions: We report alterations in fetal brainstem structure and global functional fetal brain activity associated with increased maternal stress due to pandemic related disruptions, suggesting altered fetal programming. Long term follow-up studies are required to better understand the sequalae of these early multi-modal brain disruptions among infants born during the COVID-19 pandemic.

**Keywords:** fetal brain function; maternal stress; COVID-19 pandemic

#### **1. Introduction**

The COVID-19 pandemic created many, unprecedented disruptions to everyday life particularly in 2020–2022 before vaccines were widespread. In addition to disruptions around employment, childcare, housing, and nutrition, pregnant women also suffered negative experiences related to support and care during pregnancy and childbirth. Social isolation, reduced access to child and elder care, COVID-19 infection risk, and changes to medical policies around pre and postpartum care were reported to be the most common stressors among pregnant women [1,2]. Pregnant women are particularly vulnerable to mood and anxiety related disorders [3] which are exacerbated during natural disasters or stressful events [4,5]. Unsurprisingly, pregnant women indicated elevated levels of stress

**Citation:** Rajagopalan, V.; Reynolds, W.T.; Zepeda, J.; Lopez, J.; Ponrartana, S.; Wood, J.; Ceschin, R.; Panigrahy, A. Impact of COVID-19 Related Maternal Stress on Fetal Brain Development: A Multimodal MRI Study. *J. Clin. Med.* **2022**, *11*, 6635. https://doi.org/10.3390/ jcm11226635

Academic Editor: Alfonso Troisi

Received: 10 October 2022 Accepted: 5 November 2022 Published: 9 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

during the COVID-19 pandemic [6]. In addition to health consequences for the mother, increased maternal stress has an intergenerational impact on fetal development [7,8]. Increased maternal stress during pregnancy is known to alter the fetal brain and adversely impact postnatal neurodevelopmental outcomes [9–12].

Studies of infants born during the COVID-19 pandemic have reported reduced cognitive, motor, and emotional development compared to those born pre-pandemic [7,8], with increased prenatal stress directly associated with adverse affect and temperament [13,14]. Simultaneously, changes to brain structure and function have also been reported in infants born during the pandemic [15]. Lu et al. [16] reported volumetric reductions in the brain among fetuses of women pregnant during the pandemic compared to a pre-pandemic cohort. Their findings showed a negative relationship between general maternal stress and fetal brain volumes. However, their cohort did not show an increase in maternal stress or anxiety during a pandemic, and they did not measure maternal stress or anxiety specifically linked to the pandemic. Additionally, there is no data on if or how emerging functional networks in the fetal brain, which are known to be sensitive to ma-ternal stress, were impacted by pandemic related maternal stress. Early aberrations to functional organization of the brain are well known to have deleterious downstream effects in brain and behavioral development. As such, a multimodal imaging study is important to better understand how prenatal maternal stress sets up the offspring's brain for a trajectory of compounding aberrant development.

Understanding the impact of pandemic related maternal stress on fetal development allows us to identify risk and resilience factors to mitigate maternal stress and consequently minimize the intergenerational effect of pandemic related stress. Coping behaviors, in response to stressful events, are known to be modifiable targets to mitigate maternal stress and anxiety [17,18]. Given the extraordinary nature of pandemic related stressors, there is little information on various coping behaviors that pregnant women have adopted during the pandemic [19–21]. Despite its observational nature, information on coping behaviors to pandemic related stressors allow clinical care teams to design and implement support programs aimed at improving maternal mental health during pregnancy and child outcomes.

In this work, we investigated the impact of maternal stress due to pandemic related disruptions in pregnancy support and care on structural and functional development of the human fetal brain. Our primary hypothesis is that increased maternal stress would predict quantitative alterations in structural and functional characteristics of the fetal brain. Secondarily, we compared coping behaviors between pregnant women reporting high vs. low levels of pandemic related stress.

#### **2. Materials and Methods**

#### *2.1. Subject Demographics*

Pregnant mothers, living in the greater Los Angeles area were recruited using flyers, social media ads, and referrals from community partner clinics at Children's Hospital Los Angeles (CHLA) from November 2020–November 2021. Enrollment eligibility included healthy, pregnant women between 18–45 years with singleton, uncomplicated pregnancies (confirmed by ultrasound) between 21–38 gestational weeks (GW). Exclusion criteria were multiple gestation, fetal or genetic anomalies, congenital infection, and maternal contraindication to MRI. Informed consent for the study was obtained under a protocol approved by the Institutional Review Board at CHLA. Demographics, perinatal health history, and self-assessment surveys of consented participants were gathered via online survey within 24 h prior to MRI.

#### *2.2. Stress and Coping Behavioral Assessments*

Participants were asked to complete the Coronavirus Perinatal Experiences-Impact Survey [22] (COPE-IS). This is a self-assessment questionnaire, available in multiple languages, to assess feelings and experiences of pregnant women and new mothers in relation

to disruptions caused by the COVID-19 pandemic. Questions in this assessment were adapted from multiple validated questionnaires such as the Brief Symptom Inventory [23] PTSD checklist from DSM-5 [24], and the Johns Hopkins Mental Health Working Group. In this study, we only included questions pertinent to the prenatal period. Perceived maternal stress was computed as described here [21,22] and will be referred to as COPE-Stress going forward. Participants also completed the Brief COPE questionnaire [25], which is an abbreviated form of the COPE (Coping Orientation to Problems Exposed) questionnaire [26]. This is a self-assessment of a wide range of coping behaviors including both maladaptive coping (includes substance use, venting, behavioral disengagement, denial, self-blame, and self-distraction) [27] and adaptive coping (includes humor, planning and seeking social support, use of emotional and instrumental support, positive reframing, religion, and acceptance) [26,28]. This questionnaire has been validated in multiple languages and cultural contexts to be correlated to perceived stress and mental well-being.

#### *2.3. Child Opportunity Index (COI)*

Neighborhood socio-economic environment (SEE) is a known modifier of overall maternal stress during pregnancy [29], pandemic related stress [30], and offspring outcomes [31]. Family income is often used to measure SEE. However, the quality of life associated with absolute income number varies regionally based on cost of living, social policies, environmental factors, etc. To overcome these limitations, we chose to represent SEE using childhood opportunity index (COI). COI is a multi-dimensional, nationally normed measure of the quality of social, environmental, health, and educational resources available at each zip code [32]. We extracted maternal COI using self-reported zip code at the time of the MRI visit and will be referred to as COI-SEE going forward.

#### *2.4. Image Acquisition*

Pregnant mothers were prospectively recruited between 24–38 GW and imaged on 3.0 T Philips Ingenia scanner (Philips Healthcare, Best, The Netherlands). Multiplanar single-shot turbo spin echo imaging was per-formed (TE = 160 ms, TR = 9000–12,000 ms, 3 mm slice thickness, no interslice gap, 1 × 1 mm in plane resolution). Fetal brains were scanned in each of three planes for three times resulting in nine images per subject and images were repeated if excessive motion was present. Echo-planar imaging (EPI) BOLD images were also collected with the following parameters: FOV = 300 mm TR = 2000 ms, TE = 31–35 ms (set to shortest), flip angle = 80◦, with an in-plane resolution of 3 × 3 mm2, slice thickness of 3.0 mm and 0.0 mm intra-slice gap. 150 timepoints were recorded for each BOLD image and two images were collected for each subject.

#### *2.5. Image Processing*

#### 2.5.1. Brain Structure

All structural brain images were verified as being typical for gestational age by a board certified neuroradiologist (SP). For each subject, various 2D stacks of the T2 images were visually assessed to identify and discard stacks with large, spontaneous fetal motion. In each stack, the fetal brain was localized from surrounding tissue. For each subject, multiple 2D stacks were motion corrected and reconstructed, using a slice-to-volume reconstruction [33] into a 3D volumetric T2 image with an isotropic resolution of 1 mm3. Reconstructed fetal brains were processed through a bespoke, automated fetal segmentation pipeline. Each fetal brain was normalized (affine followed by non-rigid) to a probabilistic atlas [34] of equivalent gestational age using Advanced Normalization tools [35]. Segmentations were manually inspected for accuracy and subjects with failed segmentations were discarded. The resulting segmentation maps were subsequently refined. To ensure consistency across different gestational ages, transient structures only present in the tissue atlas from 21–30 weeks of gestation such as the subplate, intermediate zone, and ventricular zone were combined with the corpus collosum and labeled as developing WM (WM). Cerebrospinal fluid (CSF) segmentation was refined as intra-ventricular (within lateral ventricles) and extra-axial CSF.

Due to the small size and relative difficulty in segmenting the hippocampus and amygdala, both structures were combined into a hippocampus-amygdala complex. Deep grey tissue was defined as the combination of the caudate, putamen, thalamus, fornix, internal capsule, subthalamic nucleus, and hippocampal commissure. Right and left hemispheric labels were combined into a single volume for each structure. The final segmentation yielded volumes of the following structures: cortical plate, developing white matter, intra-ventricular CSF, extra-axial CSF, deep gray tis-sues, cerebellum, hippocampal-amygdala complex, and brainstem. A total brain volume (TBV) was generated for each subject as the sum of all tissues.

#### 2.5.2. Brain Function

BOLD imaging of the fetal brain is prone to spontaneous fetal motion which is compounded by lower signal to noise ratio and spatial resolution. While modern motion correction algorithms effectively attenuate the effects of subject motion on the temporal data, they are limited in effect beyond small degrees of motion. Any robust voxel-wise approach to functional fetal imaging would yield a prohibitively low number of subjects with usable data. We therefore chose to implement a whole-brain temporal signal approach to fetal functional imaging. Resting state images were first motion corrected using FSL's MCFLIRT routine, using the first frame as the registration target, and a mean framewise displacement threshold >0.2 mm to eliminate frames with excessive motion. As the intent of this study was to use minimally processed data using framewise measures, as opposed to voxel wise measures, we made no prior assumptions on physiological or nuisance frequency thresholds in fetal functional imaging and did not apply any bandpass filtering. A mean brain signal image was then generated by averaging across every frame in the sequence. This mean signal image was used as the source image for brain extraction to generate a brain mask. Brain extraction was done by using an adaptive routine that iterated between using FSL's Brain Extraction Tool (BET) [36] and AFNI's Skullstrip, using decreasingly smaller thresholds for brain tissue [37]. This approach yielded a good approximation of the fetal brain, with a minimal manual correction step required for final brain masking. The brain mask was then propagated across each frame in the temporal sequence to extract only fetal brain voxels.

Using the mask generated above, we averaged the whole brain BOLD signal in each frame and generated statistical measures across time. The measures generated were temporal mean (average of the mean signal across frames), temporal variability (average of the standard deviation of the signal across frames), variance of the mean (variance of the mean signal in each frame), kurtosis of the mean (kurtosis of the mean signal in each frame). Finally, to assess for any signal or physiological drift, we calculated the autocorrelation of the mean signal in each frame, and the kurtosis and autocorrelation of the normalized signal across frames.

#### *2.6. Statistical Analysis*

#### 2.6.1. Brain Structure

Regression analysis was performed in Python (3.7) using the Statsmodel.api v0.13.2. We used multiple, linear regression to model the relationship of COPE-Stress Score, COI-SEE, and their interaction on TBV after adjusting for gestational age at MRI. Nested models of the covariates without interaction were also evaluated. Models were deemed to be significant if one or more of the covariates were statistically significant, and models including the interaction term were only selected over the simpler counterpart if they had a higher explained variance (R-squared) and/or lower Bayes' Information Criteria (BIC). Using similar regression models, we individually assessed the relationship of COPE-Stress score and COI-SEE for each tissue volume listed in Section 2.5.1 (as a dependent variable). Secondarily, we also evaluated the relationship of COPE-Stress score and COI-SEE on tissue volumes normalized by TBV after adjusting for gestational age.

#### 2.6.2. Brain Function

Statistical analysis for brain functional metrics was similar to Section 2.5.1. A separate regression model was evaluated for each, individual functional metric (Section 2.5.2) with COPE Stress, COI-SEE, and their interaction as predictor variables after accounting for GA at MRI.

#### 2.6.3. Comparison of Coping Behaviors

Coping behaviors, both the Brief-COPE and COVID specific, were analyzed for differences between low and high stress mothers. Mothers were split into low, medium, and high stress categories based on tertiles of COVID Stress scores. Using Fischer Exact test, we compared if mothers reporting low and high stress used each coping behavior at significantly different amounts.

#### **3. Results**

#### *3.1. Subject Demographics*

Pregnant mothers were recruited prospectively for this study with a total of 45 motherfetus dyads completed the MR imaging session. Three subjects had missing zip code information, and which resulted in missing COI-SEE data and was thus excluded from any analysis. After imaging, three subjects failed brain segmentation resulting in 39 subjects for structural regression results. A total of 43 subjects of the original 45 subjects had analyzable BOLD imaging and were used for the functional regression results (Table 1).



#### *3.2. Brain Structure*

There were no significant associations between absolute volumes of the various brain structures and perceived maternal stress, COI-SEE, or their interaction (Table 2). However, there was a significant positive association between normalized brain stem volume and perceived maternal stress (*p* = 0.03) but not with COI-SEE and the interaction of COI-SEE and maternal stress (Table 3) There were no significant associations between normalized volumes of other structures with COPE-Stress or COI-SEE.


**Table 2.** Raw brain structure volumes relationship to COVID stress and COI-SEE.

**Table 3.** Brain structure volumes', after normalization to total brain volume, relationship to COVID stress, COI-SEE, and their interaction. \* denotes statistically significant relationships.



#### *3.3. Brain Function*

Lack of significant relationship between autocorrelation metrics and the predictor variable confirmed the absence of any systematic signal or physiological drifts. We found a significant negative relationship between temporal variability and COPE Stress (*p* < 0.028) (Table 4). The temporal variability model including the interaction term between Cope Stress Score and COI SES had a slightly improved R-squared (0.267) but lower BIC and reduced statistical significance of the covariates, likely due to co-linearity. We there-fore report the original model without the interaction term. We found no other statistically significant relationships between fetal brain functional characteristics with COPE Stress or COI SEE.

**Table 4.** Brain functional metrics' relationship to COVID stress and COI-SEE using linear modeling. \* denotes statistically significant relationships.


#### *3.4. Comparison of Coping Behaviors*

We compared coping behaviors between participants reporting high and low stress in our cohort. Figure 1 shows the prevalence of use of various coping behaviors, reported as percentage of total, among the participant in the study. Humor (*p*-value = 0.025) and

venting (*p*-value = 0.048) were used more commonly by participants reporting low stress compared to those reporting high stress (Figure 1). We observed differential patterns of coping behavior use among mothers who reported high and low stress. Figure 2 shows, self-reported importance levels of potential resources for management of stress associated with COVID-19 related disruptions among pregnant women in the study. Access to a mental health provider (*p*-value = 0.038), and information about how to reduce stress (*p*-value = 0.038) were chosen as being 'Very Important' to women reporting low stress at a high amount than in women reporting high stress (Figure 2). No other behaviors were found to be significantly different between high and low stress mothers. A full summary of the results can be seen in Figures 1 and 2.

**Figure 1.** Prevalence of use of various subscales of coping behaviors within pregnant women in the study. The first row for each subscale represents the entire cohort. The second, third, and fourth rows correspond to prevalence of coping behaviors in participants reporting high, medium, and low COVID-19 related stress. \* Denotes subscales with significantly different prevalence of use between mothers reporting high and low stress levels.

**Figure 2.** Self-reported importance of potential resources for management of stress associated with COVID-19 related disruptions among pregnant women in the study. The first row for each subscale represents the entire cohort. The second, third, and fourth rows correspond to prevalence of coping behaviors in participants reporting high, medium, and low COVID-19 related stress. \* Denotes subscales with significantly different prevalence of use between mothers reporting high and low stress levels.

#### **4. Discussion**

Our findings show that perceived maternal stress, in the setting of COVID-19 related care disruptions, impacts with structural and functional developmental of the fetal brain. Higher maternal stress was associated with increased brainstem volume (suggesting accelerated brainstem maturation) and globally decreased temporal variability of function (suggesting reduced functional connectivity) in the fetal brain. Additionally, we also found differences in the prevalence of specific coping behaviors between pregnant women who reported high stress compared to those who reported low stress. Our study is novel in the following aspects: (1) use multi-modal imaging to characterize fetal brain developmental changes due to maternal stress during COVID, and (2) characterization of adaptive coping behaviors which may provide resilience during this period of increased maternal stress.

We found that increased levels of maternal stress correlated with increased normalized brainstem volume suggesting relatively increased acceleration of brainstem maturation relative to cortical/supratentorial cerebral regions. Importantly, these results are consistent with prior studies that have correlated prenatal maternal stress and neonatal brainstem auditory evoked potentials (the speed at which the brainstem auditory evoked potential is conducted through the auditory nerve serves as a proxy for greater neural maturation) [38,39]. These studies have found significant relations between higher maternal prenatal distress and faster conductance, suggesting that greater maternal prenatal stress is associated with accelerated subcortical/brainstem neural maturation in neonates [40]. Our

results are also consistent with the recent study by De Asis-Cruz et al. [41] which found that altered functional connectivity between brainstem and sensorimotor regionals were associated with high maternal anxiety scores.

We found that higher perceived maternal stress was associated with lower temporal variability in the fetal brain suggesting aberrations to foundational characteristics of connectivity and organization of emerging brain networks [42]. It has been well-established that perturbations to early brain connectivity architecture, during the critical fetal period, has long-standing effects on behavioral and psychiatric development among these children [43–45]. Altered temporal variability of brain BOLD are known to associated with adverse neurocognitive functioning of the brain [46–48]. Our findings of altered brain connectivity agree with previous findings of altered brain connectivity in infants of mothers who reported higher stress during the pandemic [15]. Behavioral and functional deficits particularly in the motor, cognitive and temperamental domain have been widely reported in various studies investigating the impact of maternal stress during the pandemic on child outcomes [7,8,13,14]. Increased maternal stress and anxiety traits (outside the setting of the pandemic) have been shown to alter functional architecture of the fetal brain [49]. Collectively, our and prior findings suggest that in utero alterations to brain architecture, associated with maternal stress during the pandemic, could underlie developmental deficits reported in these children. Further meta studies are needed to investigate the trajectory of brain development in children conceived and born during the pandemic.

Our findings suggest key differences in coping behaviors between pregnant women who reported low and high stress. Increased use of adaptive coping behaviors (particularly humor and venting) was more common among pregnant women who reported lower stress compared to those who reported higher stress. This association between in-creased use of adaptive, active coping and lower stress perception was reported across multiple studies of mental health in peripartum women during COVID-19 pandemic [21,50,51]. Our findings are also in agreement with generalized findings of positive relationship between active coping behaviors and improved mental well-being in pregnant women [52]. In questions regarding COVID-19 specific coping behaviors, pregnant mothers reporting low stress endorsed access to mental health information and providers as being key to wellness. Routine screening for prenatal stress, provision of stress management information, and improved access to prenatal mental health care provide potential avenues for improving mental health and associated outcomes in pregnant women regardless of pandemic conditions.

This study's limitations include small sample size and recruitment limited to a single geographical area in the USA during the pandemic. Since the greater Los Angeles area was disproportionately affected by pandemic related disruptions, comparison to a multi-site cohort will provide greater statistical power thereby increasing the generalizability of our findings. The cross-sectional nature of prenatal stress assessment limits our ability to associate time-varying stress levels and fetal outcomes. However, all participating women became pregnant after pandemic-related restrictions were put in place. Lack of a pre-pandemic cohort limits our ability to pin-point if the differences in coping behaviors between pregnant women reporting low and high stress are specific adaptations to stress experienced during the pandemic. Consistent with the demographics of Los Angeles County, over 50% of the pregnant women in our study cohort identified as Hispanic. Cultural norms around coping behaviors, care and family support during pregnancy or postpartum periods should be factored into the interpretation of our findings. Future work could examine if disparities in healthcare utilization have been altered during the pandemic among pregnant women from diverse backgrounds [53,54].

#### **5. Conclusions**

Here, we reported the first multi-modal study of the impact of COVID-19 pandemic related maternal stress on fetal brain development. Our findings showed that increased maternal stress due to pandemic related disruptions was associated with structural and functional disruptions to fetal brain development and is suggestive of altered fetal programming. Comparing coping behaviors between pregnant women reporting higher and lower stress, our study provides insight into potential avenues for improved stress management and mental health outcomes among pregnant women.

**Author Contributions:** Conceptualization, V.R., R.C. and W.T.R.; methodology, V.R., R.C. and W.T.R.; validation, R.C., S.P. and W.T.R.; resources, V.R.; data curation, V.R., J.Z. and J.L.; writing—original draft preparation, V.R., R.C., W.T.R. and J.Z.; writing—review and editing, V.R., W.T.R., J.Z., J.W., R.C., S.P., J.W. and A.P.; visualization, V.R., R.C., W.T.R. and J.Z.; supervision, A.P. and V.R.; funding acquisition, V.R. and A.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was funded NIH/NHLBI K01HL153942 and The Saban Research Institute's Research Career Development Award to VR. Funding for WR was provided by the National Library of Medicine T15 Training program grant; award number 2T15LM007059-36. AP reported receiving grants from the Department of Defense (W81XWH-16-1-0613), the National Heart, Lung, and Blood Institute (R01 HL152740-1, R01 HL128818-05), and Additional Ventures.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Children's Hospital Los Angeles (protocol code CHLA-17-00292 and 9/14/2017).

**Informed Consent Statement:** Any research article describing a study involving humans Written informed consent to include deidentified data has been obtained from the patient(s) to publish this paper.

**Data Availability Statement:** Due to limitations of informed consent, data from the study cannot be shared. However, Methodologies and techniques from the study will be made available via direct email to the corresponding author.

**Acknowledgments:** We would like to acknowledge all the women and families who participated in the study during an unprecedented global pandemic. We would like to thank Rosa Rangel, Teddy Aguilar, and Veronica Gonzalez for their help with the study.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

#### **References**


## *Review* **SARS-CoV-2 and the Brain: What Do We Know about the Causality of 'Cognitive COVID?**

**Hashir Ali Awan <sup>1</sup> , Mufaddal Najmuddin Diwan <sup>1</sup> , Alifiya Aamir 1, Muneeza Ali 1, Massimo Di Giannantonio 2, Irfan Ullah <sup>3</sup> , Sheikh Shoib <sup>4</sup> and Domenico De Berardis 2,4,5,\***


**Abstract:** The second year of the COVID-19 (coronavirus disease) pandemic has seen the need to identify and assess the long-term consequences of a SARS-CoV-2 infection on an individual's overall wellbeing, including adequate cognitive functioning. 'Cognitive COVID' is an informal term coined to interchangeably refer to acute changes in cognition during COVID-19 and/or cognitive sequelae with various deficits following the infection. These may manifest as altered levels of consciousness, encephalopathy-like symptoms, delirium, and loss of various memory domains. Dysexecutive syndrome is a peculiar manifestation of 'Cognitive COVID' as well. In the previous major outbreaks of viruses like SARS-CoV, MERS-CoV and Influenza. There have been attempts to understand the underlying mechanisms describing the causality of similar symptoms following SARS-CoV-2 infection. This review, therefore, is attempting to highlight the current understanding of the various direct and indirect mechanisms, focusing on the role of neurotropism of SARS-CoV-2, the general pro-inflammatory state, and the pandemic-associated psychosocial stressors in the causality of 'Cognitive COVID.' Neurotropism is associated with various mechanisms including retrograde neuronal transmission via olfactory pathway, a general hematogenous spread, and the virus using immune cells as vectors. The high amounts of inflammation caused by COVID-19, compounded with potential intubation, are associated with a deleterious effect on the cognition as well. Finally, the pandemic's unique psychosocial impact has raised alarm due to its possible effect on cognition. Furthermore, with surfacing reports of post-COVID-vaccination cognitive impairments after vaccines containing mRNA encoding for spike glycoprotein of SARS-CoV-2, we hypothesize their causality and ways to mitigate the risk. The potential impact on the quality of life of an individual and the fact that even a minor proportion of COVID-19 cases developing cognitive impairment could be a significant burden on already overwhelmed healthcare systems across the world make it vital to gather further evidence regarding the prevalence, presentation, correlations, and causality of these events and reevaluate our approach to accommodate early identification, management, and rehabilitation of patients exhibiting cognitive symptoms.

**Keywords:** COVID-19; SARS-CoV-2; brain; neurotropism; cognitive; prevention; diagnosis

#### **1. Introduction**

Over one year since the first case surfaced in the Chinese city of Wuhan, COVID-19 has resulted in more than 3.7 million deaths globally [1]. Initially, focus was primarily on

**Citation:** Ali Awan, H.; Najmuddin Diwan, M.; Aamir, A.; Ali, M.; Di Giannantonio, M.; Ullah, I.; Shoib, S.; De Berardis, D. SARS-CoV-2 and the Brain: What Do We Know about the Causality of 'Cognitive COVID? *J. Clin. Med.* **2021**, *10*, 3441. https:// doi.org/10.3390/jcm10153441

Academic Editor: Alfonso Troisi

Received: 17 June 2021 Accepted: 28 July 2021 Published: 2 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

managing acute conditions, but the long-term consequences of SARS-CoV-2 infection are now being highlighted with time. An interesting example is the patient-coined term 'Long-COVID' [2], which denotes long-term outcomes or lasting symptoms of COVID-19 [3].

Apart from major respiratory symptoms, there are reports of acute and post-recovery cognitive deficits occurring in COVID-19 patients [4]. Some authors [5] have also coined a more generic term 'infectious disease-associated encephalopathy' to encompass neurological manifestations of both the classical and novel infections. While it is assumed to have a separate pathophysiology than encephalopathy of a non-infectious origin [5], and although evidence of central nervous system (CNS) involvement exists for the 1918 H1N1 Influenza Virus and 2002 SARS-CoV [6], there is a lack of academic evidence necessary to evaluate the causality of cognitive impairments accurately. Nevertheless, several mechanisms have been presented to explain SARS-CoV-2's acute and 'sequelae' effects [7–10] on the brain. These include viral neurotropism, widespread systemic inflammation, and psychological burden of the pandemic across the world.

These sequelae consist of cognitive impairment after COVID-19 and have also been associated with the medical interventions, especially mechanical ventilation, provided to alleviate conditions of those with severe forms of the infection, which mainly manifested as acute respiratory distress syndrome (ARDS) [11]. Moreover, the immense psychosocial strain due to the prevailing conditions, rising mortality, and government-mandated distancing mechanisms such as lockdowns [12] may also lead to psychological and cognitive consequences in the long run [6].

There has been a critical and time-sensitive [4] need to assess the cognitive impact of COVID-19 due to possible long-term implications it could have on the overall wellbeing of those surviving the infection. This review is attempting to collect the available clinical data, etiological models, and proposed recommendations currently available in the literature to highlight 'Cognitive COVID' and determine if it could change our approach in the second year of this global pandemic.

#### **2. History of Cognitive Impairment in Previous Major Coronavirus Outbreaks and Other Classical Infectious Diseases**

Before SARS-CoV-2, two coronaviruses caused significant outbreaks—the Severe Acute Respiratory Syndrome caused by SARS-CoV in 2002 [13], and the Middle Eastern Respiratory Syndrome caused by MERS-CoV in 2012 [14]. A rapid review published in 2020 highlighted the neurological manifestations of the previous coronaviruses to extrapolate the ratio and predict the number of COVID-19 patients that will potentially show neurological deficits [15].

Furthermore, the comparative pathophysiology of SARS and COVID-19 and a similar psychological strain caused by some of the disease processes and circumstances increase the likelihood that COVID-19 will present with cognitive impairments. SARS and COVID-19 both consist of extensive systemic inflammation, the level of which determines disease severity and outcomes [16]. Furthermore, a study on three MERS patients in Saudi Arabia revealed that they had altered levels of consciousness and confusion, which was correlated to new-onset changes on MRI imaging, indicating a neurological component of the viral infection [17]. Another study on 70 patients in Saudi Arabia found that a quarter of the patients (25.7%) developed confusion during the disease [18].

This link, however, expands beyond just coronaviruses. For example, multiple studies conducted on viral infections involving the Human Immunodeficiency Virus (HIV) and Zika Virus (ZIKV) have also underscored a cognitive aspect to the disease presentation with attention, memory, and learning defects [19,20]. The Influenza viruses have also been reported to affect cognition and result in a cognitive decline. Neurological manifestations of Influenza (NMI) have been reported for both global and seasonal outbreaks of the virus and have ranged from seizures to encephalopathies [21]. A study in Taiwan reported Influenza-associated encephalitis/encephalopathy (IAE) and noted that all 10 patients had different levels of consciousness disturbance on presentation [22]. The prevalence of NMI varies geographically [21] and depends on the dominant viral strain. Rao et al.

reported around 18% of all patients of Influenza A (H3N2) in Colorado (USA) had NMI during 2016–17 season [23]. On the other hand, a large national study in Malaysia showed prevalence of NMI to be 8.3% during the 2009 Influenza A (H1N1) pandemic [24]. While the risk of hospitalization is increased by NMI [23], most authors have considered a long-term sequela of such cognitive disturbances by Influenza rare [21,22].

#### **3. Brief Review of Manifestation of Acute and Long-Term Cognitive Deficits**

Cognitive deficits and impairments have a complex presentation with variable durations [15,25]. In addition, reports for both acute manifestations and long-term sequelae exist [6].

Acute decline in cognitive functions may result due to a combination of causes, including neurotropism of SARS-CoV-2 and sedation during mechanical ventilation. Encephalopathy is then cited as a general cause for the development of cognitive disturbances [26]. Early in the pandemic, a study involving 214 patients in Wuhan, China, noted CNS-related symptoms including dizziness, headache, and diminished consciousness in 24.8% of patients [27]. In April 2020, 'altered mental status' was listed as one of the 'clinical syndromes' associated with COVID-19 and defined as an 'acute alteration in personality, behavior, cognition, or consciousness' by a survey in the United Kingdom [28]. In the same survey, 31% of the patients recorded having an altered mental status following COVID-19, and nearly 5% of the total patients had dementia-like cognitive symptoms [28]. In addition, viral encephalitis has been identified in some COVID-19 patients, and it alone is possibly linked to the development of acute and lasting cognitive losses [6,29].

'Dysexecutive syndrome' is another peculiar concept that depicts cognitive defects in individuals, particularly of attention, control, and orientation loss [30]. Empirical evidence from a French study shows that loss of executive functions was reported in almost a quarter of COVID-19 patients presenting with ARDS [31]. Furthermore, there is promising evidence of even asymptomatic COVID-19 subjects scoring significantly lower in domains of visual perception, naming, and fluency when checked via the Montreal Cognitive Assessment (MoCA) test [32]. A review also noted symptoms more prevalent in older individuals and those with severe infections [6].

Apart from lasting psychiatric conditions, cognitive impairments may follow a SARS-CoV-2 infection, causing impaired memory, confusion, and attention deficits in the long term [6,33]. A study in Zhejiang, China administered multiple tests evaluating attention, memory, executive function, and information processing, checking for cognitive function of recovered COVID-19 patients against a control group, and finding the sustained attention domain significantly lesser in COVID-19 survivors [34]. Further exploring the link between hospitalization mostly with mechanical ventilation and cognitive deficits, a study using the BMET was conducted on 57 recovering patients with severe disease [4]. In total, 81% of the cohort exhibited some form of cognitive impairment; however, there was no significant correlation of such deficits with the length of intubation.

Further evidence of long-term deficits is available in two more studies [35,36]. First, Lu et al. [35] recorded data of 60 patients during acute SARS-CoV-2 infection and at a 3-month follow-up visit. The proportion of patients with memory loss more than doubled from 13.3% during the acute disease to 28.3% at the follow-up [35], demonstrating the long-term impact of COVID-19 on an individual's cognition. In addition, Woo et al. [36] investigated the cognitive status of 18 recovered patients using the Modified Telephone Interview for Cognitive Status (TICS-M). They contacted the patients at a median of 85 days following their recovery from mild or moderate COVID-19 without an ICU admission. Results showed 18 post-COVID-19 patients scoring appreciably lower than ten control patients on the cognitive assessment, with multiple other self-reported cognitive impairments, including attention deficits (50%), memory deficits (44.4%), and incoherent thoughts (5.6%) [36].

#### **4. Causality**

Owing to the nascency of the novel coronavirus causing the pandemic, the exact pathophysiology behind the cognitive sequelae has not been entirely understood. There is no clarity regarding SARS-CoV-2 *directly* affecting the brain or the symptoms resulting from the non-specific and indirect causes; for instance, systemic inflammation and medical interventions such as ventilation. Additionally, the piling psychosocial strain could also potentially act as the source of 'Cognitive COVID' [6,9].

The evidence so far is inconclusive of whether each aspect works solitarily or all elements are working together in causing symptoms. This section briefly outlines these mechanisms (also shown in Figure 1) with references to available evidence of SARS-CoV-2 and its predecessor coronaviruses.

**Figure 1.** Summary of possible causal elements in the development of cognitive symptoms during and after a COVID-19 (coronavirus disease 2019) infection.

#### *4.1. Neurotropism and the ACE2 Receptor*

While still unclear, it is hypothesized that SARS-CoV-2, similarly to other coronaviruses, can infect and survive in nervous tissue [37,38]. Although rare, evidence of SARS-CoV-2's presence in cerebrospinal fluid (CSF) [29,39], as with other viruses [40], is available. There are numerous suggested pathways by which such neurotropism occurs. However, the exact mechanism is still uncertain. Retrograde neuronal access via peripheral nerves, hematogenous spread via directly infecting endothelial cells, and infiltration of infected cells are three main explanations [7,41–43] behind how respiratory viruses (such as SARS-CoV-2) enter the CNS.

i. Olfactory invasion: There is emerging evidence of SARS-CoV-2 affecting the olfactory and gustatory sensations, producing well-known symptoms of 'loss of taste and smell' in infected individuals [44–46]. With time, evidence has surfaced supporting

the pathobiology of olfactory and gustatory dysfunction because of a direct invasion of the mucosal epithelium and olfactory bulb [47]. The invasion can potentially be attributed to their expression of the ACE2 surface receptor and Transmembrane Protease Serine 2 (TMPRSS2), cleaving the spike protein of SARS-CoV-2 and facilitating the fusion of SARS-CoV-2 with cellular membranes [48,49]. Furthermore, having a genome that is 79% similar to that of SARS-CoV, the spike glycoprotein of SARS-CoV-2 also binds to Angiotensin-Converting Enzyme 2 (ACE2) receptor on multiple organs, including the brain, acting as the viral functional receptor [50,51]. However, SARS-CoV-2 binds to ACE2 receptors with a considerably greater affinity than SARS-CoV [52]. Animal studies focusing on SARS-CoV have shown trans-neuronal spread from the olfactory bulb to certain 'connected' regions of the brain, providing key 'circumstantial evidence' in the potential neurotropic properties of SARS-CoV-2, as well [43,53]. The entorhinal cortex and the hippocampus are such 'connected' regions. They are involved in episodic memory and other domains, illustrating how damage directed at these areas may cause lasting cognitive dysfunction [54]. ii. Hematogenous spread: Some authors [8] claim hematogenous spread via the cerebral vasculature plays a more critical role in direct brain entry and damage-causing cognitive deficits in COVID-19. Evidence of SARS-CoV-2's presence in blood samples of some confirmed COVID-19 patients exists. As many as 41% [55] of the samples showed viremia [43], showcasing the ability of the virus to easily reach the brain once the blood-brain barrier (BBB) is damaged. The distribution of SARS-CoV-2's functional (ACE2) receptor is widespread in endothelial cells and pericytes throughout the body [56]. Analysis of available genomic databases confirms noteworthy expression of the receptor in neuronal and glial tissues of the CNS [56]. Consequently, the nervous tissue is potentially vulnerable if the virus comes in direct contact and interacts with the ACE2 receptors. In addition, SARS-CoV-2's potential neurotropic properties may allow it to assume latency inside neuronal tissue of patients even after recovery from COVID-19, putting them at greater risk of long-term or delayed cognitive deficits and neurological symptoms [6]. Notably, it is still unclear how abundantly ACE2 receptors are expressed in the cerebral vasculature. However, other docking receptors, importantly basigin (BSG) and neuropilin (NRP1), have been identified as facilitators of the viral entry or internalization—making the brain vulnerable to viral inflammation even with an intact BBB [7] In addition, SARS-CoV-2 and the accompanying inflammatory cytokines, including Interleukins (IL) and Tumor Necrosis Factor (TNF), may damage the BBB [57]. Moreover, evidence shows that SARS-CoV-2 affects vasculature integrity by direct viral infection, leading to endothelium damage and increased vascular permeability in peripheral vessels [58]; extrapolated from the cerebral endothelial cells, this could explain the disruption of the BBB. Therefore, immune-mediated action or direct inflammation may be responsible for endothelial dysfunction in the BBB, enabled by the recruitment of host immune cells. Additional factors that may aid in the hematogenous spread of SARS-CoV-2 to the brain include a pre-existing or underlying neurological pathology and entry via circumventricular organs such as the median eminence of the hypothalamus [7].

iii. Infiltration of infected cells: A 2005 study aimed at SARS-CoV found a sizeable proportion of immune cells (29.7% of monocytes and 51.5% of lymphocytes) in 6 out of 22 patients to contain viral particles [59], signaling their potential as a reservoir for the virus. If immune cells were to infiltrate the neuronal space by crossing the BBB, this would allow the viral particles in them to cause direct brain damage by binding to ACE2 receptors on neuronal and glial cells [7]. However, whether these findings can be accurately extrapolated to SARS-CoV-2 remains yet to be ascertained. In addition, autopsies and studies conducted on samples obtained from infected individuals have been inconclusive about direct immune cell infiltration during COVID-19 [60].

#### *4.2. Non-Specific Systemic Inflammation, Multisystem Inflammatory Syndrome (MIS), and ARDS*

i. Widespread systemic inflammation: A significant increase in inflammatory cytokines plays a role in SARS symptoms, with inflammation persisting even after the viral clearance, and a similar ramped up an innate immune response in the form of 'cytokine storm' is behind COVID-19 as well [16,34,61]. Highly circulating amounts of Interleukins and other mediators (including IL-6, IL-1β, and TNF, and others) resulting in a pro-inflammatory status are commonly found in COVID-19 patients [62,63]. This amplified immune response may cause increased vascular permeability and vasculopathy arising from disseminated intravascular coagulation (DIC). Subsequently, the BBB is compromised, allowing cytokines to activate a microglial inflammatory response [64]. This mechanism may potentially lead to delirium and seizures due to an immune-mediated encephalopathy [6]. There is a substantial risk of Cerebral Vascular Disease in infected individuals potentially due to this exact pathophysiology, with studies showing increased incidences of hypoxicischemic conditions [8]. A study in April 2020 investigated the histopathological changes during autopsy, and all 18 patients' brain specimens depicted hypoxic changes [60]. It also drew attention to how cerebral white matter is at high risk for damage due to ischemia, manifesting as loss of vital cognitive functions during and after COVID-19 [8]. Several studies investigating Alzheimer's Disease (AD) patients found a notable inflammation in patients showing cognitive deficits compared to the control group, indicating the link between the development of cognitive impairment and increased inflammatory molecules [65]. In addition, previous studies have highlighted the long-term detrimental effects of severe inflammation on the cognitive ability of a person, especially those already with or at high risk of developing a neurodegenerative disease [66–68]. A study investigated links between serum inflammatory markers and C-reactive protein (CRP) in COVID-19 patients with cognitive functions and found loss of some domains, such as sustained attention, to be significantly correlated to CRP levels in the blood [34]. In addition, previous longitudinal studies have confirmed a significant association between CRP levels and cognitive decline [69], affirming how underlying inflammation (using CRP as a marker) likely affects an individual's cognitive functioning in the long run. Some studies have claimed the role of NLRP3 inflammasome activity in exacerbating systemic inflammation and its outcomes [16]. In addition, some proteins of SARS-CoV have shown to induce NLRP4 inflammasome activity, making it likely that SARS-CoV-2 also utilizes similar pathways to cause extensive inflammation [70]. This pathway has further been suggested to explain cognitive deficits due to high IL-1β activity in the setting of hypercapnia caused by mechanical ventilation [71]. ii. Multisystem Inflammatory Syndrome (MIS): Demographically, COVID-19 has been shown to cause more severe disease in adults, but increasing reports of COVIDassociated Multisystem Inflammatory Syndrome (MIS) have surfaced [72–74]. While more prevalent in children, as MIS in children (MIS-C), it can potentially occur in adults as well (MIS-A). A meta-analysis comparing MIS-C's clinical course to COVID-19 revealed how it can potentially lead to multi-organ failure [75]. MIS-C was also shown to have a relatively higher incidence of neurological manifestations compared to acute COVID-19 [75]. As a distinct manifestation of a SARS-CoV-2 infection even in adults [76], with a high risk of neurological symptoms, MIS warrants discussion as a potential causal factor in the development of Cognitive COVID. MIS-C is considered to cause a hyperinflammatory shock and resembles Kawasaki Disease (KD) [77] or Toxic Shock Syndrome (TSS) [78]. Several cases with serologic evidence of a SARS-CoV-2 infection reported symptoms of MIS-C such as shock, cardiac symptoms, gastrointestinal complains, and elevated markers of inflammation, particularly after it was recognized by the Centers for Disease Control and Prevention (CDC) in May 2020 [78]. The pathophysiology of MIS-C

during and after a SARS-CoV-2 infection is largely unknown [78]. Generally, MIS-C is believed to cause a dysregulated immune response possibly by viral mimicry of the host and development of autoantibodies. This leads to widespread systemic inflammation that potentially has a damaging impact on multiple systems, including the neurological system [79–81]. Interestingly, some cases depicted a milder, 'overlapping' syndrome with acute COVID-19, while other cases reported MIS-C symptoms weeks after an acute infection. However, children with an active COVID-19 infection confirmed via a positive RT-PCR test form only one-third of the total MIS-C cases, with a majority showing evidence of a past infection confirmed via serological tests [80]. Jiang et al. uses epidemiological data from different countries to suggest that MIS-C is more likely caused by an acquired, albeit dysfunctional, immune response to SARS-CoV-2 instead of direct viral involvement [80]. The above discussion on widespread systemic inflammation in severe COVID-19 in adults is of value here when discussing MIS-C or MIS-A. The pathophysiology of MIS-C is also believed to involve a cytokine storm with elevated inflammatory mediators [80,82] which may ultimately lead to neurocognitive manifestations, as elucidated previously in this text.

iii. Acute respiratory distress syndrome (ARDS), mechanical ventilation, and associated cognitive decline: Although the exact ratio of COVID-19 patients developing severe disease and requiring hospitalization or intensive care unit (ICU) admission varies extensively, there is undoubtedly a noticeable proportion that progresses to life-threatening conditions [83]. Preliminary studies from China investigating data of more than 70 thousand patients suggested that around 19% of patients with COVID-19 develop severe or critical disease, most likely necessitating hospitalization [84]. A survey of 17 studies examining statistics of hospitalized COVID-19 patients from different regions found that one-third of all hospitalized and threequarters of all ICU-admitted patients develop ARDS [83]. Cognitive impairment following ARDS of variable etiology is widely reported and reviewed [85]. Although severe inflammation, hemodynamic instability, and hypoxia have been indicted, the exact mechanism causing it is unknown. However, a review of studies has shown that cognitive impairment post-ARDS has a high incidence and ranges from 70–100% at hospital discharge, to 46–80% at one year after discharge, to 20% at five years after discharge [85]. In addition, an observational study in France described several ICU-admitted COVID-19 patients with complaints of ARDS developing encephalopathy manifesting as confusion and agitation [31]. According to Tzotzos et al., of the COVID-19 ICU-admitted patients who develop ARDS, more than 80% must receive mechanical ventilation [83]. Mechanical ventilation, regardless of ARDS, is associated with cognitive decline and reduced quality of life in the long run [86]. Since mechanical ventilation inextricably leads to the administration of sedatives, it is essential to note delirium and other cognitive consequences that may accompany, both in the short and long term [87]. The likelihood of a systemic inflammation playing a significant role in the development of cognitive loss compared to direct viral damage is underscored by the sparse evidence of the virus being found in the CSF [16]. Furthermore, instead of being two entirely independent processes, the neurotropism of SARS-CoV-2 and the widespread parallel inflammation may also operate in conjunction [7] and collectively lead to direct and indirect neuronal damage with cognitive deficits. Lastly, it is crucial to not trivialize non-specific but potentially key elements in developing cognitive sequelae, namely COVID-19 complications such as ARDS and subsequent mechanical ventilation [11].

#### *4.3. The Psychosocial Strain of the Pandemic and Associated Lockdowns*

i. Psychological stressors: While countries battle their second or third waves, confinement due to lockdowns and the fear of one or one's loved ones contracting

COVID-19 are just some of the reasons that continue to cause an unprecedented psychological burden on people across the world [12,88]. With psychological conditions such as anxiety and depression now being recorded globally, cognitive consequences can be reasonably expected as a unique symptomatic presentation [89]. A systematic review remarked that some studies had shown the prevalence of post-traumatic stress disorder (PTSD) ranging from 7% to as high as 53.8% during the pandemic [90]. Moreover, this psychological disorder has been correlated with diminishing cognitive function, especially in the elderly [91], showing how 'Cognitive COVID' is possibly related to an individual's psychological state.

ii. Social isolation and government-mandated lockdowns: An article published in late 2020 had reviewed the available evidence and stipulated that social distancing/isolation and lack of human interaction may have a detrimental effect on a person's cognition [92]. Echoing these findings, a study conducted in Italy during May 2020 investigated the effects of psychological stressors as a result of isolation in the form of national lockdown as a mitigation technique on the global cognitive function of the public [93]. Findings suggested cognitive function such as barring memory deteriorated during lockdowns. Furthermore, with a greater prevalence of anxiety, depression, and other mental health changes, a significant deleterious impact on cognitive function(s) was noted in those who had lesser social interactions [93].

Further research needs to correlate lockdowns and various psychosocial factors of the pandemic with cognitive ability to gather experimental evidence. Consequently, the findings may aid in ascertaining if this psychological burden is responsible for the reason why COVID-19 survivors may develop cognitive sequelae following their recovery. Likewise, any significant conclusions may also illustrate if psychosocial distress in the wake of lockdowns increases an individual's risk of being affected and the severity by which they are affected due to other causal factors outlined in this text earlier.

#### **5. COVID-19 Vaccination, Autoimmunity, and Cognitive Impairment**

The vaccine roll-out for COVID-19 began recently, but nearly 3.57 billion doses have been administered worldwide already [1]. Various vaccines were approved by the World Health Organization (WHO) for emergency use but all of them fall under three major subtypes: messenger RNA (mRNA), viral vector, and inactivated whole-virus [94]. As the pace of vaccine administration increases, more data is surfacing regarding post-vaccination adverse events. Neurocognitive symptoms following vaccinations COVID-19 vaccinations are rare but emerging case reports require due attention to accurately evaluate the pathophysiology and risk-factors carefully and accurately.

Two cases of encephalopathy within one week following inoculation via an mRNA vaccine were reported in patients with no prior neuropsychiatric history [95]. Furthermore, an 89-year-old patient developed delirium after a first dose of an mRNA vaccine [96]. The mRNA in the vaccines encodes antigen S-2B, which includes SARS-CoV-2 spike glycoprotein. The spike glycoprotein during a viral infection initiates a cascade of inflammatory reactions after attaching to ACE2 receptor, leading to COVID-19 encephalopathy [95]. Authors hypothesize that cells translating this vaccine mRNA may produce the same glycoprotein and in turn mimic the encephalopathy caused by an active viral infection [95]. Furthermore, acquired immunity via anti-spike antibodies linking to spike protein of SARS-CoV has been known to boost inflammation by activating macrophages [80]. A similar mechanism following development of anti-spike antibodies against SARS-CoV-2 after administration of mRNA vaccines may lead to widespread inflammation, macrophage activation, and development of neurological symptoms [80,95,96].

An interesting case is of an adult, who recovered from COVID-19 6 weeks ago, developing MIS following a second dose of an inactivated virus vaccine [97]. Features of shock and cardiac dysfunction were present in the patient along with elevated inflammatory markers, indicating MIS. The authors postulate that the vaccine may have accentuated their

body's immune response which was 'already primed' following SARS-CoV-2 infection and therefore led to an uninhibited inflammatory condition in the body [97].

Some authors have warned against the use of certain immunogenic proteins of SARS-CoV-2 in vaccines that are homologous to the human immune system [98]. With most of SARS-CoV-2's immunogenic epitopes matching human proteins, there is a reasonable risk that vaccines containing these epitopes will lead to autoimmunity [98]. Excessive inflammation, creation of autoantibodies, and a series of biochemical processes due to autoimmunity may lead to neuroinflammation, damage to neuronal integrity and cognitive impairment [99]. Using Alzheimer's Disease as a parallel in mouse models, a temporal association between increasing autoimmunity and declining cognitive competence was found [100], highlighting the damaging effect accelerated autoimmunity may have on brain function.

#### **6. Discussion**

We have identified three major areas of discussion when debating on causality of cognitive symptoms occurring during and after a SARS-CoV-2 infection. The direct neurotropism of SARS-CoV-2 is largely based on information available for SARS-CoV. Several hypotheses on mechanisms of direct neurotropism have been outlined, such as infiltration of virus-laden immune cells, hematogenous spread through CVOs or breaks in BBB, and retrograde neuronal transmission through invasion of the olfactory system. Furthermore, non-specific systemic inflammation which also manifest as multisystemic inflammatory syndrome (MIS) due to a 'cytokine storm' reported in COVID patients predisposes them to vascular injuries, leading to neuroinflammation. As mentioned earlier, treatment methods such as intubation to treat ARDS can also lead to cognitive symptoms. Lastly, a neuropsychiatric vantage point allows us to underscore the importance of prevailing psychological stressors and their effect on a person's cognitive ability.

For the first time, to our knowledge, in this text we have also reviewed case studies of cognitive impairment and adverse-events following COVID-19 vaccinations, especially the mRNA subtype. We have discussed the occurrence of a heightened immune response to spike glycoprotein encoded by the mRNA in the vaccine which leads to a condition similar to COVID-19 encephalopathy. By discussing the potential occurrence of autoimmunity following inoculation, we have identified the risk of using viral epitopes that are homologous to proteins in the human immune system.

Furthermore, despite the varying prevalence and presentations of cognitive deficits, due to the sheer scale of the pandemic, with global cases crossing 173 million [1], any proportion will result in substantial implications on health systems and a massive influx of patients with cognitive complaints. Therefore, with more significant evidence through research, greater awareness regarding the probable emergence of 'Cognitive COVID' in some patients is required—both for the public, early seeking medical care, and healthcare workers, for readiness and early detection.

As described above, the varying levels of cognitive impairment will require a thorough evaluation, planned follow-ups, and in-patient management if required. Consequently, facilities and institutions should allocate adequate resources and enable their healthcare workers via training to effectively respond to those COVID-19 survivors at high risk for developing cognitive sequelae [6].

Additionally, the therapeutic significance of understanding and ascertaining the etiology of 'Cognitive COVID', particularly on a cellular level, is manifold. There are available interventions that may mitigate the negative impact of high inflammation levels with a potential cytokine storm, including cytokine antagonists and other anti-inflammatory modulators [64]. Importantly, due to the requirement of physical distancing to avoid transmission of SARS-CoV-2, telemedicine for diagnosis and cognitive rehabilitation is an exciting and promising avenue to be utilized [101]. Moreover, as a psychosocial burden and mental health disorders are possible causal elements, telepsychiatry services can play a crucial role in preventing cognitive impairments [102].

#### **7. Conclusions, Limitations and Way Forward**

Cognitive COVID is an oft-ignored aspect of the pandemic, but with greater attention now being paid to the non-respiratory and long-term cognitive consequences of COVID, it is vital to collect further evidence regarding the prevalence, presentation, correlations, and causality of these events. Furthermore, the potentially long-term nature of these deficits and their devastating effect on quality of life, especially the elderly, makes it more pressing to review our current approach in early identification, management, and rehabilitation of patients exhibiting cognitive symptoms [6]. In this article, we have highlighted the probable causality of Cognitive COVID by reviewing the available hypotheses, case reports, and clinical data that has been published after the pandemic started and used previous coronaviruses as a basis to form a parallel to the novel coronavirus 2019.

Due to the relative recency of the pandemic and unavailability of coherent data, it is extremely challenging to reach a plausible conclusion regarding the intricate interplay of causal factors. Criteria to determine causality such as the Bradford-Hill criteria [103] are difficult to apply on the causal factors. The lack of consistent data available from different regions across the world and various methods of measuring inflammation or cognitive deficits disallow direct comparison. The biological plausibility criterion of Bradford-Hill criteria, however, has been discussed at depth in the text to decipher neurological and inflammatory mechanisms that lead to clinical symptoms. Another criterion of analogy can be extrapolated to use of available data from SARS-CoV and MERS-CoV to predict SARS-CoV-2's effect on the brain. We are also limited by a lack of research conducted into different variants of SARS-CoV-2 and their effect on cognition.

The way forward is to develop a standardized protocol for neurocognitive assessment of COVID-19 patients, especially at times of discharge from hospitalization and end of medical interventions such as intubation. These steps will mitigate the threat posed by 'Cognitive COVID' and will undoubtedly decrease the burden on already overwhelmed healthcare systems.

Moving forward, greater attention should be paid to cognitive impairment during and after COVID-19 and vaccination. With the emergence of new strains of COVID-19, such as the Delta and the Lambda variant [104], the variation in prevalence of cognitive manifestations of the viral infection needs to be ascertained. Therefore, it is imperative to collect empirical data from multiple demographics in order to attain uniform clinical and biochemical information regarding causality and risk-factors in developing cognitive impairments.

**Author Contributions:** Conceptualization, I.U.; methodology, H.A.A. and A.A.; formal analysis, H.A.A.; data curation, M.N.D. and M.A.; writing—original draft preparation, H.A.A., M.D.G., A.A. and M.A.; writing—review and editing, S.S. and D.D.B.; visualization, I.U. and D.D.B.; supervision, I.U. and D.D.B.; project administration, I.U. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** No new data were created or analyzed in this study. Data sharing does not apply to this article.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Review* **Impact of COVID-19 on Neuropsychiatric Disorders**

**Niloufar Zia 1, Parsa Ravanfar 2, Sepideh Allahdadian <sup>3</sup> and Mehdi Ghasemi 4,\***


**Abstract:** Since the Coronavirus disease 2019 (COVID-19) pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), many studies have shown that besides common COVID-19 symptoms, patients may develop various neuropsychiatric conditions including anxiety, mood disorders, psychosis, neurodegenerative diseases (e.g., dementia), insomnia, and even substance abuse disorders. COVID-19 can also worsen the patients underlying neuropsychiatric and neurodevelopmental conditions during or after the system phase of disease. In this review, we discuss the impact of SARS-CoV-2 infection on development or status of neuropsychiatric conditions during or following COVID-19.

**Keywords:** coronavirus disease 2019 (COVID-19); severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2); neuropsychiatry; neurodegenerative diseases; neurodevelopmental diseases

#### **1. Introduction**

In December 2019, a novel Coronavirus named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) was identified in Wuhan, China. Soon after it became an epidemic throughout the world. SARS-CoV-2 has some spikes on its surface which are the membrane-anchored tiners consisting of receptor binding s1 and membrane-fusion s2 segment. The Cov name originated from these spikes. S1 segment consists of the receptor binding domain (RBD) that causes pathogenicity and infects the host cell via binding to the angiotensin-converting receptor-2 (ACE-2) on all tissues [1,2]. RBD has greater affinity for ACE-2 on cells of the ileum, kidney, heart, brain, lung, and vasculatures. It is responsible for the different manifestations it has, including respiratory disease and mild pneumonia; its usual symptoms include fever, shortness of breath, cough, and fatigue [3]. Besides systemic manifestation, accumulating reports indicate that patients with COVID-19 may develop a variety of neuropsychiatric conditions during or after COVID-19 (Figure 1) [4–7]. SARS-CoV-2 infection may also affect the patients' current neuropsychiatric conditions in various ways. Given the high burden of neuropsychiatric conditions on society besides COVID-19 itself, in this review, we will discuss the impact of SARS-CoV-2 infection on new-onset and current neuropsychiatric conditions in patients with COVID-19.

**Citation:** Zia, N.; Ravanfar, P.; Allahdadian, S.; Ghasemi, M. Impact of COVID-19 on Neuropsychiatric Disorders. *J. Clin. Med.* **2022**, *11*, 5213. https://doi.org/10.3390/ jcm11175213

Academic Editor: Alfonso Troisi

Received: 6 August 2022 Accepted: 2 September 2022 Published: 3 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Figure 1.** Neuropsychiatric manifestations and possible underlying mechanisms after SARS-CoV-2 infection. ACE2, angiotensin-converting enzyme 2; AD, Alzheimer's disease; BBB, blood-brain barrier; IL-6, interleukin-6; LRRK2, leucine-rich repeat kinase 2; PD, Parkinson's disease; TNF-α, tumor necrosis factor-α.

#### **2. Neuropsychiatric Complications of COVID-19**

Acute neuropsychiatric presentations and potentially long-term complications have been reported in people infected with COVID-19 as well as those who recovered from it [5]. In a large study using the TriNetX Analytics Network (a federated network recording anonymized data from electronic health records in 62 healthcare organizations, primarily in the United States [US]) and including 62,354 patients with confirmed diagnosis of COVID-19 [8], it was shown that COVID-19 was associated with increased incidence of a first psychiatric diagnosis in the following 14 to 90 days compared with six other health events (i.e., influenza, other respiratory tract infections, skin infection, cholelithiasis, urolithiasis, and a large bone fracture). The hazard ratios were greatest for anxiety disorders, insomnia, and dementia. The incidence of any psychiatric diagnosis in the 14 to 90 days after COVID-19 diagnosis was 18.1%, with 5.8% having a first diagnosis (e.g., first diagnosis of dementia 1.6% in patients older than 65 years) [8,9]. There is a wide range of underlying etiologic factors both within and beyond the CNS that lead to neuropsychiatric sequelae (Figure 1) [7].

In two large sample retrospective studies of 40,469 patients who recovered from COVID-19 in the TriNetX database, diagnosis of anxiety (and related disorders) and mood disorders was established in 4.6% and 3.8%, respectively, on or within one month after diagnosis of COVID-19 [6] (Table 1). Another retrospective study on 44,779 COVID-19 patients without previous psychiatric illness in the TriNetX Analytics Network revealed that the rate of all diagnoses of psychiatric disorders (i.e., including relapses) was higher within 14 to 90 days after COVID-19 diagnosis than after control health events (i.e., influenza, other respiratory tract infections, skin infection, cholelithiasis, urolithiasis, and a large

bone fracture) [9]. The most common psychiatric diagnosis after COVID-19 diagnosis was anxiety disorder (12.8%, 95% confidence interval [CI] 12.4–13.3), followed by mood disorders (9.9%, 9.5–10.3). The probability of a first diagnosis of mood disorder within 14 to 90 days after COVID-19 diagnosis was 2% (95% CI 1.7–2.4), with depressive episode as the most common first diagnosis of mood disorder (1.7%, 95% CI 1.4–2.1) [9]. In another retrospective cohort study on 236,379 survivors of COVID-19 in the TriNetX Analytics Network, estimated incidences for first-time anxiety disorders in the following 6 months in the whole cohort and those admitted to intensive care unit (ICU) were 17.39% (95% CI 17.04–17.74) and 19.15% (95% CI 17.90–20.48), respectively [8]. Moreover, the estimated incidences for first-time mood disorders in the following 6 months in this study were 4.22% (95% CI 3.99–4.47) and 5.82% (95% CI 4.86–6.97) [8]. The prevalence of self-reported symptoms of depression and anxiety was much higher. Studies using self-report tools have suggested a markedly greater frequency of depressed mood (29.2%), anxiety, and post-traumatic anxiety symptoms (20.8–96.2%) in different countries including China and Italy [10–13] (Table 1).



There are multiple factors that contribute to the development of mood and anxiety disorders associated with COVID-19. These factors can be categorized into three domains: contextual factors and life stressors, disease-related reduction in quality of life such as fatigue and breathlessness, and biological factors impacting the brain such as neuro-immunological phenomena. Based on the evidence provided by Taquet et al., this higher occurrence of first-time mood and anxiety disorders cannot be solely attributed to the contextual factors such as economic and social adversities/challenges/stressors associated with COVID-19. Here, we focus on the disease-related pathophysiological mechanisms that may account for the occurrence of these psychiatric manifestations. The association between COVID-19 and depression and anxiety can be due to neurotropism of the virus, or the immunological reactions of the body such as cytokine storm [18]. In a study by Mazza et al. in Italy, there was a significant association between systemic immune-inflammation index (SII) with measures of anxiety and depression [12]. COVID-19 has been characterized by an exaggerated inflammatory response known as a "cytokine storm", and inflammatory cytokines have been associated with depression [19]. Conversely, the presence of a previous diagnosis of mood disorder is associated with higher mortality rates after prolonged hospitalization [20].

First-time psychosis may also occur among patients with COVID-19 as assessed in the TriNetX Analytics Network as well as systematic reviews of case reports and case series from different regions worldwide [8,9,21]. Although there has been shown a low probability of being newly diagnosed with a psychotic disorder in the 14 to 90 days after COVID-19 diagnosis (0.1%, 95% CI 0.08–0.2), broadly similar to the probability after control health events [9], the estimated incidences of psychotic disorders within 6 months after COVID-19 was higher by 1.40% (95% CI 1.30–1.51). It was even more increased in those patients initially admitted to the ICU due to COVID-19 (2.77%, 95% CI 2.31–3.33) [8]. A more recent systematic review of case reports and case series of 57 patients with COVID-19 (collected from six electronic databases including PubMed, Scopus, Web of Science, PsycInfo, PsycArticles, and CINAHL) also reported that the mean age for onset of psychotic symptoms (predominantly delusions and hallucinations) was the early 40s (men: 43.4 and women: 40.3), more than two thirds (~69%) of patients did not have any prior psychiatric disorders [21]. Only 26.3% of patients presented with moderate-severe COVID-19-related disease and complications. Overall, psychotic symptoms resolved markedly in 63.2% of cases after treatment with antipsychotics, benzodiazepines, valproic acid, and electroconvulsive treatment [21]. COVID-19-related new-onset psychosis and mania have been also reported in children and adolescents, even in those with asymptomatic COVID-19 infections as described in some case reports in the U.S. [22].

New onset dementia and cognitive dysfunction can occur in relation with COVID-19 [8,9,16,23]. A retrospective analysis of 50 patients with COVID-19 in the U.S. found cognitive impairment in 26% of cases [23]. Another study on 153 COVID-19 cases in the United Kingdom (UK) also showed that overall, 23 (15.3%) patients developed neuropsychiatric disorders related to COVID-19, among which 10 (43%) patients had newonset psychosis, 6 (26%) had a neurocognitive (dementia-like) syndrome, and 4 (17%) had an affective disorder [16]. Further larger cohort studies showed that the estimated incidence of dementia during the first 14 to 90 days after a diagnosis of COVID-19 is 0.44% (95% CI 0.33–0.60); which was higher in those patients older than 65 years (1.6%,

95% CI 1.2–2.1), with a hazard ratio (HR) between 1.89 and 3.18 [9]. The estimated incidence of new dementia in the following 6 months was even higher by 0.67% (95% CI 0.59–0.75) and again with an increased rate in those patients initially admitted to ICU due to COVID-19 (1.74%, 95% CI 1.31–2.30) or in those who had encephalopathy by 4.72% (3.80–5.85) [8].

Both short-term (i.e., within 14 to 90 days) and 6-month onset of other neuropsychiatric conditions, such as insomnia and substance use disorder, may rise in the COVID-19 population as assessed in the TriNetX Analytics Network [8,9]. A post-discharge evaluation of 120 COVID-19 patients in France after an average of 110.9 days following admission showed that the most frequently reported persistent symptoms were fatigue (55%), dyspnea (42%), loss of memory (34%), difficulty in concentration (28%), and sleep disorders (30.8%) [24]. The probability of a first diagnosis of insomnia in the 14 to 90 days after COVID-19 diagnosis was further shown to be 1.9% (95% CI 1.6–2.2), more common than after controlled health events (HRs 1.85–3.29), in agreement with predictions that circadian disturbances will follow COVID-19 infection [9,25]. Close to 60% of the insomnia diagnoses were not accompanied by a concurrent diagnosis of an anxiety disorder [9]. The estimated incidence of first insomnia within 6 months post-COVID-19 was 2.53% (95% CI 2.37–2.71) [8]. The estimated incidence of new substance use disorder within 6 months following COVID-19 was 1.92% (95% CI 1.77–2.07) [8]. Overall, the above data indicate that COVID-19 is followed by remarkable rates of long-term neuropsychiatric diagnoses.

#### **3. Impact of COVID-19 on Neuropsychiatric Disorders**

The COVID-19 virus infection and the impact of the virus on society causes several different sequels and comorbidities in physical and mental health [26]. Infection with COVID-19 may cause the worsening of neuropsychiatric disorders and/or the development of neuropsychiatric disorders [26,27]. In addition, the societal consequences of the pandemic (i.e., isolation and quarantine) have caused the worsening and development of neuropsychiatric disorders [26]. Examples of relevant neuropsychiatric disorders in the context of COVID-19 include depression, anxiety, delirium, mood compulsivity, cognitive impairment, and obsessive-compulsive symptoms [26,28]. There are a variety of mechanisms that likely cause the worsening of the development of neuropsychiatric disorders in the context of COVID-19 infection and the ongoing pandemic [26].

Recent research examining the impact of COVID-19 infection on mental health functioning suggests that the virus causes an inflammation of the CNS which, in turn, may stress mental health processes and functioning through cytokine secretion [26]. Other research suggests that COVID-19 increases the risk of or exacerbates neuropsychiatric disorders via hypoxemia, which is common among individuals infected with COVID-19 [29,30]. Moreover, other research identifies additional ways by which COVID-19 infection and the pandemic negatively impact mental health (e.g., isolation, changes in social support, economic stressors) [26].

Individuals with pre-existing neuropsychiatric disorders and symptoms are likely at greater risk of poor mental health outcomes in the context of COVID-19 [26,31]. Indeed, the worldwide virus appears to affect people differently based on their baseline psychiatric functioning. As such, individuals with mood disorders, anxiety disorders, and psychotic disorders, may have a different baseline of psychiatric functioning and could be at higher risk of developing neuropsychiatric disorders or experiencing a worsening of neuropsychiatric symptoms following COVID-19 infection, as assessed in different studies and regions including the U.S. and Brazil [8,9,26,31,32].

Individuals with neuropsychiatric disorders demonstrate different and worse symptoms and functioning as compared to their counterparts in the context of the ongoing COVID-19 pandemic [26,33]. Specifically, individuals with pre-existing mood and anxiety disorders report increased stress and fear of pollution in the current pandemic [31]. In addition, these symptoms reached clinically significant or concerning levels. Among individuals with a pre-existing neuropsychiatric disorder who contracted COVID-19, many experienced an exacerbation of mental health symptoms [26,31,34,35]. Specifically, individuals may have experienced serious neuropsychiatric complications, including delirium, cognitive impairment, significant mood alterations, or even psychosis [26,31,32]. Indeed, even among people without pre-existing neuropsychiatric disorders, delirium occurs in most individuals who contract COVID-19 [26,36].

#### *3.1. Anxiety Disorders*

Individuals with anxiety disorders are at risk of poorer outcomes in the context of the COVID-19 pandemic [34]. Using an online subject pool from the U.S. and Canada, researchers found that individuals with anxiety-related disorders experienced higher stress as well as greater fear, socioeconomic consequences, xenophobia, and traumatic stress as compared to individuals with mood disorders and individuals without clinically significant mental health concerns in the context of the COVID-19 pandemic [34]. As such, individuals with anxiety disorders are at greater risk of certain poor outcomes even as compared to individuals with mood disorders in the context of the ongoing COVID-19 pandemic [34]. Regarding mechanisms behind the relationship between anxiety disorders and poor outcomes, individuals with anxiety disorders were more likely to engage in effortful isolation and ineffective coping strategies [34]. Perhaps these behaviors, in combination with other associated behaviors, increased the risk of certain poor outcomes for individuals with anxiety disorders as opposed to their counterparts [34]. The increased risk of poor outcomes across several domains (i.e., psychological or economic) among individuals with anxiety disorders in the context of the COVID-19 pandemic cannot be ignored. There is a need for research, intervention, and policy to address the increased risk and poor outcomes among this population.

#### *3.2. Mood Disorders*

Based on a recent meta-analysis, individuals with a pre-existing mood disorder have a significantly higher chance of hospitalization and death as compared to those without a pre-existing mood disorder following COVID-19 infection [37]. These meta-analytic results highlight the vulnerability of individuals with pre-existing mood disorders for poor health outcomes in the context of contracting COVID-19. As such, researchers and policymakers should consider those with pre-existing mood disorders when creating policy regarding vaccinations and other relevant public health decisions [37]. Mood disorders may confer a greater risk of susceptibility to COVID-19 for a variety of reasons. Individuals with pre-existing mood disorders experience increased psychological stress during the COVID-19 pandemic, which was further associated with maladaptive life and behavioral changes [34,38]. Certain mood disorders are associated with a greater risk of poor outcomes. For instance, a study in Australia demonstrated that specifically men with bipolar disorder are the most at-risk group among individuals with mood disorders regarding risk of depression and financial concerns [38]. As such, it is important to consider how individuals with pre-existing mood disorders may be at risk of certain poor health and psychological outcomes in the context of the ongoing COVID-19 pandemic.

#### *3.3. Neurodevelopmental Disorders*

Neurodevelopmental disorders typically emerge during early-to-middle childhood and may result in functional impairment and/or limitations regarding neuropsychological, cognitive, and adaptive development. Examples of neurodevelopmental disorders include autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), intellectual disabilities, and specific learning disabilities [39].

Unfortunately, children with neurodevelopmental disorders (e.g., ASD and ADHD) are at higher risk of poor mental health and difficulties with functioning than their counterparts in the context of the COVID-19 pandemic, as evidenced by a cross-sectional parent-reported study in the UK [40]. Indeed, children with neurodevelopmental disorders experienced an increase in emotional difficulties and conduct problems as well as a decrease in prosocial behaviors during the pandemic [40]. Depression and anxiety among children and their parents with ASD have increased as reported by studies in different countries such as the US and Switzerland [41,42]. Among children with neurodevelopmental disorders studied in the UK, female children with ASD experienced the highest emotional symptoms [40]. The severity of neurodevelopmental symptoms increased during the course of the pandemic among children and adolescents in the U.S. [43]. In addition, a recent study on 238 adolescents (ages 15–17 years) from two sites in the Southeastern and Midwestern U.S. showed that associated behavioral considerations, including opposition/defiance and impulsivity, also increased during the pandemic [44]. The stressors and increased symptomatology of children with neurodevelopmental disorders also impacted parents; indeed, parents of children with neurodevelopmental disorders reported worse mental health than their counterparts during the pandemic in the UK [40].

Research has provided potential mechanisms or avenues by which individuals with neurodevelopmental disorders experienced worse outcomes in the context of the pandemic. Specifically, individuals with neurodevelopmental disorders experienced closures in or abrupt disruption of their health services [45], which likely caused decreases in physical and mental health [46,47]. The lack of access to physical exercise during lockdown and quarantine(s) may have been particularly detrimental to individuals with neurodevelopmental disorders as exercise may be used to regulate several symptoms of neurodevelopmental disorders [46]. Researchers have also postulated that boredom and associated decreases in motivation have engendered rises in depression and poor outcomes among children and young people with neurodevelopmental disorders [48]. In addition, changes in remote and hybrid schools were associated with increased dropout rates from school among children and young adults with ADHD in the U.S. [43].

Certain aspects of the ongoing pandemic may have increased risk of poor outcomes among children with ASD specifically. Indeed, individuals with ASD may have certain vulnerabilities that were exacerbated by the pandemic [48]. Quarantine and isolation policies in response to the pandemic caused severe disruption to the routines and services typically accessed by people with ASD [49]. In some cases, individuals with ASD lost complete access to services partially because individuals with ASD were not considered a marginalized population [20]. In addition, the evaluation of individuals with suspected ASD was also interrupted and services and diagnoses were delayed [20]. As such, individuals with ASD and individuals yet to be diagnosed with ASD experienced specific stressors that likely impeded effective intervention and support services. As a result, the research described above regarding poor outcomes among individuals with ASD and their families is not surprising.

Although initial research indicates that the negative impact of quarantine and lockdown on mental and physical health does not appear to be long-lasting among children and young people with neurodevelopmental disorders, longitudinal research has not yet been conducted to examine the long-lasting impact of COVID-19 on individuals with neurodevelopmental disorders [44].

#### *3.4. Psychotic Disorders*

Psychotic disorders are characterized by cognitive and perceptual dysfunction, usually hallucinations or delusions [50]. Psychotic disorders may be accompanied by mood disturbances and can be caused by [50]. In the U.S., between 0.25% and 0.64% of the population is diagnosed with a psychotic disorder [51]. However, over the course of the pandemic, the incidence of psychotic disorders and symptoms has risen [51]. Moreover, the risk of psychosis among individuals with COVID-19 is higher than the average population, with between 0.9% and 4% of individuals with COVID-19 experiencing psychosis [51].

As among individuals with neurodevelopmental disorders, services and supports for individuals with psychotic disorders were severely disrupted over the course of the pandemic. In addition, because of the symptoms and vulnerabilities associated with psychotic disorders, this population is highly vulnerable to the changes in routine and access to care caused by the COVID-19 pandemic [51]. Individuals with psychotic disorders may

have experienced disruptions in their access to medication and face-to-face services [51–54]. These changes and barriers to services caused significant decompensation among many individuals with psychotic disorders and were also associated with increased paranoia and anxiety [51,52]. Decompensation refers to a significant deterioration in psychological functioning and an increase in severity of symptoms that is not typical [51]. Because of the transition to telehealth and phone services, providers may not have been able to accurately track and identify patients' decompensation over the course of the pandemic [51]. As such, individuals with psychotic disorders may have been uniquely at risk for experiencing worsening symptoms without proper support or intervention [51].

A single-center retrospective and observational study in Spain found that SARS-CoV-2 infection has also increased risk of and severity of psychosis symptoms [55]. Indeed, psychosis (e.g., delusions) has emerged among individuals with no history of these symptoms following infection with the virus [55]. However, it is unclear to what extent psychosis following infection with COVID-19 is caused by the virus itself versus the medications used by health providers to treat the virus [55]. Certain researchers posit that COVID-19 causes inflammation in the central nervous system, which then causes individuals to experience psychosis [55]. Regardless of the exact cause, individuals with COVID-19 are at increased risk of experiencing psychosis, even those without a prior history [55]. More research studies are needed to understand the cause of psychosis among individuals with COVID-19 as well as assessment and intervention strategies.

#### *3.5. Cognitive Disorders*

Cognitive disorders are overall characterized by executive function impairment and are associated with difficulties with organization, regulation, and perception [56]. Individuals with cognitive disorders may experience difficulties with processing speed, reasoning, decision-making, awareness, attention, learning, impulsivity, memory, or language [56]. More than 16 million individuals in the United States are currently diagnosed with a cognitive disorder [57].

Individuals with cognitive disorders are at significantly higher risk of contracting the virus given baseline difficulties with executive functioning [54]. Indeed, because individuals with cognitive disorders may struggle to care for themselves, it may be harder for them to adhere to safety standards regarding isolation, social distancing, and quarantine [58]. As part of this concern, individuals with cognitive disorders may live in collective housing or treatment units and the functioning of these structures has been severely impacted by the pandemic. As such, it is highly likely the functioning of individuals within these communities has also been severely affected. In addition, such facilities for individuals with cognitive disorders may allow the virus to spread easily across patients and staff putting individuals with cognitive disorders at even higher risk of contracting the virus [59]. If infected with COVID-19, individuals with cognitive disorders may also have worse outcomes, especially if their access to caregivers or supervision is limited in the context of ongoing social distancing and quarantine policy [58,60]. Research is needed to identify and develop standards for communal living facilities for individuals with cognitive disorders that prioritizes the delivery of services and the safety of patients.

Intellectual disabilities are a subset of cognitive disorders. Unfortunately, the health and well-being of individuals with cognitive disorders has been negatively impacted by the ongoing pandemic. Specifically, research indicates that mental health and physical activity have significantly decreased among children and young adults with both physical and intellectual disabilities [47]. Decreases in physical activity were due to quarantine and isolation policies as well as the closing of community centers and exercise facilities [47]. Among parents of children with intellectual disabilities, 90% reported worsening mood and increased behavioral problems among their children [47]. Although this study identified the potential for the development of solutions to the problems and barriers experienced by individuals with intellectual disabilities, no such solutions have been systematically implemented [47]. Thus, both research and advocacy are needed to address the problems and outcomes among individuals with intellectual disabilities over the course of the ongoing pandemic.

#### *3.6. Neurodegenerative Disorders*

Studies have revealed that ACE-2 is also expressed on neurons, glial cells, epithelial cells of blood-brain barrier (BBB), as well as oligodendrocytes. Interestingly a high concentration of substantia nigra may help the virus to enter, which itself may be the cause of reported neuropsychiatric sequelae of the infection [61]. Varieties of neurological symptoms including headache, ageusia, anosmia, and different forms of neurodegenerative disorders have been reported as consequences of SARS-CoV-2 infection [62]. Inflammation as a consequence of viral entry to the brain can cause oxidative damage and apoptosis of the cells in the brain, which has been reported as a cause of neurodegeneration and neurodegenerative disease [63]. Alzheimer's disease (AD) and Parkinson's disease (PD) are two neurodegenerative disorders reported to be caused by prolonged inflammation in patients infected by SARS-CoV-2 as a possible postinfectious manifestation. Inflammatory processes are also involved in other neurodegenerative disorders such as progressive supranuclear palsy syndrome (PSPS) [64], corticobasal syndrome (CBS) [64], and multiple system atrophy-parkinsonian type (MSA-P) [65]. COVID-19 has been reported to affect the international classification of functioning, disability, and health functioning in patients with MSA [66]. Accumulating evidence also suggests that SARS-CoV-2 infection can induce neurodegenerative disorders [67].

In view of the above, SARS-CoV-2-induced reactive oxygen species (ROS) caused by oxidative damage has been reported to cause accumulation of the amyloid beta (Aβ) proteins which itself is involved in the pathogenesis of AD. In addition, increased tau level as a result of neuroinflammation followed by viral entry to the brain has been reported as another possible cause of cognitive impairment within patients of SARS-CoV-2 [67,68]. COVID-19 incidence and complications are increased in patients with AD and related dementia. Patients with AD or related dementias have a cognitive impairment which causes difficulty understanding and remembering the recommendations. This disease is also more associated with other comorbidities such as cardiovascular disease, which may be another reason for high mortality of COVID-19 in these patients (90%) [69,70]. It has been also reported that APOE4 isoform of AD which causes a decreased amount of APO has increased the risk for COVID-19 infection and progression [71].

Several possible pathogeneses have been reported as a cause of PD in COVID-19 patients. Vulnerability of the basal ganglia and dopamine-rich region as well as the neuroinflammation caused by SARS-CoV-2 has been reported as a possible cause of failure of dopamine synthesis in COVID-19 patients [67]. Moreover, DJ1 and Leucine-rich repeat kinase 2 (LRRK2), key proteins in dopamine regulation and oxidative reaction, can be affected by SARS-CoV-2, resulting in dopamine dysregulation and inflammation in substantia nigra and α-synuclein aggregation [67,72].

Apart from the effect of COVID-19 on patients' access to different medical and psychological care, decreased physical activity and family support [73] and different neuropsychiatric outcomes have been reported that may be due to dopamine depletion in PD patients as a neurotransmitter to help with adoption to different changes in healthy individuals, which is lacking in PD patients. Anxiety and stress may remain and become chronic due to lack of adaptation as a result of dopamine depletion [74]. There are three different hypotheses about the effect of COVID-19-induced stress in PD patients. Stress can interfere with the levodopa effect and reduce it [75]. In addition, chronic stress has been revealed to decrease the dopamine activity within the rodent's ventral tegmentum compared to healthy controls and has been reported as a cause of neurodegeneration in PD and AD [76]. Emotional stress as a trigger for freezing gait has been reported as well [77]. One important aspect of COVID-19 infection is dehydration, diarrhea, fever and decrease in water intake which may be another important aspect to consider when adjusting the medication dosage. However, COVID-19 infection has been reported as a trigger for worsening of PD motor

and nonmotor symptoms. Although, the patients with higher age and duration of illness had been reported to have more mortality risk compared to the participants with younger age and shorter disease duration [73,78], worsening of motor and non-motor symptoms, such as rigidity, tremor, fatigue, pain, and concentration, has not been related to disease duration or severity [79]. In view of the above, it can be concluded that COVID-19 can cause different effects on PD via decreasing dopamine secretion, neurodegeneration, and can be a trigger for the developing or worsening of symptoms in PD and AD.

#### **4. Conclusions**

Since the COVID-19 pandemic, accumulating lines of evidence have revealed that besides common clinical manifestations, SARS-CoV-2 infection is associated with development or worsening of a variety of neuropsychiatric conditions. These can occur during or shortly after the onset of COVID-19. However, more recent longitudinal studies have revealed that COVID-19 increases the estimated risk of developing or worsening of neuropsychiatric conditions, such as mood, anxiety, psychotic disorders, dementia, insomnia, or even substance abuse disorders, after 6 months following COVID-19. These data indicate that close and long-term neuropsychiatric and cognitive monitoring of patients with COVID-19 are critical in order to diagnose these sequels and take appropriate therapeutic approaches as early as possible, and eventually improve patients' and their families quality of life.

Although SARS-CoV-2 infection itself can be associated with the above-mentioned neuropsychiatric sequela, one should not overlook the multiple factors that contribute to the development or worsening of neuropsychiatric conditions during the COVID-19 pandemic. Overall, we can categorize these into three major domains: (i) the pandemic's burden on the society acting as psycho-socio-economic stressors (e.g., deterioration of economic state, lockdown or quarantine, disruption of health care provision, and loss of job or family members due to COVID-19), (ii) disease-related reduction in quality of life (e.g., fatigue and breathlessness), and (iii) the impact of infection itself on brain function. These factors separately or in combination with each other may play critical roles in development or exacerbation of neuropsychiatric conditions during the pandemic. For instance, a population-based longitudinal study on 16,910 participants in the UK revealed an independent association between COVID-19 and increased risk of economic vulnerability among participants, measured by both household income sufficiency and sickness absence from work [80]. The economic recession resulting from the COVID-19 pandemic has adversely affected many people's mental health and created new barriers for people already suffering from mental illness [81]. This is not only limited to COVID-19 pandemic, as such negative impacts on mental health and neuropsychiatric conditions have been seen in other global or regional shocks including wars [82–84], natural disasters [85,86], or other disease outbreaks, e.g., with Ebola [87,88], Zika virus [89], Middle East respiratory syndrome coronavirus (MERS) [90,91], and severe acute respiratory syndrome coronavirus (SARS-CoV) [90]. Social isolation due to lockdown or quarantine may have been particularly detrimental to individuals with neuropsychiatric and neurodevelopmental disorders, as social interaction and exercise may be used to regulate several symptoms of these disorders [46,92]. Data from an online questionnaire from 55,589 participants from 40 countries during the COVID-19 pandemic have revealed that lockdown significantly increases anxiety and depression at every degree of lockdown intensity, especially in combination with the presence of prior mental health issue [92]. This emphasizes the need for a proactive intervention to protect mental health of the general population but more specifically of vulnerable groups [92].

Almost over 2.5 years have passed since the COVID-19 pandemic; and as our knowledge about different neuropsychiatric manifestations directly or indirectly related to COVID-19 is improving, it overall highlights the critical need for global preparedness in the mental health sector during outbreaks of such infectious diseases. Important steps and innovations have been taken over the last 2 years to enable better service delivery to the affected populations. For instance, use of telemedicine and electronic prescriptions

have become pivotal tools to be implemented globally [93]. This could be an essential element of continuity of care, especially during the lockdown or quarantine period. The promotion and empowerment of community-based mental health services especially in low-and-middle-income countries can also decrease the present treatment gap even during infectious disease outbreaks [94]. The development of global or local guidelines or consensus recommendations for such neuropsychiatric issues during or after the outbreak seems to be another effective way in reducing such psychiatric and mental health complications [94].

In the end, with an improvement of our knowledge about the pathophysiology of SARS-CoV-2 in brain dysfunction and neuropsychiatric conditions as well as the development of mediations targeting the infection itself or its related pathologic molecular signaling pathways, we may improve or even prevent the development of such neuropsychiatric manifestations. For instance, due to the presence of immune dysfunction and cytokine storm in COVID-19 patients, anti-inflammatory, immunomodulatory, or immunosuppressive medications have been tested or even in animal models of SARS-CoV-2 infections with variable benefits [95,96]. Vaccination has also created a new epoch in improving survival rate and acute critical illness related to COVID-19 [97]. Notably, a large retrospective cohort of 9479 individuals who developed COVID-19 despite SARS-CoV-2 vaccination, well-matched to controls, found that vaccination protected against severe acute illness, stroke, seizures, and psychotic disorders after breakthrough COVID-19 as assessed within 6 months post-vaccination. However, it may not protect from fatigue and other post-COVID-19 behavioral and cognitive symptoms [98]. Overall, research in this area is advancing. For instance, some studies have found an association between low neurotrophic factors and COVID-19-related neuropsychiatric complications [99]; thus, neurotrophic drugs such as cerebrolysin may serve as a potential therapeutic approach to improve neuropsychiatric manifestations of COVID-19 [100]. Clearly, more research studies and clinical trials are needed in this regard.

**Author Contributions:** N.Z., P.R., S.A. and M.G. conceived and designed the review, outlined the performed rigorous literature search, designed the table and figure, and wrote the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** M.G. is supported by a clinical research training scholarship in ALS funded by The ALS Association and The American Brain Foundation, in collaboration with the American Academy of Neurology as well as NIH-funded Wellstone fellowship training grant (NIH 5P50HD060848-15) for research on FSHD.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Journal of Clinical Medicine* Editorial Office E-mail: jcm@mdpi.com www.mdpi.com/journal/jcm

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34

www.mdpi.com ISBN 978-3-0365-7194-2