Next Article in Journal
Supporting the Development of Grassroots Maternal and Childhood Health Leaders through a Public-Health-Informed Training Program
Previous Article in Journal
Assessing the Impact of Quarrying as an Environmental Ethic Crisis: A Case Study of Limestone Mining in a Rural Community
Previous Article in Special Issue
The Relationship between Patients’ Demands and Workplace Violence among Healthcare Workers: A Multilevel Look Focusing on the Moderating Role of Psychosocial Working Conditions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Bullying against Healthcare Professionals and Coping Strategies: A Scoping Review

by
Ana Rita Valente Ribeiro
1 and
Ana Isabel Sani
1,2,3,*
1
Faculty of Human and Social Sciences, University Fernando Pessoa (UFP), 4249-004 Porto, Portugal
2
Fernando Pessoa Research, Innovation and Development Institute (FP-I3ID), Observatory Permanent Violence and Crime (OPVC), University Fernando Pessoa (UFP), 4249-004 Porto, Portugal
3
Research Centre on Child Studies (CIEC), University of Minho, 4710-057 Braga, Portugal
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2024, 21(4), 459; https://doi.org/10.3390/ijerph21040459
Submission received: 10 February 2024 / Revised: 1 April 2024 / Accepted: 3 April 2024 / Published: 9 April 2024
(This article belongs to the Special Issue Violence Intervention in Workplace and Family)

Abstract

:
Violence against healthcare professionals is an event that further burdens the daily lives of those who try every day to care for and assist those who need it most. In an attempt to overcome these events, there are coping strategies that can be used to reduce the stress caused. Therefore, this study aims to analyse the phenomenon of violence against healthcare professionals and the relationship between the bullying suffered by these professionals and the coping strategies they developed to overcome these moments. To this end, a scoping review was conducted in which eight articles were selected for final analysis from a total of 276 articles found in three electronic databases (EBSCO, PubMed, and Web of Science). This review concludes that the most common workplace bullying behaviours include excessive workloads, humiliation and ridicule, impossible deadlines, and verbal attacks. Professionals reported negative impacts, such as helplessness, depression, stress, insomnia, and the desire to change jobs. Victims of workplace bullying often expressed their intention to leave their current job or even abandon the profession. Problem-focused coping strategies are the most used. The studies indicated that workplace bullying negatively affects professionals in physical and mental terms, as well as in terms of quality of life at work, requiring more research and adoption of preventive measures to identify and combat the problem.

1. Introduction

Bullying understood as aggressive behaviour, intentional and repeated over time, by one or more people towards someone with the purpose of causing harm [1] is not a behaviour exclusive to the school environment and carried out only at children and young people. Bullying is a conduct that a person, of any age group, gender, or profession, may adopt in various other contexts, such as the workplace [2].
The definition of bullying in the workplace is multifaceted and complex, with several definitions and terminologies that different authors have adopted to describe this phenomenon. In the United States of America, Brodsky [3], an American psychiatrist, began to use the expression harassment, which he defined as repeated and persistent behaviour with the intention of frustrating or receiving some type of reaction from the victim. On the other hand, Leymann [4,5], a Swedish researcher, began to use the expression mobbing, as he stated that bullying was more physical and threatening violence and then defined mobbing as a more sophisticated behaviour, giving as an example isolation of the victim. Hirigoyen [6], a French psychiatrist, uses the expression “moral harassment” where she considers that behaviours can affect a person’s dignity and integrity and even put their job in danger. The term incivility, although less frequent in articles that address the topic, is sometimes used to refer to acts of rudeness and discourtesy, rumours and gossip, and disrespect for the dignity of the professional, including by colleagues [7]. The literature states that these and other actions that compromise a safety culture are extremely dangerous for both patient outcomes and the general well-being of healthcare personnel [8].
Although there are several definitions of bullying in the workplace, there are some similarities between all the definitions. In general, ‘workplace bullying’ can be defined as a type of violence involving recurrent practices through aggressive behaviours that occur over a period of time, which are undesirable by the victim [9,10]. Bullying in the workplace is a reality in many countries [11,12,13,14], particularly affecting professionals in the health sector [15,16,17,18].
Bullying in the workplace can lead to the emergence of physical and psychological problems, such as anxiety, depression, stress, and risk of death [19]. For some healthcare professionals, such changes may interfere with their ability to perform their work, such as providing care and referrals to health care. These experiences can be a factor of great stress in the lives of healthcare professionals, leading to serious consequences not only for professionals, but also for users and organizations [20]. To this end, there are some strategies that are studied as mediating variables (e.g., intention to quit, job engagement, job satisfaction) or moderators (e.g., social support, constructive leadership, attribution style) to face situations of harm, threat, and challenge to which are confronted [21].
The coping process is perceived as a set of efforts made by individuals when faced with some situations that may cause threat or discomfort to the person [22]. Coping strategies can be divided into two types, emotion-focused and problem-focused [23]. In emotion-focused coping strategies, energy is directed at somatic and/or feeling levels and aims to reduce the uncomfortable physical sensation caused by the state of stress. On the other hand, in problem-focused coping strategies, the victim tries to understand and know which problem is causing stress and acts directly to resolve the problem to try to change or avoid it in the future [23]. Authors like Rodriguez et al. [24] studied the differences in coping strategies between genders, concluding that men tend to use problem-focused strategies more, while women prefer to use emotion-focused strategies more.
Coping strategies are related to contextual factors. The individual may choose to adjust their strategy based on the specific situation that is causing the stress. It is important to note that there are no adaptive or maladaptive coping strategies [25]. The decision on which strategy to adopt is a personal choice, the individual chooses the strategy that they consider positive and that helps to alleviate discomfort and reduce negative feelings associated with stress. In contrast, coping strategies are considered negative and ineffective if they are applied, but the stressful situation continues to persist and maintains an imbalance in several aspects [25,26].
Folkman and Lazarus [27] created a transactional model of coping where they argue that stress is contextual, representing a process of interaction in constant evolution between the individual and the environment. In this context, stress is defined as a situation that the individual evaluates as significant and that imposes needs that exceed the resources available to deal with the moment. This model comprises four essential elements: (1) the conception of coping as a process that occurs between the individual and the environment; (2) its main purpose being to deal with stressful situations; (3) the involvement of assessment, that is, how the person perceives stress; (4) the implication of the mobilization of resources, in which individuals use cognitive and behavioural resources to deal with claims that arise both from within and outside the interaction with the environment [22,27].
According to Angst [28], the term resilience is directly related to coping strategies, as it can be considered as a procedural form between the individual and the environment that surrounds them, and the latter must evaluate and interpret the phenomena they perceive. Thus, people who use coping strategies can be considered resilient people.
Therefore, this scoping review aimed to explore the phenomenon of bullying suffered by healthcare professionals and the coping strategies used to combat it. A review would be important to explore the phenomenon of bullying suffered by healthcare professionals and the coping strategies used, as this is a significant and harmful problem that affects the mental health and well-being of healthcare professionals. Bullying can have profound impacts on the quality of work, team morale, and patient safety. This scoping review can contribute to raising awareness about this issue, promoting changes in organizational practices, and providing support to healthcare professionals facing this form of violence in the workplace. Understanding the coping strategies used by healthcare professionals can help identify which ones are most effective and develop interventions to support those experiencing bullying. This scoping review provides a comprehensive overview of existing research on this topic, allowing for the delineation of studies in this area.
Thus, this scoping review is presented in an operationalizable and replicable manner. Its goal is to encompass a specific body of literature on the phenomenon of violence against healthcare professionals and the relationship between the bullying suffered and the coping strategies used to face these most stressful moments, examine its research trends, and potentially pave the way for a systematic review to pinpoint research gaps and provide recommendations for future studies. Therefore, we sought to identify the types of bullying behaviours that affect healthcare professionals and examine the possible consequences of these behaviours in terms of physical and mental health. Such as investigating the coping strategies that healthcare professionals adopt to deal with bullying in the workplace. To this end, the following questions arise:
  • What are the main bullying behaviours identified by healthcare professionals and what are their consequences?
  • What coping strategies are used by healthcare professionals who are victims of bullying?

2. Methodology

2.1. Research Procedures

For this scoping review, a search was carried out in the following databases: EBSCO, Web of Science, and PubMed. To this end, the following combinations of keywords were used: (Profissionais de saúde OR healthcare professionals OR médicos OR doctors OR physicians OR nurses OR enfermeiros) AND (Bullying OR mobbing OR *civility) AND (Estratégia* de coping OR coping strategie* OR cope OR coping OR coping skill*) OR (NAQ-R OR Negative Acts Questionnaire-Revised). To obtain the most relevant results from all databases, some restrictions were used. In the EBSCO database, the following restrictions were used: “Full Text”, “Peer Reviewed”, “Available to library collection”. In Web of Science and PubMed, they were restricted to “Open Access” and “Full Text”, respectively. On all platforms, the language was restricted to Portuguese, English, and Spanish, and the search was restricted to article dates greater than the year 2010 (Supplementary Materials). The research was carried out in May 2023.

2.2. Inclusion and Exclusion Criteria

In order to guide the selection of studies, some criteria were defined:
Inclusion criteria: (a) studies that addressed violence against healthcare professionals and coping strategies; (b) studies in which the population was doctors and/or nurses; (c) studies in Portuguese, English, and Spanish; (d) publications dated after 2010.
Exclusion criteria: (a) studies with a student population; (b) scoping reviews, systematic reviews, or meta-analyses; (c) studies carried out with professionals other than doctors and/or nurses; (d) studies that covered only one of the variables under study; (e) publications written in a language other than Portuguese, English, and Spanish; (f) publications dated earlier than 2010 (a report on the Assessment of Episodes of Violence Against Health Professionals was released by the General Directorate of Health [29]). By focusing on a period after 2010, we can ensure that the data and findings discussed in the report are still accurate and reflective of the current situation regarding violence against health professionals. Additionally, as the review was carried out in May 2023, it is important to consider more recent information to provide the most current and reliable analysis and recommendations.

2.3. Article Selection

The study followed the guidelines for scoping review [30] and was conducted in multiple stages involving article selection based on predefined criteria [31]. The project was registered [DOI 10.17605/OSF.IO/H7TAS]. The process ultimately yielded a final set of articles for analysis (cf. Figure 1) [32].
Initially, a search across three electronic databases yielded a total of 276 articles (117 from EBSCO, 43 from PubMed, and 116 from Web of Science). After excluding 75 duplicate articles, the remaining 201 articles underwent further screening.
During the second stage, articles were reviewed based on title and abstract to exclude those not relevant to the dissertation topic. This step resulted in the exclusion of 191 articles (86 from EBSCO, 13 from PubMed, and 102 from Web of Science): 65 articles focused on only one variable, 62 were unrelated to the intended topic, 52 did not study the intended sample, and 12 were literature reviews.
In the final stage, the ten selected articles were thoroughly reviewed to identify their relevance to the study objectives. As a result, eight articles were deemed suitable for inclusion, while two were excluded for not meeting the defined criteria—one did not study the intended sample, and the other focused on only one of the variables of interest in this study.

2.4. Article Quality Analysis

To carry out the analysis of the quality of the articles, a checklist [32] was used in which 13 evidence-based items (Table 1) were evaluated to assess the solidity of the articles. In this checklist, six important points are evaluated: (I) the clarity of the title; (II) the structure of the summary; (III) the state of the art and the objectives of the study; (IV) the study protocol, eligibility criteria, sources of information, inclusion/exclusion criteria, information collection and study design; (V) summary of the main results; (VI) limitations found in the study.
In the checklist, articles are classified using a Likert scale in which the score varies between 0 and 2. A score of 0 indicates that the study lacks information on the topic at hand, whilst a score of 1 indicates that the researchers briefly acknowledged the problem without offering thorough explanations. A score of 2 indicates that the researchers extensively examined the issue and presented their technique in a clear and understandable way. Two researchers independently examined each article using the same grid. Items with different scores were assessed together to achieve the final values shown in Table 1. The researchers agreed by 95%. Table 1 shows the eight articles selected for this scoping review.

3. Results

The articles analysed all highlighted the negative impact of workplace violence on healthcare professionals in both their professional and personal lives. To conduct a more focused analysis, key data were extracted from each article, considering the fact that they were all based on empirical studies. Table 2 includes information on the authors and publication year, goals, countries where the studies were conducted, sample size, research instruments, design, and main findings related to the research questions. The full reading of the screened articles facilitated an exhaustive comparative analysis (horizontal and vertical) of the aspects studied, leading to an integrated cross-sectional analysis and extraction of the main results to answer the initial research questions.

3.1. Multiple Approaches to Workplace Bullying Research

Except for the study by Berry et al. [33], which is mixed in nature (i.e., both qualitative and quantitative), all of the other investigations were quantitative in nature. Some studies had more specific goals, like examining the effects of bullying in the workplace and various symptoms of post-traumatic stress disorder caused by bullying, as well as anxiety states and the coping strategies used by nurses [33,35]. These selected empirical articles were based on objectives as diverse as determining the type and frequency of negative behaviours in the workplace as well as the coping strategies used when exposed to such behaviours [34,40]. In a different study that was examined, the authors aimed to determine whether bullying victims’ coping mechanisms were less adaptive to stressful situations than those of nurses who were not bullied and whether bullying and anxiety are mediated by coping mechanisms [38]. A few of the chosen studies sought to understand the mediating function of resilience in a study on workplace bullying and professionals’ quality of life, or they sought to ascertain the relationship between workplace bullying and nurses’ quality of life [37,39]. Another study that was chosen examined the possibility that nurses’ leadership and humour orientation styles could affect how they perceive bullying in the workplace [36].
Overall, the studies emphasize the multifaceted nature of workplace bullying research and the importance of considering various perspectives and factors in addressing this prevalent issue. The inclusion of both qualitative and quantitative approaches, as seen in the study by Berry et al. [33] allows for a more holistic understanding of the issue.

3.2. Methodological Aspects of International Research on Workplace Bullying

The studies provide a glimpse into various aspects of nursing practice and research from different countries, including Australia [34,39], Korea [35,40], the United States of America [33,36], Norway [38], and China [37]. Each study had specific inclusion criteria for participants, ranging from newly licensed nurses to experienced ones, working in different healthcare settings (e.g., clinicals, hospitals). The methods used for data collection also varied, from disseminating questionnaires in contexts to utilizing online platforms like Survey Monkey [36,39] and Google Forms [35]. Some studies [33,38] stood out for involving two phases of investigation, showcasing a longitudinal approach to understanding nursing practices. The recruitment methods also differed, from contacting nurses through professional organizations to utilizing social media platforms like WeChat [37].
In this review and for the development of empirical studies on the topic, it was particularly important to know the diversity of assessment instruments used and their scope.
Hawkins et al. [34] used two quantitative instruments and a group of questions with specific objectives. One of the instruments was the Negative Acts Questionnaire—Revised (NAQ-R) [41], which aims to measure exposure to bullying in the workplace, consisting of 22 items that are divided into three subscales. The other instrument was the Ways of Coping Questionnaire [23], designed to examine an individual’s coping strategies in stressful situations, consisting of 66 items from eight domains of three types. The group of questions was developed from the literature and consisted of nine items.
In the study by Yoo and Ahn [40], three quantitative instruments were used: the Workplace Bullying in Nursing—Type Inventory (WPBN-TI) [42], developed to measure experiences of bullying in the workplace and consisting of 16 items; the Workplace Bullying in Nursing—Consequence Inventory (WPBN-CI) [43], built to measure responses to bullying in the workplace and consisting of 13 items; and finally, the Way of Coping by Lazarus and Folkman [23], revised and translated into Korean by [44], used to measure coping strategies and consisting of 33 items.
Mills et al. [36] used three quantitative instruments to achieve the objective of the study: the Multidimensional Sense of Humour Scale (MSHS) [45], consisting of 24 items and used to assess the sense of humour; Bass’s Multifactor Leadership Questionnaire (MLQ) [46], a measuring instrument consisting of five factors to identify three leadership styles; and finally, the Negative Acts Questionnaire—Revised (NAQ-R) [41].
In the study by Hong et al. [35], the authors used three quantitative instruments: the Workplace Bullying in Nursing—Type Inventory (WPBN-TI) [42]; the Korean version of the Impact of Event Scale—Revised by Lim et al. [47], consisting of 22 scales divided into three subscales, used to assess nurses’ post-traumatic stress; the Korean version of the Ways of Coping Checklist (WCCL) by Park and Lee [48], modified from the original version by Folkman and Lazarus [23], consisting of 39 items divided into two subscales that aim to evaluate the coping strategies used by nurses.
Reknes et al. [38] used three quantitative instruments: the Negative Acts Questionnaire—Revised (NAQ-R) [41]; the Hospital Anxiety and Depression Scale (HADS-A) [49], used to measure the anxiety symptoms and consisting of seven items; and the reduced version of the Utrecht Coping List (UCL) [50], consisting of 22 items divided into two subscales, which measure the frequency with which respondents act in a specific way when faced with problems or unpleasant situations, defining seven coping styles.
The study by Peng et al. [37] used a total of three quantitative instruments: the Negative Acts Questionnaire—Revised (NAQ-R) [41]; the Chinese version of the 10-item Connor–Davidson Resilience (CD-RISC-10) [51], consisting of 10 items and used to assess psychological resilience; and the Chinese version of Professional Quality-of-Life Scale (ProQOL-CN) by Shen et al. [52], consisting of 25 items divided into three subscales.
Tabakakis et al. [39] used three quantitative instruments: the 10-item Conner–Davidson Resilience (CD-RISC-10) [51]; the Negative Acts Questionnaire—Revised (NAQ-R) [53]; and finally, the Practice Environment Scale of the Nursing Work Index (PES-NWI) [54], consisting of 31 items divided into five subscales and used to evaluate the work environment.
Regarding the study by Berry et al. [33], four quantitative instruments and interviews were used. The four instruments used were the Negative Acts Questionnaire—Revised (NAQ-R) [41], the 10-item Perceived Stress Scale [55], which aims to assess perceived stress in the last month, the 20-item subscale of the State Trait Anxiety Inventory [56] that assesses how participants feel at the time of the survey, and finally, the Post-traumatic Stress Disorder Checklist—Civilian Version (PCL-C) [57], which tracks post-traumatic stress symptoms and contains seventeen assessment items. In relation to the second part of the study, telephone interviews were carried out with a semi-structured script which addressed the behaviours suffered from bullying, what they did to deal with or prevent these behaviours, and other actions taken to continue working in the same hospital unit.
Overall, the diversity in research environments, participant criteria, data collection methods, and recruitment strategies in the studies reviewed highlights the broad scope and depth of healthcare professionals practice and research around the world.

3.3. Bullying Behaviours, Consequences, and Coping Strategies

Some articles in this review mentioned the main workplace bullying behaviours that their participants were exposed to. In studies by Hawkins et al. [34] and Berry et al. [33], nurses who were victims of bullying expressed that the most common type of behaviour was work-related bullying, that is, exposure to excessive workloads, being humiliated or ridiculed in relation to their work, and receiving tasks with delivery of impossible deadlines, among others. The healthcare professionals in the study by Hong et al. [35] also mentioned that they received verbal attacks and inadequate work instructions.
Some authors also mentioned the negative impacts that healthcare professionals had after being exposed to various acts of bullying in the workplace. For example, in the study by Yoo and Ahn [40], nurses complained of helplessness, depression, stress, insomnia, and physical discomfort, and even complained that they made more mistakes in their work and that they wanted to change jobs. They also reported that long-term exposure to bullying behaviours significantly reduces nurses’ coping resources [37].
More than 50% of the participants in three of the articles studied reported that they already had the intention of leaving their current job, changing units, or even giving up the nursing profession, since the bullying behaviours to which they were exposed were a very important factor for dissatisfaction with current employment, which is a major consequence of workplace violence [34,39,40].
These results demonstrate that the bullying suffered by these healthcare professionals can have negative effects in physical and mental terms and also interfere with the quality of the service they provide and the quality of personal life.
Studies have shown that nurses used different coping strategies, both active and passive, to deal with the most stressful moments of bullying in the workplace. Some of the strategies used were, for example, turning to a family member or friend for advice, focusing on what you had to do next, finding a “friend to work with”, avoiding anyone who exposes you to these behaviours, and using music and prayer as a form of distraction, among others [33,34].
Studies that identify which strategies are used by healthcare professionals indicate that, according to the answers given in surveys, problem-focused coping strategies are the most used in this profession, such as focusing on what had to be done next, trying to analyse the problem to better understand it, and focusing on work to distract your mind [33,34].

4. Discussion

The main bullying behaviours identified by healthcare professionals in the workplace include personal attacks, such as insults, humiliation, and ridicule, as well as work-related behaviours like excessive workloads and tasks with impossible deadlines. Understanding workplace bullying behaviours typically involves retrospective self-report questionnaires, where participants recall and report acts, they may have experienced. The choice of assessment instruments depends on the specific goals of each study and the cultural context in which it is conducted. Researchers utilize various instruments to measure exposure to bullying, coping mechanisms, psychological effects, and organizational factors. This diversity in assessment tools contributes to a comprehensive understanding of the incidence, consequences, and associations of workplace bullying in healthcare settings. To explore the temporal relationships between exposure to workplace bullying, coping strategies, and outcomes such as psychological well-being among healthcare workers, mixed-method studies with a longitudinal design are important [33,38]. Such studies can provide insights into how experiences of bullying evolve over time and their impact on individuals’ well-being.
Upon reviewing the various themes and investigations within the scope of this review, it becomes evident that bullying constitutes a detrimental act with far-reaching consequences for healthcare professionals, spanning physical, mental, professional, and quality-of-life domains [19,20,58]. Individuals subjected to frequent or daily bullying at work often experience heightened levels of stress, anxiety, helplessness, depression, and post-traumatic symptoms [33,34,35]. Furthermore, healthcare professionals may lose their motivation to work in such environments, leading to the desire to change jobs or even leave the profession altogether [37]. Notably, workplace violence detrimentally impacts job satisfaction, ultimately affecting clinical decision making and performance [34,39,40], particularly among nurses who often bear the brunt of workplace bullying [7].
Identifying high-risk subgroups among healthcare professionals susceptible to workplace bullying underscores the need for targeted interventions aimed at mitigating violence and its repercussions [35]. The findings hold crucial implications for specific professional groups, such as nurses, necessitating a comprehensive understanding of prevalent bullying behaviours, including excessive workloads and verbal attacks, to enable effective recognition and resolution [7]. Given the adverse effects of bullying, such as helplessness, stress, and attrition from the profession, urgent interventions are warranted to support and safeguard healthcare workers [19].
Resilience emerges as a critical mediator in the association between workplace bullying and professional quality of life [37,39], highlighting the significance of resilience-focused programs for healthcare professionals, particularly early in their careers. These initiatives should address structural empowerment’s impact on professional satisfaction and burnout, while also enhancing interpersonal interactions and addressing vulnerabilities (primary prevention). Additionally, interventions aimed at mitigating the adverse effects of workplace violence (secondary or tertiary prevention) must be implemented promptly and continuously to curb the phenomenon [19]. Proposed interventions may include skills training, cognitive-based problem-solving programs, and education on workplace violence [59,60,61]. Providing support for victims and implementing beneficial activities to alleviate perceived negative repercussions are equally essential components of intervention efforts.
Moreover, the role of leadership styles and organizational culture is pivotal in preventing bullying within the healthcare context. Cultivating supportive and respectful work environments, where leaders engage with staff, employ positive interaction methods, and establish long-term goals, serves as a deterrent to workplace bullying [36]. For healthcare organizations and management teams, these findings underscore the imperative of fostering a conducive workplace environment to prevent bullying and safeguard the well-being of healthcare professionals. By understanding the coping strategies employed by healthcare professionals, organizations can furnish appropriate resources and support to help employees effectively manage workplace bullying [7]. Employers should assess the type and frequency of bullying incidents using empirical measures (see Section 3.2) and develop strategies to manage and reduce stress, combat employee fatigue, and bolster coping mechanisms associated with exhaustion, should workplace bullying be identified.
Despite the prevalence of bullying experienced by healthcare professionals, many employ coping strategies such as problem-focused mechanisms and seeking social support to mitigate its impact on their lives and professions [34]. However, studies indicate that bullied healthcare professionals exhibit lower frequencies of goal-oriented active coping strategies, which are effective only under minimal exposure to bullying behaviours [38].
Nevertheless, this scoping review has its limitations. Despite thorough searches, some relevant studies may have been missed due to unavailability or inaccessibility. Additionally, studies with nonsignificant findings might not have been included, potentially biasing the review. Nonetheless, this review contributes to understanding bullying against healthcare professionals and their coping strategies.
Future research should concentrate on developing and evaluating interventions to prevent and address workplace bullying in healthcare settings. Longitudinal studies can offer insights into intervention effectiveness over time and identify best practices for supporting healthcare professionals. Moreover, investigating the long-term effects of workplace bullying on the physical and mental health of healthcare professionals, as well as patient care quality, is essential. Exploring the effectiveness of training programs, support systems, and organizational policies can foster a positive work environment, enhance satisfaction and retention, and reduce instances of bullying. Addressing these issues will create safer, more supportive environments for healthcare professionals and ultimately improve patient care.

5. Conclusions

In conclusion, workplace bullying in healthcare settings poses a significant threat to the well-being and professional quality of life of healthcare professionals. The detrimental effects of bullying, such as stress, anxiety, depression, and attrition from the profession, highlight the urgent need for targeted interventions to support and safeguard healthcare workers. Resilience-focused programs, interventions to mitigate workplace violence, and support for victims are crucial components of efforts to address workplace bullying. Leadership styles and organizational culture play pivotal roles in preventing bullying, emphasizing the importance of cultivating supportive work environments. By understanding coping strategies and providing appropriate resources and support, healthcare organizations can effectively manage and reduce the impact of workplace bullying on healthcare professionals. Further investigation into workplace violence, particularly in sensitive sectors like healthcare, public safety, and education, is imperative due to its personal and societal ramifications. Collaboration among researchers, policymakers, and healthcare organizations is essential for translating research findings into actionable interventions that benefit healthcare professionals and enhance patient care.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph21040459/s1, File S1: Bullying against Healthcare Professionals and Coping Strategies: Scoping Review Protocol.

Author Contributions

A.R.V.R. was involved in the conceptualization, investigation, writing—original draft preparation. A.I.S. is responsible for the methodology, writing—review and editing, and funding acquisition. All authors have read and agreed to the published version of the manuscript.

Funding

This work was partially financed by national funds through the Foundation for Science and Technology (FCT) within the framework of the Research Centre for Child Studies (CIEC) of the University of Minho projects under the references UIDB/00317/2020 and UIDP/00317/2020.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data are publicly available due to the nature of the review.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Carvalho, M.; Matos, M.G. Bullying. In Dicionário Crime, Justiça e Sociedade; Maia, R., Nunes, L., Caridade, S., Sani, A., Estrada, R., Nogueira, C., Fernandes, H., Afonso, L., Eds.; Edições Sílabo: Lisbon, Portugal, 2016; pp. 67–69. [Google Scholar]
  2. Munro, C.E.; Phillips, A.W. Bullying in the workplace. Surgery 2023, 10, 653–658. [Google Scholar]
  3. Brodsky, C.M. The Harassed Worker; Lexington Books: Toronto, ON, Canada, 1976. [Google Scholar]
  4. Leymann, H. Mobbing and Psychological Terror at Workplaces. Violence Vict. 1990, 5, 119–126. [Google Scholar] [CrossRef] [PubMed]
  5. Leymann, H. The content and development of mobbing at work. Eur. J. Work. Organ. Psychol. 1996, 5, 165–184. [Google Scholar] [CrossRef]
  6. Hirigoyen, M. Assédio, Coacção e Violência no Quotidiano; Pergaminho: Lisbon, Portugal, 1999. [Google Scholar]
  7. American Nurses Association. American Nurses Association Position Statement on Incivility, Bullying, and Workplace Violence. 2015. Available online: http://nursingworld.org/DocumentVault/Position-Statements/Practice/Position-Statement-on-Incivility-Bullying-and-Workplace-Violence.pdf (accessed on 20 March 2024).
  8. Layne, D.M.; Nemeth, L.S.; Mueller, M. Negative Behavior Among Healthcare Professionals: Integrative Review of Instruments. J. Nurs. Meas. 2020, 28, E30–E65. [Google Scholar] [CrossRef] [PubMed]
  9. Einarsen, S.; Hoel, H.; Zapf, D.; Cooper, C.L. The concept of bullying and harassment at work: The European tradition. In Bullying and Harassment in the Workplace: Developments in Theory, Research, and Practice; Einarsen, S., Hoel, H., Zapf, D., Cooper, C.L., Eds.; CRC Press: Boca Raton, FL, USA, 2011; pp. 3–39. [Google Scholar]
  10. Lee, H.; Ryu, Y.M.; Yu, M.; Kim, H.; Oh, S. A Qualitative Meta-Synthesis of Studies on Workplace Bullying among Nurses. Int. J. Environ. Res. Public Health 2022, 19, 14120. [Google Scholar] [CrossRef] [PubMed]
  11. Barros, C.; Meneses, R.F.; Sani, A.; Baylina, P. Workplace Violence in Healthcare Settings: Work-Related Predictors of Violence Behaviours. Psych 2022, 4, 516–524. [Google Scholar] [CrossRef]
  12. Pompeii, L.A.; Schoenfisch, A.L.; Lipscomb, H.J.; Dement, J.M.; Smith, C.D.; Upadhyaya, M. Physical assault, physical threat, and verbal abuse perpetrated against hospital workers by patients or visitors in six U.S. hospitals. Am. J. Ind. Med. 2015, 58, 1194–1204. [Google Scholar] [CrossRef]
  13. Rahm, G.; Rystedt, I.; Wilde-Larsson, B.; Nordström, G.; Strandmark, K.M. Workplace bullying among healthcare professionals in Sweden: A descriptive study. Scand. J. Caring Sci. 2019, 33, 582–591. [Google Scholar] [CrossRef]
  14. Sun, P.; Zhang, X.; Sun, Y.; Ma, H.; Jiao, M.; Xing, K.; Kang, Z.; Ning, N.; Fu, Y.; Wu, Q.; et al. Workplace Violence against Health Care Workers in North Chinese Hospitals: A Cross-Sectional Survey. Int. J. Environ. Res. Public Health 2017, 14, 96. [Google Scholar] [CrossRef]
  15. Antão, H.S.; Sacadura-Leite, E.; Manzano, M.J.; Pinote, S.; Relvas, R.; Serranheira, F.; Sousa-Uva, A. Workplace Violence in Healthcare: A Single-Center Study on Causes, Consequences and Prevention Strategies. Acta Médica Port. 2020, 33, 31–37. [Google Scholar] [CrossRef]
  16. Goh, H.S.; Hosier, S.; Zhang, H. Prevalence, Antecedents, and Consequences of Workplace Bullying among Nurses—A Summary of Reviews. Int. J. Environ. Res. Public Health 2022, 19, 8256. [Google Scholar] [CrossRef] [PubMed]
  17. Norton, P.; Costa, V.; Teixeira, J.; Azevedo, A.; Roma-Torres, A.; Amaro, J.; Cunha, L. Prevalence and Determinants of Bullying Among Health Care Workers in Portugal. Work Health Saf. 2017, 65, 188–196. [Google Scholar] [CrossRef]
  18. Torres, A.; Costa, D.; Sant’Ana, H.; Coelho, B.; Sousa, I. Assédio Sexual e Moral no Local de Trabalho; CITE: Lisbon, Portugal, 2016. [Google Scholar]
  19. Yosep, I.; Mardhiyah, A.; Hendrawati, H.; Hendrawati, S. Interventions for Reducing Negative Impacts of Workplace Violence Among Health Workers: A Scoping Review. J. Multidiscip. Health 2023, 16, 1409–1421. [Google Scholar] [CrossRef] [PubMed]
  20. Purpora, C.; Cooper, A.; Sharifi, C. The prevalence of nurses’ perceived exposure to workplace bullying and its effect on nurse, patient, organization and nursing-related outcomes in clinical settings: A quantitative systematic review protocol. JBI Database Syst. Rev. Implement. Rep. 2015, 13, 51–62. [Google Scholar] [CrossRef]
  21. Rai, A.; Agarwal, U.A. A review of literature on mediators and moderators of workplace bullying: Agenda for future research. Manag. Res. Rev. 2018, 41, 822–859. [Google Scholar] [CrossRef]
  22. Lazarus, R.; Folkman, S. Stress, Appraisal and Coping; Springer: Berlin/Heidelberg, Germany, 1984. [Google Scholar]
  23. Folkman, S.; Lazarus, R.S. If it changes it must be a process: Study of emotion and coping during three stages of a college examination. J. Pers. Soc. Psychol. 1985, 48, 150–170. [Google Scholar] [CrossRef]
  24. Rodriguez, S.Y.S.; Otto, M.S.; Barcinski, M. Estratégias de enfrentamento utilizadas por psicólogos: Uma análise de gênero [Coping strategies used by psychologists: A gender analysis]. Salud Soc. 2015, 6, 110–119. [Google Scholar] [CrossRef]
  25. Santos, N.A.R.; Gomes, V.S.; Rodrigues, C.M.A.; Santos, J.; Passos, J.P. Estratégias de enfrentamento utilizadas pelos enfermeiros em cuidados paliativos oncológicos: Revisão integrativa. Cogitare Enferm. 2016, 21, 1–8. [Google Scholar]
  26. Acquadro Maran, D.; Giacomini, G.; Scacchi, A.; Bigarella, R.; Magnavita, N.; Gianino, M.M. Consequences and coping strategies of nurses and registered nurses perceiving to work in an environment characterized by workplace bullying. Dialogues Health 2024, 4, 100174. [Google Scholar] [CrossRef]
  27. Folkman, S.; Lazarus, R.S. An Analysis of Coping in a Middle-Aged Community Sample. J. Health Soc. Behav. 1980, 21, 219–239. [Google Scholar] [CrossRef]
  28. Angst, R. Psicologia e resiliência: Uma revisão de literatura. Psicol. Argum. 2009, 27, 253–260. [Google Scholar] [CrossRef]
  29. General Directorate of Health. Relatório de Avaliação dos Episódios de Violência Contra os Profissionais de Saúde. Departamento da Qualidade na Saúde. 2011. Available online: https://app.parlamento.pt/webutils/docs (accessed on 20 March 2024).
  30. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L.; et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [PubMed]
  31. Mak, S.; Thomas, A. Steps for Conducting a Scoping Review. J. Grad. Med Educ. 2022, 14, 565–567. [Google Scholar] [CrossRef] [PubMed]
  32. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Moher, D. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, 71. [Google Scholar] [CrossRef] [PubMed]
  33. Berry, P.; Gillespie, G.; Fisher, B.; Gormley, D.; Haynes, J. Psychological Distress and Workplace Bullying Among Registered Nurses. Online J. Issues Nurs. 2022, 21, 8. [Google Scholar] [CrossRef] [PubMed]
  34. Hawkins, N.; Jeong, S.; Smith, T. Negative workplace behavior and coping strategies among nurses: A cross-sectional study. Nurs. Health Sci. 2020, 23, 123–135. [Google Scholar] [CrossRef] [PubMed]
  35. Hong, S.; Kim, H.; Choi, E.K.; Park, C.G. Workplace bullying and different levels of post-traumatic stress symptoms of nurses: A quantile regression approach for effective coping strategies. J. Nurs. Manag. 2021, 30, 1445–1453. [Google Scholar] [CrossRef] [PubMed]
  36. Mills, C.B.; Keller, M.; Chilcutt, A.; Nelson, M.D. No Laughing Matter: Workplace Bullying, Humor Orientation, and Leadership Styles. Work Health Saf. 2018, 67, 159–167. [Google Scholar] [CrossRef] [PubMed]
  37. Peng, J.; Luo, H.; Ma, Q.; Zhong, Y.; Yang, X.; Huang, Y.; Sun, X.; Wang, X.; He, J.; Song, Y. Association between workplace bullying and nurses’ professional quality of life: The mediating role of resilience. J. Nurs. Manag. 2021, 30, 1549–1558. [Google Scholar] [CrossRef]
  38. Reknes, I.; Einarsen, S.; Pallesen, S.; Bjorvatn, B.; Moen, B.E.; Magerøy, N. Exposure to bullying behaviors at work and subsequent symptoms of anxiety: The moderating role of individual coping style. Ind. Health 2016, 54, 421–432. [Google Scholar] [CrossRef]
  39. Tabakakis, C.K.; McAllister, M.; Bradshaw, J.; To, Q.G. Psychological resilience in New Zealand registered nurses: The role of workplace characteristics. J. Nurs. Manag. 2019, 27, 1351–1358. [Google Scholar] [CrossRef] [PubMed]
  40. Yoo, S.Y.; Ahn, H.Y. Nurses’ Workplace Bullying Experiences, Responses, and Ways of Coping. Int. J. Environ. Res. Public Health 2020, 17, 7052. [Google Scholar] [CrossRef] [PubMed]
  41. Einarsen, S.; Hoel, H.; Notelaers, G. Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised. Work Stress 2009, 23, 24–44. [Google Scholar] [CrossRef]
  42. Lee, Y.; Lee, M. Development and Validity of Workplace Bullying in Nursing-Type Inventory (WPBN-TI). J. Korean Acad. Nurs. 2014, 44, 209–218. [Google Scholar] [CrossRef] [PubMed]
  43. Lee, Y. Development of Workplace Bullying in Nursing-Consequence Inventory (WPBN-CI). J. Korean Acad. Ind. Coop. Soc. 2016, 17, 479–488. [Google Scholar] [CrossRef]
  44. Han, J.S.; Oh, K.S. A study on the grief and coping pattern of mothers who have a chronically ill child. Korean Nurse 1990, 29, 73–87. [Google Scholar] [CrossRef]
  45. Thorson, J.A.; Powell, F.C. Development and validation of a multidimensional sense of humor scale. J. Clin. Psychol. 1993, 49, 13–23. [Google Scholar] [CrossRef]
  46. Muenjohn, N.; Armstrong, A. Evaluating the structural validity of the Multifactor Leadership Questionnaire (MLQ), capturing the leadership factors of transformational-transactional leadership. Contemp. Manag. Res. 2008, 4, 704. [Google Scholar] [CrossRef]
  47. Lim, H.-K.; Woo, J.-M.; Kim, T.-S.; Kim, T.-H.; Choi, K.-S.; Chung, S.-K.; Chee, I.S.; Lee, K.-U.; Paik, K.C.; Seo, H.-J.; et al. Reliability and validity of the Korean version of the Impact of Event Scale-Revised. Compr. Psychiatry 2009, 50, 385–390. [Google Scholar] [CrossRef]
  48. Park, A.S.; Lee, Y.H. Influence of the sex role characteristics and stress perception of married women on the coping style for stress. Korean J. Couns. Psychother. 1992, 4, 69–79. [Google Scholar]
  49. Zigmond, A.S.; Snaith, R.P. The Hospital Anxiety and Depression Scale. Acta Psychiatr. Scand. 1983, 67, 361–370. [Google Scholar] [CrossRef] [PubMed]
  50. Eriksen, H.R.; Olff, M.; Ursin, H. The CODE: A revised battery for coping and defense and its relations to subjective health. Scand. J. Psychol. 1997, 38, 175–182. [Google Scholar] [CrossRef] [PubMed]
  51. Campbell-Sills, L.; Stein, M.B. Psychometric analysis and refinement of the connor–davidson resilience scale (CD-RISC): Validation of a 10-item measure of resilience. J. Trauma. Stress 2007, 20, 1019–1028. [Google Scholar] [CrossRef] [PubMed]
  52. Shen, J.; Yu, H.; Zhang, Y.; Jiang, A. Professional quality of life: A cross-sectional survey among Chinese clinical nurses. Nurs. Health Sci. 2015, 17, 507–515. [Google Scholar] [CrossRef] [PubMed]
  53. Nielsen, M.; Notelaers, G.; Einarsen, S. Measuring exposure to workplace bullying. In Bullying and Harassment in the Workplace: Developments in Theory, Research, and Practice; Einarsen, S., Hoel, H., Zapf, D., Cooper, C.L., Eds.; CRC Press: Boca Raton, FL, USA, 2011; pp. 149–174. [Google Scholar]
  54. Lake, E.T. Development of the practice environment scale of the Nursing Work Index. Res. Nurs. Health 2002, 25, 176–188. [Google Scholar] [CrossRef] [PubMed]
  55. Cohen, S.; Kamarck, T.; Mermelstein, R. A global measure of perceived stress. J. Health Soc. Behav. 1983, 24, 385–396. [Google Scholar] [CrossRef]
  56. Spielberger, C.D.; Edwards, C.D.; Montuori, J.L.R. State-Trait Anxiety Inventory for Children; Consulting Psychologist Press: Palo Alto, CA, USA, 1976. [Google Scholar]
  57. Blanchard, E.B.; Jones-Alexander, J.; Buckley, T.C.; Forneris, C.A. Psychometric properties of the PTSD Checklist (PCL). Behaviour Research and Therapy 1996, 34, 669–673. [Google Scholar] [CrossRef] [PubMed]
  58. Hampton, D.; Rayens, M.K. Impact of Psychological Empowerment on Workplace Bullying and Intent to Leave. J. Nurs. Adm. 2019, 49, 179–185. [Google Scholar] [CrossRef]
  59. Asi Karakaş, S.; Okanli, A. The Effect of Assertiveness Training on the Mobbing That Nurses Experience. Work. Health Saf. 2015, 63, 446–451. [Google Scholar] [CrossRef]
  60. Botha, E.; Gwin, T.; Purpora, C. The effectiveness of mindfulness based programs in reducing stress experienced by nurses in adult hospital settings: A systematic review of quantitative evidence protocol. JBI Database Syst. Rev. Implement. Rep. 2015, 13, 21–29. [Google Scholar] [CrossRef]
  61. Glass, N.; Hanson, G.C.; Anger, W.K.; Laharnar, N.; Campbell, J.C.; Weinstein, M.; Perrin, N. Computer-based training (CBT) intervention reduces workplace violence and harassment for homecare workers. Am. J. Ind. Med. 2017, 60, 635–643. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Article selection flowchart (retrieved from http://www.prisma-statement.org/, accessed on 20 march 2024).
Figure 1. Article selection flowchart (retrieved from http://www.prisma-statement.org/, accessed on 20 march 2024).
Ijerph 21 00459 g001
Table 1. Quality table of articles under analysis.
Table 1. Quality table of articles under analysis.
Articles/Topics12345678910111213Total
Berry et al. [33]111122212221119 Average
Hawkins et al. [34]222222222221225 High
Hong et al. [35]222222222222226 High
Mills et al. [36]122221222222022 High
Peng et al. [37]122222222222124 High
Reknes et al. [38]112222212222223 High
Tabakakis et al. [39]122122212222122 High
Yoo & Ahn [40]111122222222020 Average
Topics analysed: 1—title; 2—abstract; 3—rationale; 4—objectives; 5—protocol; 6—eligibility criteria; 7—information sources; 8—inclusion/exclusion criteria; 9—data collection process; 10—study design; 11—main measures; 12—summary of main results; 13—conclusion and study limitations. Grades: 0—not reported/not specified; 1—unclear/reported to a certain extent; 2—adequately done.
Table 2. Summary of selected articles.
Table 2. Summary of selected articles.
Authors
(Year)
GoalsLocal/
Participants
InstrumentsDesignMain Results
[Bullying, Symptoms, and Coping Strategies]
Berry et al.
[33]
Determine differences in the perception of stress, state anxiety, and post-traumatic stress symptoms based on levels of exposure to bullying, as well as determine the strategies usedUSA
1st phase = 84 nurses
2nd phase = 11 nurses
Negative Acts Questionnaire;
10-Item Perceived Stress Scale;
20-Item Subscale of the State Trait Anxiety Inventory;
Post-traumatic Stress Disorder Checklist Civilian Version;
Individual Questionnaire
Quantitative and qualitative studyThe study identified significant differences in the perception of stress, anxiety, and post-traumatic symptoms that have been reported by people with frequent or daily exposure to workplace bullying behaviour.
Hawkins et al.
[34]
Understand the type and frequency of negative behaviours in the workplace and the strategies used when exposed to these behavioursAustralia
74 nurses
Negative Acts Questionnaire—Revised;
Purpose-designed questions;
Ways of Coping questionnaire
Quantitative studyThe most common type of negative workplace behaviour reported was “work-related bullying”, and they reported using a variety of coping strategies, including problem-focused strategies and seeking social support.
Hong et al.
[35]
Investigate the effects of workplace bullying and different symptoms of post-traumatic stress and coping among hospital nursesKorea
233 nurses
Workplace Bullying in Nursing—Type Inventory;
Impact of Event Scale—Revised;
Ways of Coping Checklist
Quantitative studyThe study explores bullying in the workplace of nurses, detecting high-risk subgroups, and suggesting the development of coping interventions to reduce workplace bullying and symptoms of post-traumatic stress.
Mills et al.
[36]
Determine whether nurses’ humour orientation styles and leadership styles can influence perceptions of workplace bullyingUSA
459 participants
Multidimensional Sense of Humour Scale;
Bass’s Multifactor Leadership Questionnaire;
Negative Acts Questionnaire—Revised
Quantitative studyOne of the four humour subscales, Humour Appreciation, affected perceptions of workplace bullying. The other three, Humour Recognition, Humour Production and Humour for Coping, had no effect. However, managers’ leadership styles affected reports of negative acts.
Peng et al.
[37]
Determine the relationship between workplace bullying and nurses’ quality of life and also the mediating role of resilience between workplace bullying and quality of lifeChina
493 nurses
Negative Acts Questionnaire—Revised;
10-item Connor–Davidson Resilience;
the Chinese Version of Professional Quality-of-Life Scale
Quantitative studyBullying in the workplace had negative and direct effects on nurses’ quality of professional life. Resilience mediated the relationship between workplace bullying and quality of professional life.
Reknes et al.
[38]
To investigate whether bullied nurses have a more negative coping style when faced with stressful events than nurses who are not bullied and to determine whether coping style moderates the relationship between bullying and anxietyNorway
1st phase = 2059 participants
2nd phase = 1582 participants
Negative Acts Questionnaire—Revised;
Hospital Anxiety and Depression Scale;
Utrecht Coping List
Quantitative studyBullied nurses use a goal-oriented active coping style less frequently than non-bullied nurses. Furthermore, active goal-oriented coping appears to be beneficial only when exposure to bullying behaviours
is very low. Victims of bullying appear to deal more negatively with stressful events than others.
Tabakakis et al.
[39]
Investigate the impact of workplace factors on nurses’ psychological resilience, including bullyingAustralia
480 nurses
The Connor–Davidson Resilience 10 scale;
The Practice Environment Scale of the Nursing Work Index;
Negative Acts Questionnaire—Revised
Quantitative studyThe work environment and perception of exposure to workplace bullying play a significant role in shaping nurses’ psychological resilience.
Yoo & Ahn
[40]
Analyse the relationship between workplace bullying experiences, responses, and coping strategiesKorea
113 nurses
Workplace Bullying in Nursing-Type Inventory;
Workplace Bullying in Nursing—Consequence Inventory;
Way of Coping
Quantitative studyNurses complained of helplessness, depression, stress, insomnia, and physical discomfort, and complained that they made more mistakes in their work and wanted to change jobs.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ribeiro, A.R.V.; Sani, A.I. Bullying against Healthcare Professionals and Coping Strategies: A Scoping Review. Int. J. Environ. Res. Public Health 2024, 21, 459. https://doi.org/10.3390/ijerph21040459

AMA Style

Ribeiro ARV, Sani AI. Bullying against Healthcare Professionals and Coping Strategies: A Scoping Review. International Journal of Environmental Research and Public Health. 2024; 21(4):459. https://doi.org/10.3390/ijerph21040459

Chicago/Turabian Style

Ribeiro, Ana Rita Valente, and Ana Isabel Sani. 2024. "Bullying against Healthcare Professionals and Coping Strategies: A Scoping Review" International Journal of Environmental Research and Public Health 21, no. 4: 459. https://doi.org/10.3390/ijerph21040459

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop