**Preface to "Social Public Health System and Sustainability"**

This Special Issue reprint contains 18 articles published in *Sustainability* from late 2018 to early 2021. During that time, the world faced the fatal and widespread health crisis, COVID-19, which had threatened the social and public health systems at every corner for quite some time.

As the Guest-Editors and also a contributing authors, we are glad that the academic contents from the Special Issue will now be put together in this reprint, making the authors' hard work and efforts accessible to the larger audience.

We would like to thank the authors for spending time and effort contributing to the Special Issue. My appreciation also goes on to MDPI for organizing and supporting this SI volume.

> **Quan-Hoang Vuong and Khuat Thu Hong** *Editors*

## *Review* **Sustainable Harm Reduction Needle and Syringe Programs for People Who Inject Drugs: A Scoping Review of Their Implementation Qualities**

**Danielle Resiak 1,\*, Elias Mpofu 1,2,3,\* and Rodd Rothwell <sup>1</sup>**


**Abstract:** While substance use disorders (SUD) continue to be a global concern, harm reduction approaches can provide sustainable harm minimization to people who inject drugs (PWID) without requiring abstinence. Yet, the evidence for the sustainable implementation of harm reduction approaches is newly emerging. This scoping review sought to map the evidence on implementation qualities of sustainable harm reduction needle and syringe programs (NSPs). We searched the Cochrane Database of Systematic Reviews, PubMed, ProQuest Central, and Directory of Open Access Journals for empirical studies (a) with an explicit focus on harm minimization NSPs, (b) with a clearly identified study population, (c) that described the specific NSP implementation protocol, (d) that provided information on accessibility, affordability, and feasibility, and (e) were published in English between 2000–2020. Following narrative qualitative synthesis, the evidence suggests individual implementer characteristics directly influenced sustainable availability and scope of NSP provision while implementation processes explained the predictability and continuity of service provision across services. External factors including community perceptions of NSPs and policing activity influenced the sustainability of NSP implementation. The emerging evidence suggests that sustainable NSP programs for PWID require provider, consumer, and community engagement, supported by enabling health policies.

**Keywords:** NSP; harm reduction; harm minimization; low threshold settings; PWID; sustainable implementation qualities

#### **1. Introduction**

Substance misuse remains an ongoing health crisis affecting every region of the world, increasing the burden of disease globally [1]. Across the globe, approximately 250 million people use addictive substances every year of which 63.5 million have a substance use disorder (SUD) [2]. Despite substance use disorder mitigation efforts [3], relapse rates remain high at 40–60% [3], with mortality from drug and alcohol use disorders at 6.9 deaths per 100,000 globally [4]. Harm minimization approaches appear to hold promise for those with a history of addiction or dependence not wishing to obtain abstinence [5,6]. Harm reduction approaches that support people with a poor prognosis for abstinence-based treatment would make for sustainable needle and syringe program (NSP) practices [5]. Sustainable NSPs are human-centered, cost-effective, socially embedded, aligned to the health policies of jurisdictions [6], and offered at sufficient intensity to achieve program goals and population outcomes in the long-term [7].

While, harm reduction approaches to substance use dependency and addiction are increasingly being adopted, a paucity of research exists pertaining to the sustainability of their implementation protocols across service types [8]. More specifically, emerging

**Citation:** Resiak, D.; Mpofu, E.; Rothwell, R. Sustainable Harm Reduction Needle and Syringe Programs for People Who Inject Drugs: A Scoping Review of Their Implementation Qualities. *Sustainability* **2021**, *13*, 2834. https:// doi.org/10.3390/su13052834

Academic Editor: Quan-Hoang Vuong

Received: 29 December 2020 Accepted: 26 February 2021 Published: 5 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

evidence as to the sustainable implementation qualities of NSPs is yet to be aggregated. As such, we aimed to scope the evidence for implementation qualities of sustainable harm reduction NSPs. Such evidence would inform NSP services for those with a history of addiction or dependence for which abstinence would be less successful.

#### *1.1. Harm Reduction Approaches: Sustainable Implementation Considerations*

In the context of drug treatment policy, harm reduction refers to minimizing the health, social and economic costs of drug use to both individuals with addiction or dependency and the communities in which they participate [9]. Early adopters of harm reduction approaches included Australia, Canada, Portugal, Switzerland, the United Kingdom, and The Netherlands, and have since spread through Asia, Latin America, and Central Eastern Europe [10]. The sustainability of harm reduction policies and programs would greatly depend on their implementation design, responsiveness, and resourcing, alongside financial longevity, social acceptance, and accessibility. We aim to review the evidence on implementation design, responsiveness, and resourcing of harm reduction focused NSP.

Needle and syringe programs are low threshold services for people who inject drugs characterized by few to no access obstacles [11]. They comprise of primary, secondary, mobile and outreach services, syringe vending machines and pharmacies that sell or provide injecting equipment free of charge, predominately run within publicly funded health services [12]. The primary goal of an NSP is the distribution of sufficient injecting equipment, supported by educational interventions to reduce or eliminate the reuse of injecting equipment among people who inject drugs (PWID) [13]. Needle and syringe programs are proven to reduce the risks of blood borne virus transmission through the provision of sterile injecting equipment to people who inject drugs (PWID) [14] and come at comparatively lower cost than alternative approaches [15]. Despite their relative costeffectiveness, NSP implementation is largely shaped by a country or regions philosophical approach to drug treatment [16], often with little research evidence on implementation guidelines. Furthermore, while NSPs continue to operate funded through public health agencies, political, media or community campaigns mean they remain vulnerable to closure in the absence of the evidence for their sustainability [13].

#### *1.2. Health Policy Frameworks*

Health policy frameworks provide the context for implementation of programs aimed at aligning a country's priorities with its populations health needs, in partnership with government, health and development partners, civil society and the private sector for improved use of available resources [17]. Consensus building across multiple stakeholders in health policy framing slows the implementation of NSPs, given the politics of drug policy and stigmatization of people who inject drugs [18]. Moreover, policy framing and planning that occurs at all levels of a countries health care system can support sustainable NSP implementation due to increased public buy in [19]. Partners in sustainable NSP implementation would commit, if they perceived evidence of quality, affordability, acceptability, and accessibility [6]. We aim to apply a theory driven framework to aggregate the emerging research evidence on NSP program implementation qualities with PWID using the Consolidated Framework for Implementation Research (CFIR) [20].

#### *1.3. Implementation Study Framework*

The CFIR defines five implementation determinants: (1) individuals involved in service delivery (e.g., their knowledge and beliefs about the intervention), (2) the internal organization setting (e.g., leadership engagement), (3) the implementation processes (e.g., executing the innovation), (4) the program/intervention characteristics (e.g., complexity, accessibility, quality, affordability and acceptability) and (5) the external setting (client needs and resources). We expect sustainable NSPs would be designed to these qualities, yet the evidence is unclear as to how the interactions among these determinants would influence implementation outcomes. For instance, trust building among potential service

users would be influenced by implementation processes, particularly given that social stigma has been linked to the mis-trust of health services experienced by people who inject drugs (PWID) [13]. Similarly, the context of service provision by organizational, geographical, political, and cultural factors would influence the resourcing of NSPs [21], while acceptability of NSP services is dependent upon how much they are trusted by prospective adopters [13]. For those with a substance use disorder, social stigma is behind the mistrust of health services and therefore detracts from service engagement [13].

Legalization of NSPs increased their visibility which in some contexts, resulted in client arrests [22]. Noted as a structural barrier to the sustainability of an NSP is police arrest and prosecution activity. Arrest or prosecution of NSP clients when accessing a legal NSP undermines the trust PWID have in the inclusiveness of public health laws [23] and compounds the risk of unsafe drug injecting [22] while also deterring NSP service uptake. This in turn places PWID at a greater risk of blood borne virus transmission and overdose [23]. User-oriented NSP implementation can reduce stigma as it is mutually understood that PWID may not require assistance beyond the provision of clean injecting equipment and associated paraphernalia (tourniquets, alcohol wipes and so forth) [13]. There is no expectation from staff that PWID will require, nor want assistance for their drug use [13].

#### *1.4. Goal of the Review*

We aimed to conduct an exploratory scoping systematic review [24,25] to identify both gaps and trends in literature, clarifying definitions, and report practices that can inform future research and practice. A scoping systematic review is appropriate for summarizing the emerging evidence on sustainable NSP health interventions.

#### Objectives

This scoping review aims to map evidence for the sustainability of NSPs defined by the CFIR framework of implementation determinants inclusive of; program implementers and internal setting, implementation process and characteristics, and external factors. Our specific research question was: What is the emerging evidence pertaining to sustainable NSP implementation qualities including implementers, internal setting, implementation process and characteristics, and external factors?

Findings would be important for the sustainability of NSPs framed on implementation qualities evidence in harm reduction studies considering the perspective of those who are most directly affected by their operation. Moreover, the evidence would inform future studies on NSP implementation, providing benchmarks for evaluation and improvement at both an individual, program, and systems level.

#### **2. Materials and Methods**

#### *2.1. Search Strategy for Identification of Studies*

We searched the Cochrane Database of Systematic Reviews for previous reviews on the sustainable implementation of NSPs and yielded a null result, which justifies this systematic scoping review. We then searched electronic databases including PubMed, ProQuest Central and the Directory of Open Access Journals for peer-reviewed studies pertaining to NSP implementation, restricting our search to the period 2000 to 2020. Our selection of databases prioritized studies that described implementation protocols at a sufficient level of detail to allow for determination of their relevance to the study aims. Moreover, we searched field specific journals inclusive of *Substance Use and Misuse*, *Addiction, Aids and Behaviour, Harm Reduction Journal* and *The International Journal of Drug Policy* for relevant studies. We searched each database and journal using a combination of search terms inclusive of needle and syringe program implementation, implementation science, addiction, and NSPs (see Table 1).


**Table 1.** Overview of search procedure key topics, terms and criteria.

#### 2.1.1. Eligibility Criteria

We included for review empirical studies that (a) were published in English, and (b) had an explicit focus on harm minimization needle and syringe program implementation. To be eligible, studies were (i) case studies, cross-sectional, or longitudinal studies, (ii) on a clearly identified study population, (ii) with description of their specific harm reduction/NSP implementation procedure or protocol and (iv) with information on sustainability characteristics of user involvement in design accessibility, affordability, and feasibility. In doing so, we prioritized studies that included a process and/outcome evaluation of the harm reduction/NSP implementation protocol by users and/or providers as well as how program characteristics aligned with health care policy. We excluded studies that were in languages other than English and did not describe the harm reduction/NSP implementation protocol they used in a manner that enabled determination of their sustainability. See Table 2 for variable inclusion and exclusion criteria.


**Table 2.** Study variable inclusion and exclusion criteria.

#### 2.1.2. Search Tree Procedure and Outcomes

We retrieved a total of 722 articles for screening (see Figure 1), and screened the articles by titles and abstracts, excluding a total of 652 at this stage. Our selection process yielded a preliminary list of 70 articles for further scrutiny against inclusion criteria, excluding 42 that did not meet the inclusion criteria. The first and second listed authors then accessed the full text of the 28 articles, applying the pre-determined eligibility criteria. We excluded 23 articles at this stage leaving five articles. To optimize our search yield, we browsed the reference lists of each of the articles that met the inclusion criteria for any additional articles of relevance. From this manual search, we found an additional study, resulting in six articles for this study. Final study inclusion was by consensus between the first two listed authors, moderated by the third listed author as needed.

**Figure 1.** Search strategy.

#### *2.2. Data Extraction and Management*

For the data capture and organization relevant to the research aim, we utilized a modified version of Arksey and O'Malley [24] data extraction tool. This tool organizes according to the following categories: (1) author/year, (2) study design, (3) methods and recruitment, (4) description of study objectives, and (5) outcomes or findings [24]. Our modification was to include an implementation protocol description and evaluation. This method modification allowed for data mapping identifying both implementation approach and NSP outcomes. Table 3 presents the studies included from our search procedure.




#### *2.3. Data Analysis and Synthesis*

We summarized findings from the studies using narrative qualitative synthesis [30]. This data synthesis approach allowed for identifying NSP implementation determinants framed on the CRFI according to implementers and their inner setting, the implementation process, and its characteristics (e.g., quality, complexity, affordability, accessibility, and acceptability) and community based external factors.

#### **3. Results and Discussion**

Findings from our scoping review indicate the importance of implementer qualities (willingness and beliefs), implementation process factors (inclusion of PWID), program intervention characteristics (accessibility and acceptability) and external factors (policy, community acceptance and policing) for improved NSP sustainability. Above all, the evidence suggests sustainable harm reduction NSPs require support from multiple stakeholders, given the multifaceted requirements of their implementation [31], and interactions among each of the implementation determinants. The specific findings of each are discussed below.

#### *3.1. Implementers and Internal Setting Factors of Sustainable NSPs*

Implementer willingness and self-efficacy or beliefs about NSP feasibility would enhance sustainability [18], as would leadership resourcing [26,29]. For instance, pharmacists who agreed with the public health impact of NSPs were more likely to provide clean needles and syringes to PWID and dispose of used needles and syringes within the pharmacy [26]. Kentucky community pharmacists expressed 3.56 times more willingness to provide clean needles. However, the perceived barriers to selling needles and syringes without a prescription differed between Kentucky independent and chain/supermarket pharmacists with independent pharmacists reporting workflow barriers contrary to chain/supermarket pharmacists who reported clientele safety concerns as a barrier to selling needles and syringes without a prescription.

Local business support for NSPs influenced implementation feasibility in that they might show token support for NSPs at fixed sites as opposed to widespread automatic dispensing machines (ADM) [28]. Automatic dispensing machines are a sustainable and inexpensive method of increasing needle and syringe distribution to PWID. In Australia, although general support for harm reduction programs was high among survey business leaders, their awareness of such services operating in the immediate vicinity was less so. For instance, significantly lower proportions of businesses indicated awareness of fixed-site NSPs generally (63% vs. 83%, *p* = 0.01), the existence of pharmacy NSP (29% vs. 50%, *p* = 0.03) and an ADM (31% vs. 53%) (*p* = 0.03) in the local area comparatively to residents [28].

Peer implementers of NSPs with high literacy demonstrate service provision aligned to community sentiment and are more likely to remain engaged with NSP service provision then those less literate. As such, supporting improved literacy and employment retention could benefit the sustainability of peer led NSPs [27], suggesting both an education and internal program function.

#### *3.2. Sustainable Implementation Process and Characteristics*

NSP implementation process and characteristics of (a) community coalitions, (b) community activists or (c) bottom-up approaches enhanced NSP sustainability [18], as did implementers flexibility to adopt emerging empirically based interventions [28]. This could be explained by the fact that grassroots based approaches have user buy-in, and flexible hours are a well-known low threshold service access quality. Evidence-based practices are important for legitimizing NSPs [29]. For instance, Strike, Watson, Lavigne, Hopkins, Shore, Young, Leonard and Millson [15] reported implementation success of the Ontario Needle Exchange Best Practice that followed evidence-based needle and syringe recommendations, including distribution of sterile water ampoules and safer

inhalation equipment. NSP processes that (a) are respectful of political and cultural norm sensitivities, and (b) prioritized coalition building and community involvement, would be more sustainable than those lacking in these qualities [29]. Greater community support for NSPs would enhance sustainability of implementation [18], so too would long-term financial commitment [8] minimizing risk for rapid depletion of the pool of available resources [29].

#### *3.3. Sustainable NSP External Setting Characteristics*

The study by Ngo, Schmich, Higgs and Fischer [27] conducted in Northern Vietnam found community support was a critical component of NSP implementation. A finding consistent with that of Downing, Riess, Vernon, Mulia, Hollinquest, McKnight, Jarlais and Edlin [29] whereby coalition building, and community consultation were deemed critical steps required for acceptability and sustainability of NSPs. To obtain community support required intensive advocacy with community stakeholders including local government, mass organizations, local residents, PWID and their families with the acquisition of law enforcement officials [27]. As an example, White, Haber and Day [28] reported on community attitudes to harm reduction and automatic dispensing machines (ADMs) in Sydney, Australia. ADMs are a sustainable and inexpensive method of increasing needle and syringe distribution to people who inject drugs. They reported local community opposition to ADMs despite national Australian data indicating support for harm reduction. Respondents to the study had concerns about possible increases in drug related crime. However, the majority of business leader participants were in support for NSP services in general (fixed-site NSPs (83%), pharmacy NSP (82%), and to a lesser extent, ADMs (67%)). Conversely, local businesses' support was slightly lower (fixed-site NSPs (77%); pharmacy NSP (80%), ADMs (60%)) [28].

External NSP setting characteristics such as partnerships between health, law enforcement, PWID, clinicians, researchers and government officials are essential for sustainable NSP implementation [8]. Furthermore, Downing, Riess, Vernon, Mulia, Hollinquest, McKnight, Jarlais and Edlin [29] suggest community NSP support is highest in places where HIV transmission presentation is a predominant community concern.

#### *3.4. Implementation Determinants Interaction Factors*

Implementer and internal setting factors at the policy level would influence the sustainable implementation of NSPs. For instance, Clark [32] suggests that drug treatment policies are influenced by national or regional prerogatives, perhaps more so than documented evidence for NSP programs. Yet, while the provision of research evidence does not guarantee policy change, it is a necessary step for sustainable implementation [18]. Moreover, policymakers may have varying opinions on the merits and moral obligations of expanding services to meet the needs of PWID [18]. Such merit and moral obligations result in policymakers struggling to implement evidence-based policies while simultaneously addressing electorate priorities [18].

Allen, Ruiz and O'Rourke [18] examined the role of research evidence in policy change processes for the sustainable implementation of publicly funded syringe exchange services in three US cities: Baltimore, MD, Philadelphia, PA, and Washington, DC. Results indicated sustainable implementation of NSPs in Baltimore and Philadelphia were dependent on research evidence application to secure policy change, conversely policy change discussions in DC were influenced by community and stakeholder fears and concerns that NSPs would increase both substance use related crime and the number of discarded syringes found in public locations. White, Haber and Day [28], also reported perceived increases in drug related crime and drug use a barrier to sustainable implementation, despite there being no empirical evidence to support such perceptions [8].

#### *3.5. Implications for NSP Implementation Sustainability Research and Practice*

In efforts to confront the health disparities among an estimated 15.9 million people who inject drugs globally [33] NSPs have been implemented. However, sustainable access to such services is not equal across the globe, with low and middle-income countries implementing NSPs at coverage levels below that required to stabilize and reverse HIV epidemics among PWID [18]. Commonly cited implementation barriers included funding, senior management, and decision-making. The primary weakness of government-initiated implementation models includes bureaucratic systems and susceptibility to pressure from community criticism creating an inability to respond quickly or flexibly change [29]. Our findings suggest that empirically based best practice recommendations are implemented successfully within NSPs when available. Additionally, community consultation at the design stage of protocol implementation, improves community acceptance.

Needle and syringe program's sustainability is dependent upon both their accessibility and continued utilization among the population base they aim to serve [34]. The World Health Organization [35] recommends NSPs distribute 200+ needles per PWID annually for their sustainable use [35]. However, substantial variability in NSP service provision, utilization, coverage, range, needle and syringe distribution and program reach are obstacles to their sustainable implementation [14]. Moreover, provider, structural, and societal barriers to NSP access for PWID [36], alongside communities lack of knowledge surrounding NSP objectives, policies, laws, regulations, locations and stigma hinder their sustainable implementation [34].

A longstanding societal value to prevent drug misuse rather than safer use of a person's substance of choice continues to run counter to NSP implementation [14]. This is despite NSPs being a proven health intervention for reducing the transmission of blood borne viruses among PWID [37]. For NSPs to be sustainable, they need to be feasible within the intended context [31]. Essential for improved health outcomes of PWID is mutual trust and communication with NSP providers. Vuong, et al. [38] suggest that when service providers are perceived not to have a genuine interest in a client's views an asymmetrical relationship presents placing the client in a vulnerable position. Furthermore, medical distrust results in reduced service engagement [38] which in turn affects its sustainability.

#### *3.6. Limitations of the Applicability of Evidence*

Few of the studies provided descriptions of their implementation protocols, and even fewer reported on their implementation process evaluation. This limited our ability to map the implementation protocols for sustainability qualities. We synthesized the data applying the CRFI framework, which is not exhaustive. Other studies may find a different profile from using alternative criteria. Furthermore, with very few studies that met our inclusion criteria, we could not determine the influence of publication bias, the tendency for statistically significant positive results to be published in greater proportions than those of statistically significant negative or null results [39]. An included study reported an attempted contact with opponents of NSPs that was not successful. As such, the perspectives of those prospective participants were not captured in the study results [29]. Furthermore, while we were able to capture results from studies conducted in Australia, Canada, the USA and Vietnam paucity of available literature addressing implementation qualities of sustainable NSPs impacts the scope of comparison between countries. Additionally, not all countries with a harm reduction drug treatment policy were represented in the findings further justifying the need to aggregate sustainable implementation qualities as they emerge.

#### **4. Conclusions**

We found evidence to suggest the importance of implementer resourcing, engagement, and willingness on the design and implementation of sustainable NSPs. Such factors would veritably translate across diverse implementer communities inclusive of program managers, peer educators, pharmacists, and NSP providers. Sustainable implementation

process factors aim to build ownership and trust of NSPs, as measured by relevance and accessibility to PWID. External factors such as NSP user-friendly law-enforcement, community support and cost-containment improve NSP sustainability. Overall, there is more literature evaluating specific areas of policy or practice, comparatively to NSP provider engagement and consumer responsive sustainable implementation. Similarly, the mapped evidence trends toward implementer process characteristics and external policy frameworks. Such findings could guide both new and existing NSP provision, evaluation, and adaptation to be more consumer responsive.

**Author Contributions:** D.R., conceptualized the study, wrote the initial draft, completed the data curation and implemented the methodology; D.R. and E.M., carried out the formal analysis of data; E.M., the project's primary supervisor, guided the research, writing—review and editing; R.R., the project's auxiliary supervisor, provided further review and ongoing writing review. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The article is work in partial fulfillment of a PhD thesis within the Faculty of Medicine and Health, Rehabilitation Counseling Discipline at the University of Sydney.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **An Appraisal of Communication Practices Demonstrated by Romanian District Public Health Authorities at the Outbreak of the COVID-19 Pandemic**

**Mariana Cernicova-Buca \* and Adina Palea \***

Department of Communication and Foreign Languages, Politehnica University Timisoara, 300006 Timisoara, Romania

**\*** Correspondence: mariana.cernicova@upt.ro (M.C.-B.); adina.palea@upt.ro (A.P.)

**Abstract:** Communication during an ongoing crisis is a challenging task that becomes even more demanding during a public health crisis. Early in the start of the pandemic, global leaders called upon the public to reject infodemics and access official sources. This article focuses on the communicative aspects of health services management, with a particular focus on the communication strategy of the Romanian district public health authorities during the COVID-19 lockdown, as seen on official websites and social networks. The 15 most affected districts were selected, according to the officially reported health cases. The issued press releases and the posts on Facebook pages show an uneven experience on the part of district authorities in dealing with public information campaigns. In addition, the results of the study indicate a lack of sustainable communication approaches as well as the need of professional training and strategy in dealing with the public health crisis. From a communication point of view, a strategic approach on behalf of the public health sector is crucial to enhance the preparedness of appropriate institutions to act during emergencies and to respond to the needs of the media and the public with timely, correct, and meaningful information.

**Keywords:** public health; public health authorities; public communication; risk communication; social networks; lockdown; crisis; COVID-19 pandemic; sustainability

#### **1. Introduction**

The ongoing public health threat posed by the COVID-19 pandemic challenged the modus operandi of public health authorities, governments, and even international organizations, due to the length and intensity of the crisis. Health is a fundamental right of every human being, as stated in the Constitution of the World Health Organization since 1946. It is also understood as a major element of sustainability; the United Nations 2030 Agenda for Sustainable Development included health distinctively as a goal (SG3), with actions being necessary to "strengthen the capacity of all countries ( ... ), for early warning, risk reduction and management of national and global health risks" [1]. However, the year 2020 made the international community aware that the sustainability goals—SG3 included—need rethinking and that addressing the weaknesses of domestic and global governance is a matter of utmost priority. [2]. As Miriam Bodenheimer and Jacob Leidenberger bluntly put it, the lack of ecological sustainability contributed to the coronavirus outbreak, the lack of economic sustainability to its rapid and global spread, and the lack of social sustainability to its severity [3]. The international community tried to make sense of and contain the pandemic, find the appropriate responses, and mobilize all forces to overcome the effects of the multi-level crisis brought on by COVID-19. As the World Health Organization (WHO) acknowledged "humbly", the fast-evolving situation made it difficult to anticipate the evolution of the situation. The organization also recognized that "there is no one-size-fits-all approach to managing cases and outbreaks of COVID-19" and advised the public to stay informed and follow the lead of healthcare providers as well as national

**Citation:** Cernicova-Buca, M.; Palea, A. An Appraisal of Communication Practices Demonstrated by Romanian District Public Health Authorities at the Outbreak of the COVID-19 Pandemic. *Sustainability* **2021**, *13*, 2500. https://doi.org/10.3390/ su13052500

Academic Editors: Quan-Hoang Vuong and Khuat Thu Hong

Received: 26 December 2020 Accepted: 22 February 2021 Published: 25 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

and local public health authorities [4]. While the full impact of the pandemic on society is still difficult to evaluate, despite the massive mobilization of researchers in all domains to offer responses to the multiple challenges encountered throughout 2020 [5–7], it seems possible to appraise the communication efforts undertaken by public health authorities in keeping the public informed and fostering compliance with the cascading measures adopted to limit the spread of COVID-19. It is important to reflect on the capitalization of this knowledge towards improving health risk communication and ensuring social sustainability in the post-crisis period [3,8]. The lessons learned from prior global security risks show that getting information out to the public in a timely manner is a must, "but so are adequate framings of the illness, stories of the heroic efforts of those on the front line, and galvanizing metaphors that can bind the community together even when the illness is unpredictable and the chances of scientific success are uncertain" [9] (p. 10).

As Jan Servaes highlights [10] (p. 1472), "communication and information play a strategic and fundamental role by (a) contributing to the interplay of different development factors, (b) improving the sharing of knowledge and information, and (c) encouraging the participation of all concerned". In all areas of sustainable development, including the case of sustainability in the health sector, this works by "facilitating participation: giving a voice to different stakeholders to engage in the decision-making process; making information understandable and meaningful. It includes explaining and conveying information for the purpose of training, exchange of experience, and sharing of know-how and technology; fostering policy acceptance: enacting and promoting policies that increase people's access to services and resources" [10].

The United Nations and the World Health Organization alike called upon member states to communicate intensively, consistently, and in a timely fashion with the public [11] to fight an infodemic, described as "an over-abundance of information—some accurate and some not—that makes it hard for people to find trustworthy sources and reliable guidance when they need it" [12] and to counter misinformation.

Therefore, our research aimed to identify the amount and type of communication efforts undertaken by the district health authorities to manage the ongoing public health crisis and contain the effects of the pandemic on the district community, while maintaining a meaningful and relevant dialogue with the public via technologically mediated channels.

The communication management of the risk communication of the COVID-19 pandemic is a rich field for understanding the interaction between public relations, health communication, journalism and the public. According to J. Barry, for the influenza pandemic, "the single most important weapon against the disease will be a vaccine. The next important weapon will be communication" [13]. COVID-19, due to the magnitude of the pandemic, already had all the features of a newsworthy event, with intense local implications. It dominated the public agenda, and people all over the world tried to make sense of their lives through the lenses of available information on the measures undertaken to contain the disease. For authorities, as sources of information, leadership and action, it was crucial to make information available both to the public and to journalists, as disseminators of information and partners in shaping the public agenda. Here, agenda-building theory, as a new wave in understanding the shaping of public messages in society, is an appropriate scientific framework [14–16]. As McCombs convincingly states, the "basic agenda-setting role of the news media is to focus public attention on a small number of key issues and topics. Although there are dozens of issues and other aspects of the world outside competing for attention, the news media can cover only that handful deemed most newsworthy" [16]. News concerning the spread of the disease and the measures taken internationally and nationally by far dominated the news cycle at the beginning of 2020.

Information subsidies are often involved in agenda building [17]. The term "information subsidy" generously encompasses press releases, information pieces, speeches and other types of organizational communication tools used by public relations practitioners as ready-made products for the media [15,18]. Most studies focus on agenda building for traditional media, while social media has already changed the realities of the circulation of

information. There is a strong expectation that members of the public have the same access to this information as members of the media, via online newsrooms [17]. This necessity is acknowledged also by the WHO in their insistence regarding the use of all communication channels, including social media sites, to fight the pandemic and bring relevant and correct information to the public [12,19]. Our research aims to enhance knowledge about agenda building through information subsidies like press releases and posts on social media platforms as made available by the Romanian public health authorities during the hottest period of the COVID-19 pandemic, i.e., in the early stages of the crisis in 2020. It fills gaps in the literature and aims at enhancing the body of knowledge regarding crisis and risk communication. We look only at the communication efforts undertaken by the public health authorities, as official sources of information that have the possibility to shape and set the agenda by providing timely, accurate and relevant information.

In the analyzed case of COVID-19, a spatial–temporal contextualization is needed to understand the specificity of public communication outcomes. The disease began as an outbreak in the Wuhan province of China, but it spread rapidly around the globe. The COVID-19 outbreak at the beginning of 2020 took Romania by surprise. Prior global health emergencies of the 21st century, such as the Ebola virus in 2014, H1N1 (Swine Flu) in 2009 and SARS in 2003, although creating public awareness that they lurked in the world, did not affect Romania. As in other societies where large-scale disasters were absent for decades, which is the case in many countries since World War II [20], the communication of risk mitigation measures during a major crisis adds significant challenges. The severe acute respiratory syndrome Coronavirus 2, known as SARS-Cov-2 or Novel Coronavirus 2, and ultimately referred to as COVID-19, is the first major health challenge for Romanian public life. The World Health Organization (WHO) declared COVID-19 a Public Health Emergency of International Concern on 30 January 2020 [4].

On 21 February, the Romanian government took its first COVID-19-related measures, announcing a 14-day quarantine for persons coming to Romania from disease-stricken regions. The first documented case in Romania occurred on 26 February 2020, but it did not stir public concern at the time [21]. However, it soon became obvious that the novel coronavirus had the features of a pandemic, as WHO reluctantly recognized 11 March 2020. On 14 March, after over 100 people had been diagnosed with the coronavirus, Romania had enough reasons for public health concern. On 16 March, President Klaus Iohannis announced his decision to decree a state of emergency for a 30-day period, which was prolonged until 14 May. After that date, Romania entered a state of alert, which meant the relaxation of some of the measures [22].

According to the principle of subsidiarity, the district (judet) level is immediately under the national one. It organizes all public life in the territory from an administrative point of view, including health issues. In Romania, there are 42 district public health authorities (DPHAs), 41 representing districts and one for the national capital, Bucharest. They represent the Ministry of Health at the local level and are responsible for the provision of public health services locally. DPHAs are responsible for the collection of data from the territory, the monitoring of the health of the population and health determinants and the identification of public health needs of communities [23]. The reform of public services in Romania, carried out in the post-communist period, shifted competencies from the central government towards local/regional bodies, but studies show that the burden is perceived at times as overwhelming [24,25].

In times of crisis, public authorities are expected to share knowledge, communicate with relevant audiences, find alliances in society and build confidence [8]. The WHO placed special emphasis on risk communication and community engagement as one of the eight pillars of successfully managing a health crisis [11]. The newly created "Risk Communication and Community Engagement" division of the WHO recommended authorities in all countries to "implement and monitor an effective action plan for communicating effectively with the public, engaging with communities, local partners and other stakeholders to help prepare and protect individuals, families and the public's health during

early response to COVID-19". The advice was unequivocal: "Make sure that this happens through diverse channels, at all levels and throughout the response" [26] (p. 3). Such channels, beyond discussion, are represented by social media owned by the appropriate authorities. In addition, as activities were now organized remotely and carried out through digital tools, many Romanians used their social media accounts, previously employed mainly for private purposes, to access services, information and labor from home [27].

The challenges brought about by the COVID-19 pandemic have underlined the importance of professional public communication based on correct, timely, comprehensive and accessible information. Social distancing was one of the measures recommended internationally [28] as an effective (though insufficient) means of preventing the spread of the virus. Face-to-face events were reduced to a minimum, and the classical press conferences, a preferred media event in the toolkit of authorities in dealing with journalists to disseminate information, were put to a halt. Only press releases posted on the authorities' websites remained as tools pertaining to the habitual media relations strategy.

This study reports on these aspects, as seen from the institutional websites and from the social media accounts of Romanian DPHAs, from the point of view of the amount and type of content posted to meet the expectations of the general public during the COVID-19 lockdown. The following research objectives were defined:

RO1: To determine whether there is a correlation between the number of health incidents and information subsidies made available to the public by relevant authorities.

RO2: To evaluate the communication efforts of DPHAs, expressed by the information subsidies (number of press releases on website and number of posts on social media platforms), from the point of view of timeliness and rhythmicity.

RO3: To examine the type of content disseminated by DPHAs via social media, from the point of view of ownership (produced in-house vs. share of content produced by other institutions) and triggered engagement (comments, shares).

#### **2. Materials and Methods**

The findings of this paper are based essentially on a qualitative research approach, i.e., on content data analysis, although quantitative indicators were used to enhance the results. Data collection took place at the beginning of the pandemic and covered the 2 months of total lockdown in Romania: 16 March–14 May 2020. Our strategy was to analyze the communication efforts of the District Public Health Authorities of the most affected regions. Considering the official information at the time, we selected a sample of 15 regions where the number of patients with specific COVID-19 symptoms kept the attention of authorities and the media (14 districts and the national capital, Bucharest). Drawing inspiration from the occurrence of national media reports in the regions, we switched to the data provided by the National Institute for Public Health for a final selection [29]. The order of presentation herein is based on the number of COVID-19 cases registered in the analyzed region in the first two weeks after the outbreak of the disease: Suceava, Bucharest, Timis, Neamt, Arad, Hunedoara, Brasov, Galati, Cluj, Constanta, Ilfov, Iasi, Mures, Botosani and Vrancea. The order changed in time, with the national capital being at the top, but we maintained the initial choice of presentation in order to follow through with the data in a coherent manner.

As formulated in the research objectives, we monitored the frequency of press releases throughout the sample. For the analysis of website traffic, we chose a popular content marketing platform—SEMrush. It is among the most used platforms of its kind and has a friendly interface that allows for in-depth data collection [30]. The SEMrush dashboard enabled us to gather information about organic traffic, organic key words, the source of visitors on DPHA websites, etc., all of which helped us draw conclusions on the impact of the information posted on their webpages.

Moreover, we identified the existence of Facebook, Twitter, Instagram and YouTube accounts and examined the communication behavior of DPHAs in the social media world. We gathered a corpus of 412 Facebook posts, which was analyzed from the point of view of engagement, focusing on the number of shares and comments per post. Monitoring of the

websites and social media accounts curated by DPHAs was followed by an analysis of the type of content related to COVID-19. Experts acknowledge that "failing to engage users equates with ( ... ) no transmission of information from a Web site; people go elsewhere to perform their tasks and communicate with colleagues and friends" [31]. Thus, we also traced the capacity of the analyzed public health authorities to trigger shares, comments, or other types of reaction as a sign of community engagement.

We also compared the communication behavior displayed by the analyzed institutions with the principles of risk communication, as described in relevant professional literature, to draw lessons from a crisis that affected all strata of society, i.e., global, continental, national and local.

#### **3. Results and Discussion**

#### *3.1. The COVID-19 Outbreak in Romania*

In the first two months of 2020, Romania was only mildly aware of the danger posed by COVID-19. Information in the media appeared in the external news section, and in February some measures were taken to monitor incoming persons from regions already facing the health crisis. The outbreak of the disease in March brought the lockdown, and the communication space was completely dominated by the topic of the pandemic [32].

At the national level, Romania followed the pattern against which the WHO had warned; avalanches of messages concerning public health issues poured out via all available channels. Following the recommendations of the WHO, on 17 March the Romanian Government launched an online platform, www.stirioficiale.ro [33], to channel trustworthy, quality information to whomever sought information beyond issues discussed in traditional or social media. Nationally, the voices during the lockdown (coinciding with the state of emergency period) were those of top officials in the country, i.e., the President, Klaus Iohannis, the Prime Minister, Ludovic Orban, the Health Minister, Victor Costache/Nelu Tataru, and the Secretary of State at the Ministry of Internal Affairs, head of the Department for Emergency Situations, Raed Arafat. Occasionally, other ministers came to the fore to comment on topics from their area of competence (public order, education, foreign policy, labor, economy, etc.), and health experts were called upon to offer clarification regarding the evolution of the disease and the steering of public life. Because face-to-face activities were not allowed during the lockdown, the population switched to remote labor and TV watching. Media consumption grew exponentially, with the press conference organized by national authorities in the first week of the lockdown being watched by almost 6 million Romanians [34]. National media monitoring showed that Romanians started watching TV more than 6 h per day, being exposed to 4000 h of breaking news and 84,000 h of informative TV spots during the lockdown. As for sources of information, half of the Romanian population actively watched news on TV channels, and 47% gathered news from social media sources [34].

Taking the rules of crisis communication as mandatory [35], the central government took upon itself the task of coordinating and filtering communication. Thus, in the first weeks of the lockdown, only central authorities released information on the crisis, to the great surprise and dissatisfaction of other voices in society. Seventeen civic organizations, many of which dealt with journalism, public communication, and information, signed a manifesto in March 2020 to restore access to regional/local information; they succeeded in their endeavors some weeks later [36]. Local voices also took an active part in informing the public, based on official information. For instance, in Timis county, the AntiCovidTM platform was developed, at the initiative of several non-governmental organizations with a civic vocation, to create a partnership dedicated to supporting the efforts of active humanitarian initiatives/campaigns in the region and increasing the degree of public information on health issues [37].

According to Malikhao [38] (p. 99), the sustainability of health is a process of social mobilization empowered by both stakeholders (some of whom can be health communicators) and health communicators from outside, who have empathy toward the stakeholders,

to achieve two goals: first, to engage the people in the community in upgrading their health and media literacy status so that they can make an informed choice on their body, health and health care; second, to build up community capacity and networking with other communities so that people can solve problems related to community health, achieve social justice in health, prevent diseases, maintain well-being, and cultivate health knowledge, good attitude, ethical values, a cosmopolitan worldview, and health behaviors, including advocating for structural change for a local commitment to healthy lifestyle and an accommodating environment.

#### *3.2. The Public Health System in Romania and Its Reaction to the COVID-19 Pandemic—An Overview*

The pandemic of 2020 put a serious damper on these two goals, testing the capacity of countries to communicate strategically [39] (p. 14) and of the WHO to effectively manage the COVID-19 health risk.

In Romania, public health is defined by Law 95/2006 as "the organized effort of the society towards the protection and promotion of population health" [40]. Public health services include health promotion, disease prevention and improving quality of life and comprise the following activities: immunization; control and surveillance of diseases and risk factors; monitoring population health and health determinants; measuring the efficiency and effectiveness of health care; the assessment of population needs; health promotion and health education campaigns; occupational health; and environmental health, among others. The National Health Strategy 2014–2020 is still in use, and it includes public health as one of the three main priority areas [23] (p. 93). The coordination for the provision of public health services is the responsibility of the Ministry of Health, which is also responsible for the strategic planning and organization of public health services. Other institutions with responsibilities in public health are the National Centre for Environmental Monitoring of Risks in the Community, the National Centre for Communicable Diseases Surveillance and Control (NCCDSC), the National Centre for Methodological Coordination and Information on Occupational Diseases and the National Centre for Health Status Evaluation and Health Promotion. In addition, six regional public health centers, which are located in Bucharest, Cluj, Ias,i, Sibiu, Târgu Mures, and Timis,oara, function as the regional branches of the National Institute of Public Health (NIPH). The regional centers have mainly methodological and technical roles. At the local level, the Ministry of Health is represented by 42 DPHAs. Their responsibilities include [41] monitoring the health of the population and health determinants; identification of public health needs of communities; performing controls of health institutions; coordinating the implementation of national public health programs at the local level; carrying out sanitary inspection and health promotion activities, etc.

European and international reports on the Romanian health system describe it as hyper-centralized, antiquated, and failing to ensure communication between health information system players [42–45].

These features had an impact on the public communication effort during the COVID-19 lockdown. Duplicated information appeared on websites and communication projects developed by each player, most of the time without ensuring interoperability or crossfertilization of the initiatives. Traditional media monitoring showed that the main voices in television were President Klaus Iohannis, Prime Minister Ludovic Orban (who went through a period of self-isolation due to exposure to a risk population), Health Minister Viorel Costache, replaced in the middle of the crisis by Nelu Tataru, and the Head of the Department for Emergency Situations Raed Arafat, the four officials whom the population trusted most during the lockdown [46]. Other ministers or top health experts were also invited to the fore, to offer stewardship and guidance for different aspects of life, from individual behaviors to work, leisure and faith-related activities. At times, these voices did not form a coherent view, a feature identified in other countries as well, despite the warning of experts that "any conflicting information carries a risk of harm, but unfortunately it abounds"; the response to the crisis was addressed mainly at national levels in a variety

of ways. Those who analyzed the situation concluded that "the European approach has not prevailed", but that communication efforts should be consistent and coherent to build public trust as well as to maintain the response capacity of the healthcare system to react: "Where communication is well-structured, staff is more engaged and motivated to work for the sake of better healthcare, and this raises the quality of healthcare as a whole and may improve patient outcomes" [47].

In evaluating Romania's response to the crisis in its early stages, experts consider that the country "has one of the highest levels of poverty, social exclusion, and restricted access to education in the EU, and any public information campaigns would have needed to consider these aspects" [48] (p. 5). In addition, despite demonstrating an initially strong capacity to contain the COVID-19 pandemic, the Romanian healthcare system showed areas in need of improvement, among which was the "best use of the available resources and channels of communication during an ongoing health crisis' [48] (p. 5).

A terminological clarification is appropriate at this point. Although the COVID-19 pandemic is treated and discussed in terms of crisis, the communication related to the effort of limiting the spread of the disease belongs to risk communication. The terms do not overlap entirely and need clarification. The WHO framed the issue in terms of "risk communication" and announced that it was making 24/7 efforts to provide "public health information and advice on the 2019-nCoV, including myth busters, available on its social media channels (including Weibo, Twitter, Facebook, Instagram, LinkedIn, Pinterest) and website" [12] (p. 2). In this paper, social media is defined as "Internet based applications that allow users to create, exchange, or simply consume user-generated content—that is, content created, developed, and shared by individuals" [49] (p. 173). Examples of social media include, but are not limited to, collaborative projects (e.g., Wikipedia), weblogs (e.g., online diaries), microblogs (e.g., Twitter), video/photo sharing (e.g., YouTube, Instagram), social networking sites (e.g., Facebook), virtual game worlds, and virtual social worlds.

Dennis Wilcox specifies the difference between risk communication and crisis communication, with the first category being applied (explicitly) to (public) health outbreaks or other risk-related topics, and the latter being specific to organizations shaken by events that may threaten the very existence of the organization [50]. His strong advice is that, for risk communication, emphasis should be placed on the "dissemination of accurate information. The communicator must begin early, identify and address the public's concerns, recognize the public as a legitimate partner, anticipate hostility, respond to the needs of the news media, and always be honest" [50] (p. 300). For crisis communication, the crisis management plan needs activation, and reparatory measures should be part of the lessons learned from the whole process [50] (p. 300). Other authors [51,52] do not make the distinction using such nuances, including risk communication under the broader umbrella of crisis communication. The lessons Mats Eriksson draws from practice, for instance, put together risk and crisis communication and assess that "despite the powerful digitization of society in recent years, the development of social media, and the fast-growing body of research concerning social media crisis communication, the overall lessons identified here still primarily seem to be about actions like the need for pre-event planning, partnerships with the public, listening to the public's concerns, and understanding the audience's need for credible sources" [51] (p. 541). Ruhert Genc, in a recent paper, highlights the fact that "in any sustainable plan or strategy communication plays a vital role" [53] (p. 511), while Ericsson, based on American experiences and calling for more studies in other geographic regions, states that the "new social media landscape for crisis communications seems to work much like that of the old media society". His recommendation, among others, is to bridge the gap between research and practice and enhance organizations' reaction capacity to crises on the basis of evidence-based lessons and/or advice that would deal with all channels of communication. A much-quoted researcher for crisis communication, Timothy Coombs, sets forth theoretical clarifications for the stages and typology of crises, advising professionals to be present, timely and polite in communicating with the media and the public. He warns that "the Internet is many communication channels, not just one. These

channels include websites, discussion boards, blogs, microblogs, chat rooms, Listservs, image sharing, and social networking sites, to name but a few. Internet communication channels emphasize the interactive and interconnected nature of the Internet" [52] (p. 34). As opposed to private sector services, where these media are already heavily used, "social media are evidently underused by audit institutions as public service providers, both on general and on specific topics" as revealed by a recent study on European institutions [54].

Our study also gathered information about the variety of channels used for communication with the public by the relevant authorities involved in crisis management at a national level. The analysis of social media buttons displayed on their websites revealed a lack of coordination. Our conclusion is backed by the following data:


The online platform, www.stirioficiale.ro, (accessed during our research from March 2020 throughout the lockdown period, up to May 2020) towards which national authorities guide the public for verifiable, correct information, has buttons to help the public sort out fake news, but has no associated Facebook, Twitter or YouTube page. They only invite visitors to subscribe to their newsletter. We find this insufficient, as we consider that a Facebook page with posts conveying official messages would have helped the spread of correct and useful information.

Such a variety of styles and channels fails "to make the communication more sustainable" [54] (p. 2500), does not serve the interconnectedness of Internet resources, and fails to capitalize on existing resources and messages and help keep the public informed and trusting of authorities' capacity to contain the health risk situation. In addition, in the analyzed period, three top government positions had a change of occupant, due to resignation or being removed, mainly for failure of properly communicating the measures undertaken to limit the spread of COVID-19. Thus, the Health Minister, Viorel Costache, lost his portfolio exactly one month after the declaration of the state of emergency. The same happened to Dr. Adrian Streinu-Cercel, who chaired a special committee created at the level of the Health Ministry, though his removal occurred shortly after the state of emergency ended. In the monitored period, six of the directors of DHAs resigned or were removed from their position, all of this resulting in uneven efforts to communicate on COVID-19 measures. Against this background, we carried out our research. The presentation of the national level is necessary for understanding some issues at the district level, where the highly centralized system functions with constant reference to the leadership provided from the top of the authoritative pyramid, as observed also by other researchers who deal with Romanian health system topics [25] (pp. 331–359).

#### *3.3. District Public Health Authorities and Their Communication Behavior during the COVID-19 Lockdown*

At the district level as analyzed in this paper, research indicates that the information posted on the websites of the District Public Health Authorities (DPHAs) was mainly top-down, based on the official press releases provided by the Ministry of Health, the Ministry of Internal Affairs through the Strategic Communication Group, and the National Institute of Public Health (NIPH) through the National Center for Surveillance and Control of Communicable Diseases (NCCDSC).

3.3.1. Correlation between the Number of Health Incidents and Information Subsidies

Our research aimed to determine whether there was a correlation between the number of health incidents and the amount of information communicated on the media section of institutional websites and/or disseminated via social media channels. Figure 1 highlights the fact that no such correlations can be traced, and that in some cases exactly the opposite behavior can be seen. For example, the most problematic districts—Suceava and the national capital, Bucharest—had no Facebook page during the analyzed period and posted only a couple of press releases. Furthermore, the highest number of press releases was published on the webpage of the DPHA for Botosani, which was the least infected at the beginning of the lockdown period. Therefore, our assumption that the efforts to keep the community informed would be enhanced in the most affected regions was partially invalidated. Further findings point to a reluctance to communicate.

**Figure 1.** Reported COVID-19 cases vs. number of information subsidies (press releases and social media posts). Note: FB = Facebook.

It is worth mentioning that the DPHA webpage for Bucharest was very new and had no press releases available prior to 18 May, which was outside the state of emergency period analyzed. Thus, there was no traceability regarding the issued press releases or public announcements made in the timespan between 16 March and 14 May 2020. The fact that the Facebook page of the District Public Health Authority in Bucharest was created on 16 May also does not give tangible evidence regarding the way communication was handled throughout the ongoing crisis. The results, however, pointed us towards further analyzing the communication instances produced by these DPHAs.

The literature review and WHO recommendations highlighted the importance of communication for risk management and the positive impact triggered by the use of various platforms in public communication. Experts point to the fact that "it is getting difficult to be transparent, engaging and satisfying other stakeholders without digital platforms, artificial intelligence, innovative software, mobile applications, and video advertising. Moreover, the society and key strategic partners bit by bit set higher communication standards as well as expectations towards civil service experts and their communication style" [55]. Capitalizing on the above, and given the data available for the study, we took a closer look at the webpages of the 15 DHAs analyzed and aimed to identify the following:


The number and type of press releases is addressed in Figure 2. It shows that, throughout the analyzed period, most press releases issued tackle the problem of COVID-19.

**Figure 2.** Number of press releases on DPHA webpages.

#### 3.3.2. Assessment of Rhythmicity in the Communication Effort of DPHAs

When verifying whether the district authorities sought other means of informing the public, we were interested in finding out if essential information about COVID-19 was visible and accessible in other forms, like dedicated sections on the website or easy-tounderstand infographics.

The webpage of the DPHA in Hunedoara drew our attention. Even if the number of press releases issued in the state-of-emergency months (March, April, May) was small (four), we could not ignore the structure of the homepage, which was well adapted to answer most questions about COVID-19 (self-isolation, quarantine and isolation, protection, myths, shopping, public transport, etc.). Furthermore, the website had in its main menu a button labelled "COVID-19", which gathered all official statements on the pandemic as published by the district authority. Unfortunately, our research could not determine the time at which that information was published on the website and whether it was available to the public before 14 May.

We discuss distinctively the webpage of DPHA Cluj, a multi-ethnic district, which mentions the possibility of accessing the information in English and French (however, the buttons failed to provide information in these languages). We found this example interesting as, on the one side, it showed the vision of the team that created the website, but on the other side, revealed the poor implementation of the project and, we dare say, a limited interest in addressing the needs of that particular community, despite the WHO recommendation to address the communication information needs of minorities (for instance [56], expats and other groups of stakeholders).

Concerning the importance given to social media by the 15 analyzed DPHAs, the research revealed the following:


The results of the analysis are presented in Figure 3 below.

**Figure 3.** Signs of social media channels on institutional websites.

The case of the Brasov, Cluj and Iasi districts was interesting, as the DPHAs in these districts had the three most active Facebook (FB) pages (as can be seen in Figure 4), but there was no FB icon on their website. This raises the question of in-house coordination of communication efforts.

**Figure 4.** Number of COVID-19-related posts during the monitored months.

On the other hand, DPHAs from Constanta and Botosani districts integrated a Facebook button on their websites, even though no such Facebook page existed. Bucharest is the only case in which the Facebook button and the Facebook page existed but were not linked. The website of the DPHA in Bucharest also had an inactive Twitter icon.

Since Facebook proved to be the social media channel of choice, we present the results of the communication effort of DPHAs via this type of page in Figure 4.

We want to highlight the fact that the Facebook page of DPHA Arad was created on 30 April, and throughout the rest of the 16 lockdown days, it showed an intensive communication effort, with more than one post per day. The Timis DPHA displayed a similar pattern, with a Facebook page created 16 days earlier than Arad, on 14 April. As mentioned before, DPHA Bucharest also made the decision to create a Facebook page, but it was operationalized on 16 May (a day after the end of the state of emergency). Therefore, its activity was not relevant for the current research.

The changes regarding the traffic on DPHA websites were analyzed with the help of SEMrush, an online visibility management and content marketing platform [30]. The results showed that nearly all web traffic was organic and generated by search engines (more than 95%), which was not surprising as DPHA websites were not very well known before the pandemic. However, our research revealed an unexpected trend in users' behavior, as in all analyzed cases we found little or no relevant increase in traffic on the websites of District Public Health Authorities between 16 March and 14 May. An increase in traffic could be seen only towards the end of the lockdown period, and it continued to grow, with a peak in June and another in October–November, as can be seen in Figures 5 and 6.

**Figure 5.** Traffic on District Public Health Authority (DPHA) Vrancea website.


**Figure 6.** Traffic on DPHA Bucharest website.

We believe that the results are an indicator of media usage habits, and that Romanians still show a preference for television and centralized official communication. It may also mean that the information needed was available on other websites, which are not subject of this research.

#### 3.3.3. Type of Content Disseminated by DPHAs via Social Media Accounts

Our third research objective was to examine the type of content disseminated by DPHAs via social media, from the point of view of ownership (produced in-house vs. share of content produced by other institutions) and engagement (comments, shares). Figure 7 sums up the results of our analysis concerning the type of content posted on DPHA Facebook pages.

**Figure 7.** Type of content posted.

Additionally, we found a tendency of local authorities to share the data communicated by the Strategic Communication Group and the Ministry of Health, without disseminating local data, in five out of the seven active Facebook pages.

In addition, we highlight the Vrancea DPHA Facebook page, which displayed a communication approach totally different from the rest in the sample. Almost all the posts were adapted for local audiences, and besides the essential press releases, they referred to decisions made by Vrancea Prefecture, promoted local COVID-19 initiatives, presented the way in which lockdown regulations were implemented, etc.

The most active Facebook page analyzed was that of the District Public Health Authority in Iasi, with a total of 152 posts between 16 March–14 May 2020. As in the case of the Vrancea district, the amount of content produced in-house surpassed the shared content. Particularly impressive was the attention given to design, not only to content. For example, they published data about the evolution of COVID-19 cases in the format illustrated by the infographic bellow (Figure 8), or they illustrated relevant collaboration with original photos, proving a professional approach aligned to web 2.0 affordances.

**Figure 8.** Example of use of infographic on Facebook page of DPHA Iasi.

Such an approach to information is rare, even among the national authorities in charge of public health responsibilities. In the example above, the post even includes the hashtag #staiacasa (#stayathome), a rather popular hashtag at the time [57], and other useful information.

Beyond monitoring the number of posts on the DPHA Facebook pages, we analyzed the engagement displayed by the followers. The results are presented in Figure 9.

**Figure 9.** Engagement of Facebook pages' followers.

The behavior of Facebook users was very uneven. In most cases there were no or few reactions to the posts. However, we encountered a few exceptions.

For example, DPHA Brasov held the record for the most shared post. As can be seen in Figure 10, the information posted at the beginning of the lockdown (18 March) was shared 871 times. It contained contact information that people lacked at the beginning of the pandemic.


**Figure 10.** The most shared post from a DPHA Facebook page.

The next most shared post was one from Vrancea's Facebook page (16 April, 135 shares), and it contained information about the fact that testing with an RT-PCR device had begun within DPHA Vrancea.

We would also like to point out that, in the case of DPHA Iasi, almost all posts were shared by a person called "Madalina M.H". In about half of the posts, she was the only one sharing the information. This practice is not unknown and helps boost the impact of the main Facebook page, but it is not an indicator of real engagement. Most probably, once that person changes his/her job, (s)he will no longer share the content posted on that particular Facebook page.

Another indicator of engagement lays with the comments received for the published information. Our research showed that the majority of posts received no reaction, and when comments were made, they were mostly negative, relating to negative experiences regarding communication with or by the District Public Health Authorities.

Again, we identified an exception to the general behavior: the post from 17 April by DPHA Vrancea received a total number of 116 comments and 84 shares. Almost all followers wrote negative comments to the information posted, with some being impolite and even furious ("You really are not ashamed?", "Who should we trust in, when everything is done on political criteria?? All thieves!!", etc.). The post provided details about the way in which the newly acquired testing equipment was proposed to be used, and thus showed that it was really difficult for common people to gain access to the services.

#### **4. Conclusions and Lessons Learned**

We resonate with the conclusions of Woulter Jong that healthcare organizations and institutes need to establish trust and engage actively and positively with stakeholders [58]. Once the crisis is over, the quality of the response and the management need to be discussed [19,58]. Our research is an attempt to offer an evaluation of the healthcare authorities at an intermediate stage, while the crisis is still unfolding.

The COVID-19 pandemic put a stress on all walks of life and on all types of organizations. The Romanian DPHAs, as part of the public sector at an intermediate level, are faced with multiple challenges and expectations. It falls on them to respond in a timely and massive manner to public concerns. However, the assumption on which we based our first research objective was partially invalidated by the analysis. The fact that there is no correlation between the severity of the crisis and the information subsidies could lead us to state that efforts to address the situation were not taken locally. Yet, the fact that two Facebook pages were created in the middle of the lockdown period (Timis, Arad), and one immediately after (Bucharest), suggests that local teams were aware of their responsibility and tried to get closer to the public via social media. It was "on-the-job" training and "learning by doing".

The type of content posted on Facebook is also an indicator of strategic communication with the public. Roughly one third of the content was produced in-house, with only two districts having more in-house content than shared content. An in-depth analysis showed that the "courage" to put a mark on the posts grew with experience, as most FB pages were no older than one year, except for Iasi and Vrancea, which were created in 2017 (31 August and 6 November, respectively).

We can conclude that there was no clear communication strategy valid for all DPHAs and implemented at a national level. In addition, there were also no clear risk management and crisis communication procedures shared among public authorities. The spokespersons of the District Public Health Authorities analyzed were not visible during the crisis, despite handbook recommendations for crisis and/or risk periods. The task of communicating with the public was assumed by often underprepared directors, taking the stand and generating additional communication crises, a situation that in many cases cost them their positions. Romanian public health officials failed to use the situation of "lockdown and standstill as a window of opportunity to change direction and prevent future crises" and thus failed to turn towards sustainable communication strategies [3]. While communication efforts were not negligeable, a certain hesitation and lack of mastering the rules of risk/crisis communication were encountered in the analyzed sample [3,52]. Not only were the content and timeliness of communications important, but also the formats and design.

In accordance with the literature review, social media platforms emphasize the importance of design in rendering the message. Considering the data collected, we assume that older Facebook pages mean better communication skills and exercise for those involved. However, examples of digital skills and organic integration of available tools were rarely identified in the analyzed Facebook pages.

For an emerging infectious disease such as COVID-19, various forces involved in the interactions between the public authority bodies, society, and the news media could lead to strengthening the fabric of society, building unity and solidarity. This study does not examine all the intertwined factors and relationships mentioned above. The study shows, however, that communication efforts were rather chaotic and lacked coherence and coordination, despite apparent efforts to channel the public opinion towards a unified reaction to the rising risk of spreading disease. Despite WHO recommendations [11], the regional level authorities were entrusted to communicate with the public late in the game [36] and when given the chance, they lacked the exercise and the courage to mount an energetic response to the many questions posed by citizens concerning their well-being, recommended courses of action and possibilities to control their own health issues, as seen from the comments to the posts on the Facebook pages under DPHA authority.

COVID-19 had, in the context of this study, at least two major effects. First, it made the international and the local communities aware that the natural world can take everyone by surprise, and that in envisaging a (better) future, a "stop and go" strategy can be the answer to the many challenges posed by the (current) crisis [8]. Second, it tested the reaction capacity of public systems to address major crises on many levels, the communication level included. For the Romanian health system, the initial communicative response was confusing and timid. Public health authorities at the national level dealt with global information, while locally, where incidents of disease spreading were reported, the relevant authorities failed to provide their share of information and take part in agenda building. Post-lockdown, the communication improved, proving that lessons were learned, and that reaction capacity is growing [7].

#### **5. Limitations of the Study**

This study has several potential weaknesses. First, the analyzed institutional websites do not present strategic plans for communication issues related to risk communication in general, and to the 2020 pandemic as a distinct case. Interviewing media officers or spokespersons did not prove to be a successful enterprise; thus, there is a necessity to supplement data with comments from the people in charge of communication, but only when the crisis lessens in power. Second, we analyzed only 15 out of the total number of DPHAs, based on the early-stage situation of the pandemic outbreak. No centralized data are provided on the official websites of central authorities. Even the Health Ministry, which presents the table with the DPHAs, does not allow landing on the respective webpages. In addition, the crisis is ongoing, leaving appreciations on the success or failure of communication strategies to a post-crisis period, when reflection will be welcomed and possible. Further research should also be oriented towards analyzing the communication style of the posts, visibility in the media, changes in communication approaches, etc., thus exploring the behavior of media, the major partner in building the communication agenda for the COVID-19 pandemic.

**Author Contributions:** Conceptualization, M.C.-B. and A.P.; writing—original draft preparation, M.C.-B. and A.P.; writing—review and editing, M.C.-B. and A.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **The Impact of CEOs' Gender on Organisational Efficiency in the Public Sector: Evidence from the English NHS**

**Gema Gutierrez-Romero 1,\*, Antonio Blanco-Oliver 2, Mª Teresa Montero-Romero <sup>1</sup> and Mariano Carbonero-Ruz <sup>1</sup>**


**\*** Correspondence: ggutierrez@uloyola.es; Tel.: +34-607093294

**Abstract:** Increasing operational efficiency is an objective relevant for all institutions, but it is essential in public entities and even more in public health systems because of the number of resources they consume and their impact on general welfare. This research analyses the effect that CEOs' gender has on the operational efficiency of the entities they manage. Despite the impact that the management team and notably the CEO have on the development of institutions, studies on their effect on performance are practically non-existent, especially for public organisations. We have used data from acute care hospital trusts belonging to the English National Health System (NHS) concerning its development. The results were obtained from a two-stage analysis. First, the entities' economic efficiency and health/social efficiency (two operational efficiency measures) were evaluated using two data envelopment analysis (DEA) models. Secondly, the results have been regressed with the CEOs' gender. The results obtained are robust and consistent, revealing that male CEOs have greater performance than female CEOs. This result provides insight into determining features that relate to operational efficiency, which it is of interest to the research and policymakers.

**Keywords:** efficiency; gender; CEO; top management team (TMT); data envelopment analysis (DEA); truncated regression; bootstrap; upper echelon theory

#### **1. Introduction**

The analysis of efficient management in the public sector and the key factors that determine it are of great interest to researchers [1,2], especially in public health systems [3]. The analysis of efficiency in the public health sector takes on even greater importance in the context of COVID-19, which has highlighted the limitations of the health system. While recent studies have analysed the health system's present and future challenges [4], concerns about efficient management in the public sector are not new. They have been the basis for policies known as New Public Management (NPM) [5]. Public health system concern regarding efficiencies is not only to do with the impact on public opinion and the welfare of society but also due to the large volume of economic resources consumed by health systems, which on average in the EU amounts to 9.9% of a nation's GDP [6]. Since the mid-1980s, public system management and practice theories have shifted towards implementing NPM [7], whose central hypothesis is to manage public systems more similarly to the private sector to make government entities more efficient [8]. Within these management policies related to NPM, the literature pays particular attention to the upper echelon theory (UET) [9]. This theory posits that the performance of the senior management team (TMT) is one of the factors that most affects the operational efficiency of an institution, since they are the ones who carry out most of the strategic decisions [10,11]. According to the UET, the management team members' previous experience, values, and personalities influence their decision-making and, therefore, the performance of the entities for which they make such decisions [12] (p. 334).

**Citation:** Gutierrez-Romero, G.; Blanco-Oliver, A.; Montero-Romero, M.T.; Carbonero-Ruz, M. The Impact of CEOs' Gender on Organisational Efficiency in the Public Sector: Evidence from the English NHS. *Sustainability* **2021**, *13*, 2188. https://doi.org/10.3390/su13042188

Academic Editor: Quan-Hoang Vuong Received: 31 December 2020 Accepted: 14 February 2021 Published: 18 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Provided that something improves the TMT's performance, this would also enhance the firm's performance [13]. Traditionally, previous research has focused on the effects of TMTs as a single unit, using the same level for the chief executive officer (CEO) and the rest of the top management. However, several studies have refuted such an approach based on the particularity of the CEO's role compared with the role of the rest of the TMT [14]. Indeed, the CEO is considered to hold a strategic position to convey signals, nonquantitative information, and management styles to the organisations' stakeholders [15]. This aspect is especially relevant in public institutions, where controversial managerial topics, such as women's access to executive positions, are firstly addressed for signalling and subsequently drive private sector behaviour [16]. However, more research is needed to understand the impact of CEOs' attributes on public sector organisations [17]. The need for more research becomes more pertinent than ever in testing the effect of a CEO's gender on the performance of an organisation, since there is no agreement yet on this in the literature [18]. In this vein, the literature is still intensively debating whether higher female representation in TMTs leads to positive [13,19], negative [20,21], or no [22,23] effects on a firm's performance.

Therefore, this research's main objective is to study the effect of the CEO's gender on operational efficiency (note that operational efficiency is a measure of organisational performance and is defined as the capacity to optimise and adapt resources (inputs) to results (outputs); the present research does not measure this) in a case study of public hospitals' cost efficiency in the English National Health Service (NHS). To do so, we carried out a two-stage analysis. In the first stage, data envelopment analysis (DEA) ranked the hospitals according to their technical efficiency score, calculated assuming a constant returns to scale approach. In the second stage, since the efficiency scores were censored at the maximum value of the efficiency scores (1), we ran a panel Tobit and truncated regressions to analyse the effect of CEOs' gender on the efficiency of public hospitals. Additionally, due to the fact that the use of Tobit regression in the second stage caused explanatory variables to be correlated with the error term as inputs and outputs are correlated with explanatory variables, we ran a double bootstrapped procedure (Algorithm II) that permitted making valid inferences while simultaneously generating standard errors and confidence intervals for the efficiency estimates [24].

This study is fully justified, since women remain significantly underrepresented in hospital CEO positions [25]. Indeed, the healthcare sector has been considered a maledominated field for decades, despite the fact that women make up most of the healthcare workforce [26]. This paper's contribution is remarkable, since we document new evidence on the role of CEOs' gender in UK public health institutions' performance. Our findings have great value for researchers, practitioners, and policymakers and provide more clarity and instruments for regulators to design policies to improve efficiency in a critical area such as public health systems.

The rest of the paper proceeds as follows. Section 2 provides an overview of the literature on the relationships between gender and firm performance and develops our hypothesis. Section 3 presents the data and describes the methodology adopted, while Section 4 summarises the main results and discusses the significance of the findings. Lastly, Section 5 concludes the paper and highlights this research's implications, including its limitations, and makes suggestions for future studies.

#### *Review of the Literature and Hypothesis*

Although research has drawn much attention to the underrepresentation of women and minorities in TMTs, the impact of gender on organisational performance is still an open research question [18]. As stated in [27], various individual, organisational, and societal factors explain the lower proportion of women in leadership positions. Regarding the impact of individual factors, including education, expertise, and family responsibilities, on progress into CEO positions, Ref. [28] shows that women fail to progress into CEO positions despite completing graduate degrees in healthcare administration and having

equivalent expertise rates to men. That is, there is a gender driver that damages women's possibilities of labour promotion. Some studies assume that female labour discrimination suggests that female executives need a male mentor figure for career advancement [29]. Indeed, Ref. [30] indicate that it is not enough for women to have mentors; it is necessary for them to have sponsorship from a highly placed executive who advocates for them.

These arguments cause gender biases in favouring men in leadership positions in the healthcare industry [25]. However, there is no conclusive evidence demonstrating that women underperform compared to men in organisational management, whether public nor private. Indeed, the literature shows that women have superior management skills and capabilities in some labour contexts, such as in the green and third sectors [31] or in carrying out marketing tasks [32]. Additionally, Ref. [33] suggest that women often pursue less aggressive strategies and adopt more sustainable investment criteria because women are more risk-averse than men, especially where financial decisions are concerned [34]. Being less overconfident means that financial markets more favourably receive financial transactions made by women, since it is assumed that female CEOs exercise greater scrutiny and exhibit less hubris in strategic decisions [35]. Therefore, these arguments would positively impact the effect of female executives on organisations' economic outcomes.

As mentioned above, there is no clear evidence that women have lower performance in management positions. However, most studies on the effects on women's performance have been conducted in the context of diversity in management teams [21,36,37]. Even in this context, there is no consensus regarding the impact on institutions. Among the studies that discuss the possibility that the gender of managers affects the performance of entities, there are two main lines: *liberal feminist theory* and *social feminist theory* [38]. The liberal feminist theory holds that female-managed entities may perform worse due to systematic factors that limit the scope of relevant resources. In contrast, the social feminist theory holds that women and men are different by their very nature and will cope differently (not necessarily worse) with management, such as taking fewer risks [39]

Following this point of view of social feminism, we found that particularly for hospitals' social or healthcare outcomes, several arguments would support a positive relationship between female leadership and the performance of hospitals, measured in terms of patient well-being. Indeed, Ref. [40] show that the chief executive officer's (CEO) gender may affect patient experience. In this vein, Ref. [41] found that female CEOs improve interpersonal care experience faster than male CEOs, particularly in the most complex executive job environments. It has been noted that women seem to fit better in this particular industry due to healthcare services' relational and interpersonal nature. We can theoretically explain this from the social preference perspective, which suggests that women are more sensitive to social cues in determining appropriate behaviour [42] due to their more compassion [43] and inequality aversion [44].

Consequently, with women's social preferences being more situationally specific than those of men, they will be more likely to show respect for and be willing to help individuals with healthcare needs such as patients. As shown in [45], management styles are influenced by gender differences in relational orientation, since women CEOs are more prone to enact transformational versus transactional leadership behaviours. In summary, the literature supports the notion that female CEOs have a management style that focuses to a greater extent on the social and healthcare view in the decision-making process. Conversely, there is also opposing evidence questioning the ability of women to achieve superior performance in managerial contexts. For example, Ref. [46] point out that female CEOs' preference for more social or people-oriented decisions means that less importance is given to the financial performance of the entities they manage. This greater inclination towards social issues leads them to devote more significant resources to improving corporate social performance, leaving aside organisational performance factors [47]. Several authors also find women to be less ambitious and to exercise less power [37]. The capacity to influence subordinates has arisen as a critical success factor for CEOs, and even more so in a hospital context where highly trained -and hard-to-monitor individuals run separate but interconnected

production processes [38]. This feature would imply that female CEOs may underperform compared to males in a hospital environment.

Similarly, Ref. [48] finds that females' higher risk aversion could lead to the accumulation of suboptimal decisions, which would lead to the lower performance of womenmanaged entities.

In the UET exposition, although [10] does not mention gender specifically, the paper presents distinctive characteristics that can predict how managers will deal with certain situations. Consequently, such features could be used to predict the level of performance of the entities they manage. In the same vein, particularly for the public sector, the Public Sector Management theory (PSM) agrees that several particular characteristics make individuals more likely to work in the public sector [29]. In this sense, feminist theories, both social and liberal, agree with the UET and PSM in the presence of a series of personal and system characteristics which would affect (not necessarily negatively) the performance of the managed entities [39].

Liberal feminist theory, for example, argues that women face systemic constraints that make it difficult for them to access the resources necessary for the better functioning of the entities they manage, which would be an obstacle to their performance compared with entities controlled by men [38]. In the same vein, Ref. [49], points out that in highly competitive markets with resource constraints, the CEO's male power has the most significant positive effect on the institutions they manage.

For instance, we found several authors who base reputation on CEOs' educational background; previous positive experiences in similar positions; and human or social capital, which translates into more significant networks, giving them access to financial and information resources [50]. Such access to resources in a competitive environment will lead to higher performance levels [51]. When we analysed these characteristics from a gender perspective and a liberal feminist perspective concerning the systematic restrains, we could see how the females have more limited access to networks than males [52]. We also found empirical evidence that creating these links or contacts is favoured by similarity between individuals [53]. This concept of "homophily" [54], explains the entry barriers specific individuals, women, and ethnic minorities [55] experience in accessing the resources provided by such social capital, limiting their possibilities to improve their performance. In a recent study on social identity theory [56], the authors found that concerning the favouritism implied by homophily, men protect the "monopoly value generated by their elite status", which limits women's access to resources.

Among the characteristics that provide access to resources is the prestige of directors. The presence of directors on several boards is directly related to these directors' perceived prestige [57]. As presented by [58,59], the male elites will block the presence of women on boards of directors to protect their distinctive effect. This has the effect of blocking access to these boards and therefore the recognition that goes with it.

In the same vein, Ref. [60] explain that females will have limited access to resources since the "old boy network" has the most to lose from women's entry.

On the other hand, and in line with women's difficulties in accessing managerial positions, evidence shows that female CEOs tend to be selected when institutions experience problems [61]. While this could be related to women's social skills that allow them to manage people in delicate situations better [62], there is extensive literature that supports the idea of the "glass cliff" [63].

The glass cliff represents the idea that females will be more likely to jump into riskier positions, such as assuming the role of CEO in a company in a difficult situation, to gain experience that otherwise is inaccessible for females [64].

For all the above reasons, we propose the following hypothesis:

**Hypothesis 1 (H1).** *Female CEOs have a negative impact on hospitals' operational efficiency.*

#### **2. Materials and Methods**

#### *2.1. Data Source*

To test our hypothesis, we used as a case study the public health system of the United Kingdom, belonging to the NHS. The selection of this entities is very appropriate due to several aspects. The UK's NHS was the first universal and free healthcare system. However, since its birth in 1948, it has undergone several reforms and modifications. Of particular relevance for its management is the 1990 reform. The state transferred the provision of services to NHS trusts, which are semi-independent, not-for-profit organisations. The NHS trusts act under state control but on a competitive basis—i.e., the end-user can choose the centre where he/she will be treated. NHS trusts are regulated and have specific and homogeneous reporting obligations, favouring results' reliability. In this same vein, and as the second most noteworthy aspect concerning NHS hospital management, progress has been made in public–private collaboration. The first private finance initiative (PFI) was launched in 1992. This framework has since allowed the contracting of private companies to build and operate NHS facilities through long-term contracts. The final relevant managerial transformation was the creation in 2004 of the first NHS foundation trusts (FT). NHS foundation trusts were created to devolve decision-making from central government to local organisations and communities, enabling them to respond to local people's needs and wishes. A foundation trust (FT) is publicly owned and is accountable to the local population, patients, carers, and staff through a Council of Governors. The Council of Governors is appointed from stakeholder organisations such as Local Councils or is elected by FT members.

We had access to public information for data collection, such as the individual hospital trusts' financial statements published in their annual reports. Additional information was also obtained through the Health and Social Care Information Centre (now called NHS Digital). Additionally, to improve the comparability, we limited the analysis to acute care hospital trusts (known as foundation trusts and ordinary trusts) in England. Our sample includes the entire acute care trust population in England, with 128 acute care trusts in October 2009.

#### *2.2. Key Variables*

As stated previously, this paper tests the impact of CEOs' gender on hospitals' operational efficiency. Consequently, the leading variables of our analysis were (i) efficiency, which acts as the dependent variable, and (ii) the CEO's gender, which is the independent variable. Efficiency scores were obtained using a DEA model, where the input and output factors were selected according to previous research and applying a theoretical argument. It is worth noting that there is no optimal way to select inputs and outputs to perform a DEA [65]. Nevertheless, in the healthcare literature the most typical inputs are related to each hospital's capacity to care for patients, such as the number of beds available, the total number of staff, and the number of doctors [66].

In contrast, outputs are linked to healthcare organisations' singular outcomes, such as survival rates or the number of finished consultant episodes (FCE) [67]. Table 1 shows the most common inputs and outputs used by the healthcare literature to construct efficiency DEA models. As shown in Table 1, there is no general agreement regarding the suitability of the best inputs and outputs. However, there is consensus that the input variables must be related to hospitals' (material and human) resources to serve their patients. Outputs must be aligned to the outcomes that generate a hospital, which are linked to two areas: one is focused on the economic field and other has great emphasis on the healthcare dimension.

Consequently, on the one hand the outputs of the DEA model must be related to the capacity to become a financially sustainable hospital, including the ability to govern the hospital and adapt its performance to the state budget allocations. On the other hand, the output variable must be related to the patients' healthcare quality, which is the hospital's raison d'être. In other words, the outputs used in the efficiency DEA model have to consider that hospitals are hybrid organisations, in the sense of the definition of [68], where two separate variables must be combined: the economic performance and the healthcare outcomes.


**Table 1.** Output Input variables.

Based on the previous arguments, we constructed two separate operational efficiency DEA models. Both DEA models used the same input variables: (i) the number of hospital beds and (ii) the number of staff (medical and administrative personnel). These two inputs have been previously used by the literature [66,67] and are justified because they are the primary (material and human) resources that hospitals use to serve their patients.

However, these two DEA models have different outputs. Firstly, we developed a DEA model that focused more on hospitals' economic efficiency and used a more economyoriented output. This efficiency DEA model (Model 1) employed the days of inpatient care as the output variable. The days of inpatient care variable measures the days that patients stay in the hospital receiving medical care. This variable is clearly related to the healthcare dimension and incorporates economic connotations, since the managers of a hospital can accelerate the discharge of patients to reduce the occupation rate and, thus, healthcare costs. As suggested in [72], releases are argued to be a better output measure than inpatient days because unnecessary inpatient days for a hospital episode might falsely indicate a high efficiency. To solve this problem, we constructed a further efficiency DEA model (Model 2) which emphasised the healthcare dimension of the hospitals by using the average survival rate (i.e., the inverse of the average mortality rate) as an output variable, which been widely used as an output in the literature [73–75].

On the other hand, the independent variable "CEO gender" is a dummy variable that takes a value of 1 when the CEO is a woman and 0 otherwise. According to the literature and surveys focused on healthcare systems, we observed that hospitals are male-dominated organisations in our sample. We find that in our sample, there is a substantial gender difference in the CEO position: male CEOs represent 71.88% of the total.

#### *2.3. Controls*

Several control variables were included in the regression model to separate the impact of the CEO gender from the efficiency of other statistically significant potential effects. Control variables are useful to contextualise the environment where each hospital operates and, at the same time, fit the statistical significance of the regression model better. We controlled for the size of the hospital by using the number of beds. Additionally, we used a

dummy variable, "teaching status", that captured if the hospital, further than healthcare services, had learning areas. The reason behind our use of this control variable is that teaching hospitals usually deal with treatment and interventions that are more complex. Hospitals were also split into a dummy group according to their legal status. The relevance of this is that foundation trusts (FTs) are more autonomous and face more substantial external pressure to demonstrate efficiency. FT hospitals are part of the NHS and treat patients according to the NHS principles of free healthcare. Being a FT means that these institutions are better able to provide and manage its services to meet the needs and priorities of the local community, as the trust is free from central government control.

Furthermore, we used several variables that capture information related to patients that directly affect the hospitals' operational efficiency. These variables were the average age of the patients served, the length of stay, and the number of staff assigned to the hospital. Finally, we included a variable to control the hospital's outsourcing policy, which is also linked to the increase in operational efficiency, since outsourcing is often used to contract external services that are not produced efficiently internally. Therefore, we controlled our model with variables related to the efficiency of each hospital.

#### *2.4. Methodology*

#### 2.4.1. First-Stage DEA Efficiency Estimate

Hospital efficiency scores were estimated using DEA [76]. Unlike parametric efficiency models (such as Stochastic Frontier Analysis), DEA is a non-parametric method that does not impose a specific structure on an efficient frontier shape; this is its main advantage [77]. However, a non-parametric treatment of the efficiency frontier relies on general regularity properties, such as monotonicity, convexity, and homogeneity.

DEA analysis enables assessing a hospital's performance relative to a 'best practice' frontier [76]. DEA ranks, by comparison between peers, hospitals from higher to lower efficiency scores, allowing us to define the optimal situation as a minimisation input or maximisation output problem.

The first version of DEA [78] assumes constant returns to scale (CRS)—i.e., a change in inputs is followed by a change in outputs in the same proportion. We used an inputoriented DEA model with variable returns to scale (VRS) developed by [79]. VRS relaxes the constant returns to scale assumption and allows for the possibility that the hospitals' production technology may exhibit increasing, constant, or decreasing returns to scale.

We used an input-oriented VRS model, since our presumption was that hospital managers have more control over inputs than outputs. Essentially, our model offers an efficiency score for n number of Data Management Units (DMUs) using m outputs and s inputs, as presented below:

$$\theta = \max\_{\mu, y} \frac{\sum\_{r=1}^{s} \mu\_r y\_{ro}}{\sum\_{j=1}^{m} v\_j \chi\_{jo}}.\tag{1}$$

This is subject to:

$$\frac{\sum\_{r=1}^{s} \mu\_r y\_{ri}}{\sum\_{j=1}^{m} v\_j \mathfrak{x}\_{ji}} \le 1, \quad i = 1, 2, \dots, n. \tag{2}$$

$$
\mu\_r > 0, \ v\_j > 0, \text{ } for \text{ all } r, j, \tag{3}
$$

where the *j* DMU consumes inputs to produce outputs, where the weights of the outputs and inputs, respectively, have to be > 0 [49]. The efficiency scores are ranked between 0 and 1, with the value 1 showing the most efficient observations.

#### 2.4.2. Second-Stage Truncated Regression

Following [75], we regressed the CEO gender on the DEA models' efficiency scores. For that, we carried out a Tobit regression with the maximum likelihood estimation method for parameter estimations, since the efficiency scores from the first-stage analysis having a censored structure and ordinary least square regression makes them biased and they provide inconsistent estimations with censored dependent variables [80].

Therefore, we consider the following general Tobit model:

$$y\_i^\* = \beta\_0 + \beta\_1 \text{CEO}\_i + \beta\_i X\_i + u\_{i\prime} \tag{4}$$

$$y\_i = \begin{cases} \ y\_{i'}^\* & \text{if } y\_{i,t}^\* < 1 \\ 1, & \text{otherwise} \end{cases} \qquad i = 1, \ldots, N,\tag{5}$$

where the *i* subscript denotes the cross-sectional dimension. The dependent variable, *yi*, is the efficiency score obtained from the DEA. *CEOi* is the CEO gender, measured using a dummy variable; *Xi* is the vector of each hospital's control variables; *ui* is the error term. As argued previously, the control variables (*Xi*) matrix includes variables related to the efficiency levels of each hospital (dependent variable) and other controls such as the size of the hospital or the provision of the teaching services.

Additionally, to check our analysis results we conducted a truncated regression, an alternative statistical method with which we obtained the same findings.

Finally, to confirm our findings, we implemented a robustness test using the [24] procedure. This is a two-stage DEA analysis where efficiency scores are evaluated and then regressed on potential covariates using a double-bootstrapped truncated regression. From the theoretical point of view, when Tobit regression is applied in the second stage, it provokes statistical inconsistency, since the independent variables correlate with the error term [24]. The [24] procedure allows valid inferences to be made, as well as generating standard errors and confidence intervals for the efficiency estimates.

#### **3. Results**

Table 2 contains the descriptive statistics of the variables collected for the sample of public hospitals.


**Table 2.** Descriptive statistics.

Table 3 shows the results of our analysis. It can be observed in Table 2 that the gender of the CEO matters in terms of firm performance. The fact of finding influence, negative or positive, has relevance for the literature, since some studies doubt the existence of the relation between gender and firm performance (see, e.g., [18]).


**Table 3.** Efficiency score from DEA models.

Note: \* = *p* < 0.10, \*\* = *p* < 0.05, \*\*\* = *p* < 0.01. Clustered robust standard errors in parentheses.

We found that female CEOs have a negative impact on the operational efficiency of hospitals. This result remains unaltered for the two efficiency DEA models developed here. One of these two DEA models has a more economic-oriented output, while the other has a more healthcare-oriented output. These results, therefore, imply that women in CEO positions underperform in terms of economic and healthcare outcomes. The results remain stable in the three statistical models developed in the present study, reinforcing our findings' robustness and contribution. Our findings are in line with the results obtained by recent research conducted for general firms (e.g., [48]).

Theoretically, our findings can be explained from various points of view. Firstly, the literature sustains that female CEO appointments are often linked to organisations facing adverse conditions [81]. The practical implications of this are that hospitals with economic or healthcare problems ask to be managed by women. The limitation of women being promoted to leadership positions is reflected in the low proportion of female CEOs in the healthcare industry, where only 20% of CEOs are women, despite the fact that women make up 75% of the healthcare labour force [82]

Secondly, another potential explanation for the negative impact of female CEOs on hospital operational efficiency can be explained by the alternative leadership styles between women and men. In this sense, previous studies have found that women executives are more prone to adopt transformational leadership styles that emphasise team structures [83]. Under the transformational management style, the group coordinates between individuals because it is considered that the synergies bring advantages that result in an improved working atmosphere, which ultimately improves hospital performance. In other words, female CEOs use a management style that promotes the worker's welfare and, indirectly, the hospital's outcomes. In contrast, male leaders often adopt transactional leadership styles based on competition and hierarchy. Here, the achievement of the organisation's strategic objective is based on efficient structures of governance where the corporate guidelines are directly channelled from the apex to the bottom of the hospital.

Consequently, given that the hospitals are large and complex organisations where decision-making is decentralised, it is more difficult to apply the coordination mechanisms and the dynamism required by the transformational leadership styles proposed by female executives. Therefore, in the healthcare environments, women CEOs likely underperform compared to their male counterparts. Conversely, when workflow management follows the direct hierarchical structures designed by a transactional leadership style, it favours management's concretion and objectiveness, thus improving operational efficiency.

#### **4. Discussion**

Women are under-represented in management positions. As we have seen, in the healthcare sector women represent only 20% of management positions, while representing 75% of the workforce [82]. This study focuses on the existence of an association between the gender of the CEO and the institution's operational performance. While the results are clear and robust, confirming our hypothesis—i.e., higher operational efficiency levels when the CEO is male—the reasons behind these results can be analysed from different points of view. However, the robustness and consistency of the results obtained in an area where studies are scarce and with different results are relevant.

UET and PSM determined a series of personal characteristics or features that could define individuals' decision-making. The results obtained in institutions can be predicted according to their management teams' characteristics. Along the same lines, feminist social theory also finds a series of differential social factors or attributes that would justify different managers performance levels (not necessarily worse) depending on gender. In this sense, different management styles may lead to varying performance levels and would explain the lower operational efficiency resulting from our analysis. For instance, Ref. [84] found that women tend to take a more participatory and democratic style of decisionmaking, which could be beneficial in some sectors, but in complex and large organisations it may slow down decision-making, decreasing operational efficiency. In the same vein, women are more socially flexible than men [34]. This could make women more adaptable to political interference, shaping management decisions with political influences, which could harm the hospital's performance. Another feature typically associated with females is that women emphasise relationships over winning and have more excellent interpersonal skills [85]. These features could lead female directors to pay more attention to patient care and providing higher-quality services than to the performance, resulting in lower operational performance levels.

Another plausible reason for the lower levels of operational efficiency is related to social conditioning, which could be associated with overconfidence levels. Ref. [86] found that men are more predisposed to overconfidence than women. Overconfidence is related to individuals in positions of power [61]. These overconfidence levels, in turn, lead to more complex investments and organisational structures [61], which consequently result in higher levels of performance. In the same vein, Ref. [87] found that lower levels of overconfidence lead to lower indebtedness levels and lower levels of acquisitions that could, in turn, lead to a lower performance level. Additionally, Ref. [33] propose that females are more risk-averse than males, especially when financial decisions are concerned [34]. This could lead to an accumulation of sub-optimal choices that could explain a lower operational efficiency level.

Apart from these social features, some other external systematic factors could lead female CEOs to achieve lower operational performance levels. In line with the thesis of liberal feminists, we present that women could face resource constraints (both economic and in terms of access to information) that would justify possible differences in performance between men and women. We started this discussion by pointing out the substantial female under-representation, particularly in the healthcare sector. This underrepresentation is problematic in itself for various reasons. Firstly, the lower presence of women confers them an out-of-group status. In line with the concept above of homophily, the out-of-the-group status could hinder access to the necessary resources and support for the correct performance of managerial functions, which would justify the lower operational efficiency level.

Another possible reason for women's lower performance may be that, given the lesser presence of women in management positions, they are more likely to accept such positions in companies that are in difficult situations, an affect known as the "glass cliff". This fact is not exclusive to private companies. Ref. [88] found that government entities have a higher incidence of "glass cliff" in the US.

Another possible reason could be the one put forward by [89] that affirmed that gender stereotypes and male discrimination contributed significantly to gender disparities. Such a disparity affects the arrival of female CEOs. Ref. [90] present a negative effect on the performance of entities when a woman succeeds a man as CEO, as it is considered a deviation from the common practice of selecting a male CEO. In this sense, the presence of a woman in the CEO position is not necessarily related to a higher level of diversity, which would limit the benefits of her arrival in the position.

Indeed, the results of [91] show that institutions' gender diversity performance benefits are related to a balanced board of directors, not to "the mere token presence of women"—that is, the presence of a female CEO.

#### **5. Limits and Future Research**

Following the criteria of honesty and transparency that should guide scientific knowledge [92], we now develop this research's limitations and weaknesses. The present study's main limitation is related to the temporal constraint of the data analysed due to the absence of further information on CEOs' gender in other periods. This limitation is relevant when it comes to understanding the results obtained, since a longer time sequence could introduce variables that could be relevant in the explanation, such as experience in the position. For example, this temporal limitation is the basis of the research of [93]. These authors revised the results obtained by [94], who found that female CEOs were paid more than male CEOs. However, when [93] extended the sample and time frame, they found results that differed from the previous study.

Additionally, while the results are robust and undoubtedly measure an association of more efficient institutions when the CEO is male, it is necessary to recognise some additional caveats, such as potential reverse causality—that is, the possibility that underperforming institutions select female CEOs (glass cliff effect) [64]. Finally, this sample is specifically of trusts belonging to the English NHS. We should bear in mind that the results presented correspond to certain variables and regions with specific characteristics. The results obtained may vary when looking at countries with lower levels of development [48]. As indicated in [95], scientific research contributes to potential solutions to social problems, such as the under-representation of women in managerial positions. Such

scientific contribution is essential in developing countries as a measure to reach the levels of developed countries.

In line with the above limitations, we call for the development of future research—i.e., research that goes beyond extending the object to other public systems within the European region is of interest; it might be useful to add a greater temporal dimension; it would be interesting to examine together the impact of the CEO's gender with their years of experience in the position, the financial situation of the entity before the arrival of the female CEO, and the diversity of the board of directors. It would also be relevant to extend the present research to other regions, beyond the countries of the European environment, contributing to the scientific evidence in developing countries. These additions would help us to better understand the causal connection of the results obtained.

#### **6. Conclusions**

Despite the limitations, this study contributes significantly to academic and political debate, generating evidence on female CEOs' roles in England NHS FT institutions' operational efficiency. It provides information that could support the establishment of policies to help overcome the present problems, providing background characteristics that could weigh down women's operational efficiency in management positions. The visibility of such features allows for more accurate plans to solve the gender imbalance in positions of responsibility. For instance, besides the quota system, some actions could better lead to overcoming some of the internal and external factors limiting female CEOs. For example, the lack of sponsorship of women within management teams in hospital systems hampers women's entry into such positions [28]. This situation limits the benefits of a more diverse management team, maintains female under-representation, and relegates women's role as tokens with limited institutional performance effects.

**Author Contributions:** Conceptualization, G.G.-R. and A.B.-O.; methodology, G.G.-R. and A.B.-O.; validation, A.B.-O., G.G.-R., M.T.M.-R., and M.C.-R.; formal analysis, G.G.-R. and A.B.-O.; investigation, G.G.-R. and A.B.-O.; resources, A.B.-O. and G.G.-R.; writing—original draft preparation, G.G.-R. and A.B.-O.; writing—review and editing, G.G.-R., A.B.-O., M.T.M.-R., and M.C.-R.; supervision, A.B.-O., M.C.-R., M.T.M.-R. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Understanding the Spatial-Related Abstraction of Public Health Impact Goals and Measures: Illustrated by the Example of the Austrian Action Plan on Women's Health**

**Tatjana Fischer**

Institute of Spatial Planning, Environmental Planning and Land Rearrangement, University of Natural Resources and Life Sciences Vienna, Peter-Jordan-Straße 82, 1190 Vienna, Austria; tatjana.fischer@boku.ac.at

**Abstract:** The influence of spatial aspects on people's health is internationally proven by a wealth of empirical findings. Nevertheless, questions concerning public health still tend to be negotiated among social and health scientists. This was different in the elaboration of the Austrian Action Plan on Women's Health (AAPWH). On the example of the target group of older women, it is shown whether and to what extent the inclusion of the spatial planning perspective in the discussion of impact goals and measures is reflected in the respective inter-ministerial policy paper. The retrospective analysis on the basis of a document analysis of the AAPWH and qualitative interviews with public health experts who were also invited to join, or rather were part of, the expert group, brings to light the following key reasons for the high degree of spatial-related abstraction of the content of this strategic health policy paper: the requirement for general formulations, the lack of public and political awareness for the different living situations in different spatial archetypes, and the lack of external perception of spatial planning as a key discipline with regard to the creation of equivalent living conditions. Nonetheless, this research has promoted the external perception of spatial planning as a relevant discipline in public health issues in Austria. Furthermore, first thematic starting points for an in-depth interdisciplinary dialogue were identified.

**Keywords:** space–health nexus; older women; spatial planning perspective; interdisciplinary expert dialogue; retrospective qualitative study; knowledge transfer; health policy analysis

#### **1. Introduction**

Women's health is in the focus of global interest [1,2] and therefore is a central concern of the WHO [3]. In line with the health-in-all-policies approach of the WHO, health should be implemented in all policies and become a focal subject of political action [4] in order to achieve the following UN Sustainability Goals [5]: SDG 3 "Good Health and Well-being" (in particular sub-target 3.8) [5] (p. 71), SDG 11 "Sustainable Cities and Communities" (in particular sub-targets 11.3, 11.7 and 11.a) [5] (p. 73) and SDG 17 "Partnerships for the Goals" (in particular sub-targets 17.14 and 17.17) [5] (p. 76).

In Austria (women's) health is already an important public responsibility [6,7]. This becomes evident when it comes to the international comparison of the availability and quality of the supply structures of health care facilities [8,9], the life expectancy of women (at old age) [10] and self-rated state of health [11]. Nevertheless, in Austria regional differences in the provision of ambulant social and care services and the spatial distribution of in-patient health and care facilities exist [12]. Other relevant issues are the increase in the absolute and relative proportion of older women in the population and a growing heterogeneity of women relating to educational level, fertility behavior and economic status related to life phases [13,14].

The Austrian Action Plan on Women's Health (AAPWH) [15] starts exactly here and defines general and target group-specific impact goals and measures in order to

**Citation:** Fischer, T. Understanding the Spatial-Related Abstraction of Public Health Impact Goals and Measures: Illustrated by the Example of the Austrian Action Plan on Women's Health. *Sustainability* **2021**, *13*, 773. https://doi.org/10.3390/ su13020773

Received: 17 November 2020 Accepted: 12 January 2021 Published: 14 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

better satisfy the various needs and demands of women for the promotion of physical and mental health, taking into account the current position in the life cycle. Therefore, the AAPWH refers to the following three target groups: (1) "Girls and Young Women", referring to females going through puberty, or rather females aged 12 to 16 years [15] (p. 42), (2) "Women of Working Age", referring to women up to 60 years of age [15] (p. 53) and (3) "Older Women", referring to women aged 60 and older, or rather to women of retirement age [15] (p. 69).

#### *1.1. AAPWH in Brief*

The Austrian Action Plan on Women's Health (AAPWH) as an inter-ministerial strategic policy paper of the Federal Ministry of Social Affairs, Health and Consumer Protection and the Federal Ministry of Women, Families and Youth, Federal Chancellery of the Republic of Austria is unique in Europe [16].

It comprises ninety-eight pages and defines seventeen impact goals and forty measures and aims in order to achieve equity in the quality of life of women in Austria, which is fully in line with the European Health Goals [17] and in accordance with the principle of "leaving no one behind" [18].

Following the Health 2020 policy framework [19], the AAPWH was elaborated in an interdisciplinary and inter-ministerial development process consisting of sixty invited experts from different disciplines and an additional online-consultation process [15].

The AAPWH is published in German and is available online [20].

#### *1.2. The Leading Role of Public Health and the Inclusion of the Author in the Circle of Public Health Experts*

At the time the AAPWH was prepared, in Austria public health and women's health mainly were discussed among (public) health, nursing and social care experts, although health is a central subject of spatial planning due to the close interrelations between health and spatial aspects [21–23]. Moreover, spatial planning is perceived as a key scientific and policy sector-relevant discipline for public health [24]—particularly in the context of healthy cities [25].

The author of this article was therefore pleased to be invited to join the expert group on "Women in Old Age" on the recommendation of an expert who played a key role in the elaboration of the AAPHW.

With the participation in the expert group which dealt with public health impact goals and measures for older women, the author pursued two purposes: (1) raising the awareness of the working group members of the necessity of to taking into account the spatial dimension in the definition of impact goals and measures in the short run and (2) involving spatial planning as a cross-cutting, system- and action-oriented key professional discipline in the discussion on demand and supply planning in the long run by addressing:


#### **2. The Purpose of the Paper**

This article discusses whether and to what extent the interdisciplinary discussion of health policy impact goals and measures, including the expertise of spatial planning, generates merit for evidence-informed health policy [26] with a focus on the target group of older women.

In the following paper this is illustrated by the example of the AAPWH. In this context, this Austrian pilot study addresses the following aspects:


Thus, this research not only fills a knowledge gap in Austria, but also complements the findings of (recent) thematically related studies from other European countries, namely the Netherlands [27] and the United Kingdom [28], which for their part discuss the need for and the merit of inter-sectoral collaboration between the public health and spatial planning sectors for the purpose of alleviating health inequalities on the basis of selected health-related (national) policy papers. In comparison to the study from Austria presented in this article, the above-mentioned studies neither focus on one specific target group, nor do they discuss the degree of spatial-relatedness of the formulated impact goals and measures in more detail.

#### **3. The Space–Health Nexus and the Relevance of Spatial Planning**

Space and health are interlinked in manifold ways. Amongst others, the availability and quality of affordable housing, the level of infrastructural provision for daily supply as well as for social, medical and nursing care, being embedded in a stable social surrounding and neighborhood, having access to a safe public space and the availability of accessible green and open spaces determine the well-being and quality of life, particularly of older people. This applies in particular to those who suffer from health restrictions, or rather dementia [22,29–32]. Against the background of demographic and climate change, particular importance must be attached to all of these aspects [33,34].

In this context, the particularities of different spatial archetypes (e.g., cities, small towns, remote rural areas) with regard to supply structures and the degree of supply with infrastructure, as well as public and open (green) spaces, but also with regard to the availability and structure of social networks, must not be disregarded. For example, the infrastructural supply in larger cities compared to rural, sparsely populated areas is more diverse and characterized by short trips, whereas rural areas tend to be better equipped with open or rather green spaces within walking distance [12,22]. With regard to infrastructure supply and accessibility, ageing in the rural periphery, or rather in dispersed (alpine) settlement structures, is particularly challenging [35].

Due to the continuing polarization in structurally strong and weak regions, the different supply structures with hospitals [36], the trend towards the retention of doctors in rural areas [37], changes of family and household structures as well as quality of ageing and being old differ not only between urban centers and rural peripheries, but also within a single municipality in alpine areas and dispersed settlement structures due to the lack of a comprehensive, adequate public transport system and the spatial locations of infrastructure and one's own place of residence [12].

Therefore, following the principle of health-in-all policies, the main task of spatial planning is to ensure an appropriate land use in order to provide people with green spaces, building land and traffic areas and to balance competing interests and needs, in order to provide livable settlement structures for people at all stages of life [22].

In German-speaking countries spatial planning as a cross-cutting subject and in its policy advisory function in connection with health issues in general and with regard to older people in particular takes on the following important tasks:


3. The development of concepts and strategies that serve to create equal opportunities in access to infrastructure at the regional level [39].

Nonetheless, in Austria spatial planning is still is not involved in the strategic development of social care and health provision planning [12].

#### **4. The Expert Group on "Women in Old Age"**

The following initial situation formed the starting point for the expert group's discussions on impact goals and measures in the areas of health protection and health promotion of older women: their economic disadvantage compared to men of the same age; the special challenges in connection with role attributions and expectations with regard to the assumption of care for older people; and ageism [15].

In total, 35 national experts from various public health related professions and professional positions participated in this expert group in order to deal with the subject of women's health against the backdrop of various professional contexts. Four out of the 35 experts were men.

The participants held, or rather still hold, positions in administration (federal ministries, divisions for gender equality in the offices of federal governments), science and practice (amongst others, women's health care facilities, professional agencies, social insurance institutions, interest groups). An overview of the members of the expert group is provided in AAPHW [15], pages 85 and 86. Depending on their professional skills, some of the experts also joined the two working groups "Girls and Young Women" and "Women of Working Age".

Between April and November 2015, the expert group on "Women in Old Age" elaborated four target group-specific impact goals and ten measures and considered cross-target group-related impact goals and measures, taking into account already existing initiatives, projects and actors' landscapes in Austria. The number of impact goals and measures was predefined.

During this period the working group met three times for full-day workshops in Vienna (cf. Table 1).


**Table 1.** Overview of the agendas and working methods of the workshops of the expert group on "Women in Old Age".

<sup>1</sup> Sources: Minutes of Workshops No. 1 and No. 2 [40,41], <sup>2</sup> Source: Notes by the author of this paper. There are no minutes for Workshop No. 3.

The workshops were moderated and documented by the Austrian National Public Health Institute and the minutes were sent out to the expert group members by e-mail. Additionally, the experts were asked to seize the timespan between the workshops in order to prepare for the following session by completing defined work tasks.

#### **5. Materials and Methods**

#### *5.1. Research Design*

This is a qualitative, retrospective and descriptive research applying a mixed-methods approach which consists of (1) a theme-centered document analysis of the AAPWH focusing on the above-mentioned aspects and (2) semi-structured expert interviews with members of the expert group on "Women in Old Age" (cf. Figure 1) in order to avoid any misinterpretation from the author's reflection of the handwritten notes of the three workshops.

**Figure 1.** Research design and methodological steps.

As this research does not contain any experimental studies on human beings, no approval from an ethics committee was required. The research was carried out in compliance with the General Data Protection Regulation of the European Union.

#### *5.2. Procedure*

5.2.1. Development of an Analysis Grid for the Document Analysis of the AAPWH

As shown in Section 3, the interrelations between space and health (in old age) are complex and the (further) development of health protection and health promotion measures depends on the existing infrastructural level, the authorities and their legal competencies in Austria the public health sector is characterized by federalism [42]—the financial margins of action in both the public and private sectors, and the heterogeneity of the (potential) demanders (among women in old age).

Basing on the author's research findings on the spatial-related challenges of the everyday life in old age as well as of caring for the elderly for various Austrian spatial contexts, the document analysis of the AAPWH addressed the following aspects:


#### 5.2.2. Document Analysis of the AAPWH

The analysis of the spatial-related aspects of the impact goals and measures addressing the target group "Women in Old Age" was carried out by means of a theme-centered document analysis according to Boyatzis [46] without the use of specific software. The procedure was as follows:


#### 5.2.3. Expert Judgements

The idea was to reflect on the spatial-related contents of the AAPWH and the discussion on the relevance of spatial-related aspects of well-being and health in older women in the context of the three workshops of the expert group on "Women in Old Age". For that purpose, semi-structured in-depth expert interviews were conducted with those members of the respective expert group who joined at least two out of the three workshops.

#### Aim of Expert Survey and Design of Questionnaire

The aim of the survey was to identify the key arguments of the experts: (1) regarding the reasons for the degree of spatial abstraction of the AAPHW's impact goals and measures for the target group of "Women in Old Age"; (2) to grasp the interviewed experts' perceptions of spatial planning experts (both, scientists and practitioners) as dialogue partners

in the context of public health issues, and; (3) to explore the limitations of implementing profession-specific knowledge and spatial-related empirical evidence in strategic policy papers.

Based on the conceptual framework of the spatial relatedness of (women's) health in old age, the findings from the qualitative content analysis of the AAPWH, the workshop minutes as well as on the author's own handwritten notes, a catalogue of guiding questions consisting of 16 open questions was developed (cf. Table 2).


**Table 2.** Questionnaire for the expert interviews.

Sampling and Data Collection

As mentioned above, this research intended to gather the opinion of those experts who joined in with the whole discussion process on impact goals and measures for women in old age. The selection criterion was defined as presence in at least two out of the three workshops. The experts were identified by comparing the attendance lists of the available workshop minutes. Twelve experts met the selection criteria. In January 2018 the experts were contacted by e-mail. Six out of twelve could be recruited for an interview.

The questionnaire was sent out to all experts by e-mail a few days prior to the interview in order to give them the opportunity to prepare for it.

The interviews took place between January and March 2018. In order to meet the preferences of the interviewees, the interviews took place either at their workplaces, by telephone or in external locations such as cafés. The interviews lasted at least 60 min and were recorded on tape.

Five out of the six interviewees gave their oral consent to use the findings for a poster presentation at the biannual conference of the Austrian Society for Geriatrics and Gerontology in March 2018 in Austria [47] and for international publications on occasion. After the conference, the poster was sent out to the five interviewees as a PDF-file by e-mail.

Due to the fact that saturation had not yet been reached, the author decided to increase the sample size. Finally, two more national public health experts were interviewed in May 2018. One of them was invited to the expert group on "Women in Old Age", too, but did not attend the workshops; the other expert held and still holds a relevant position in the AAPHW development and implementation process. For these two interviews the same catalogue of guiding questions was applied as for the first wave of interviews—except for the questions Q 1, Q 12 and Q 13. That is why the interviews lasted about 30 min each.

The author of this paper pursued the strategy to recruit as many members of the expert group on "Women in Old Age" as possible for interviews. However, this was challenging due to the fact that some of the experts were on maternity leave or had since retired.

Finally, seven expert judgements were available for further analysis. Table 3 gives an overview of the profiles of all interviewed experts.


**Table 3.** Profile of the interviewed experts.

Due to the small sample size, the respective function of the interviewees in the elaboration process of the Austrian Action Plan on Women's Health (AAPHW) as well as the exact designation of their place of work are not listed for data protection reasons. <sup>1</sup> This interviewee withdrew her consent to publish the results of the interview without giving any reasons.

#### Data Extraction

The tape recordings of the seven expert interviews were listened to several times and transcribed mutatis mutandis. Particularly pithy statements were transcribed verbatim in order to use them as direct quotations in this article.

The transcripts, which consist of four to six handwritten manuscript pages each, were analyzed according to the method of continuous comparison by Glaser and Strauss [48].

#### **6. Results**

#### *6.1. Findings from the Document Analysis*

It should be noted that the AAPWH argues that discrimination against women in terms of access to services regardless of age is associated with income, migration background and social disintegration, and not with where women live, work and care. Nationwide equal opportunities of access to infrastructure and other services is requested.

The content analysis shows that spatial aspects serve to describe the background of the challenges or disadvantages faced by older women. For example, with regard to the issue of violence prevention (Measure 5), the reference to public space is directly addressed. The term "violence in the public social space" is used here [15] (p. 33).

With regard to those impact goals and measures which specifically address the target group of older women, spatial-related aspects found their way in the text as follows:


Impact Goal 17 "*Developing a differentiated, appreciative picture of the diverse realities of older women's lives and secure older women's opportunities for participation in society" [15] (p. 79)* identifies spatial-related challenges of organizing and coping with everyday life are referring to the need to "*design a public and safe space*" as a structural prerequisite for the "*social participation of older women*" [15] (p. 79). This is explained in more detail by "*Measure 38 Improving the living situation and ensuring the participation opportunities of older women in the long term*" [15] (p. 80f): "*Scientifically sound findings on the life situation of older women enable the targeted promotion of diverse, high-quality projects, initiatives and events close to home to ensure the participation opportunities of older women and to improve their life situation. The target group of this measure are in particular older women who are living alone, women with health restrictions, structurally caused mobility impairments and women with low income. Since there are differences between urban and rural areas with regard to older women's opportunities for participation, all activities must take into account spatially specific aspects*" [15] (p. 80f).

This measure expresses the awareness of spatial-related challenges experienced by older women with reduced mobility and limited participation opportunities and points out the importance of local daily supply structures and supportive initiatives.

The terms "urban", "rural" and "space-specific" are used in the text for the purpose of pointing out spatial differences.

Impact Goal 14 "*Ensuring gender-appropriate, individualized medical, psychosocial and nursing care up to old age, regardless of the [spatial] setting*" [15] (p. 71) addresses the need for nationwide and accessible infrastructures. This is specified by "*Measure 32 Establishing regional platforms for women's health*" [15] (p. 72), which is envisaged as a short-term measure and which refers to a specific spatial reference of action.

The regional level as an appropriate spatial reference of action is also addressed in Impact Goal 16 "*Women at old age and at risk of poverty are given conditions that enable them to maintain their capabilities for self-help and to live self-determined and autonomous lives*" [15] (p. 78). "*Measure 37 Establishing a one-stop shop" for the application for and processing of social benefits and for care counselling*" [15] (p. 79), which is assigned to this impact goal, may—among other measures—facilitate access to counselling and information services for women at risk of poverty in structurally weak rural areas—especially if the so-called "one-stop shops" are established regionally.

However, the term "region" is not further specified with regard to its catchment area or accessibility for older people who are not, or rather who are no longer capable of, driving a car.

On the other hand, the following objectives of the author of this paper are not directly addressed in the AAPHW:


#### *6.2. Findings from the Expert Interviews*

According to the interviewed experts, the empirical findings presented in this section can be understood as the outcome of many years of professional experience, being up to date with the latest scientific knowledge (in some cases) as well as personal experiences and stories from the experts' private lives.

#### 6.2.1. The Meaning of Space and Place for the Health of Older Women and the Urban–Rural Mindset

The interviewees conceptualized the terms "space" and "place" in different ways, addressing the material, or rather physical and social, characteristics of place, which in their view are relevant to the health of older women. Thus, they addressed selected features that are also assessed as important by geographical gerontologists [49]. From the point of view of the interviewed experts, "place" is a construct of natural surroundings, human intervention and political categorization. The significance for (older) women's health shows in several ways (cf. Figure 2):



**Figure 2.** Public health experts' conceptualization of "space" and "place".

The interviewees thought in distinct spatial categories: "urban" and "rural" areas. Above all, they associated urban areas with high density and diversity of infrastructure, and rural areas with the exact opposite; low density and challenges with regard to the supply of goods and services in daily demand, and the provision of socio-medical services as well as the resulting restrictions in freedom of choice and accessibility of services and facilities. 6.2.2. Perceived Spatial-Related Challenges for Maintaining and Improving Quality of Life and Health of Women in Old Age in Urban and Rural Areas and the Tendency of Marginalization of Women Living in Rural Areas

From the interviewees' perspective, the quality of the built environment and of nature have a significant impact on health (see also [50]). In their opinion, it is essential to point out that the quality of housing and the living environment as well as the characteristics of the social networks of older women living in urban and rural areas differ fundamentally, which in turn results in differences in health care provision and quality of life for women in old age.

According to the interviewees, infrastructural deficiencies, long distances and a lack of public transport, eroding social networks and structural barriers in the built and residential environment coupled with health restrictions lead to challenges for living an independent life and participating in social life for women of all ages, regardless of the location of the place of residence. In the opinion of one interviewee, "*security in and of public spaces*" (I 3) is a key issue for older women living in cities.

The marginalization of especially very old women without a driving license in rural areas is the result of a thinned-out infrastructure supply, the lack of adequate cultural offerings, and poor public transport outside the main centers and villages. The changes of family structures and social environments imply a decline in informal support and increased loneliness. One interviewee commented as follows: "*In my opinion the interrelations of rurality, quality of life and women's health are more negative than positive—in the sense of a double devaluation. First, women care for men. The women are left behind. Second, women have no voice in society. Therefore, it is not so important to take care of them*." (I 1)

Furthermore, related to the need of action, rural areas are perceived as subordinated to urban areas. According to one interviewee, the reasons for that are also rooted in misunderstandings, for example with regard to the quality of social cohesion: "*When they think of rural areas people are of the opinion that everything is still all right, and that is why many women are left behind*" (I 1).

One interviewee stated that the rhetoric of a healthier old age and growing old (as a woman) in the countryside compared to the city is also the result of a non-reflected use of clichés and therefore needs to be readjusted: "*You need to have to look at this in a more differentiated manner. In rural areas the availability of health care providers such as doctors, nurses, informal caregivers and various professional groups is more problematic*" (I 2).

At the same time, the same interviewee points out that the urban–rural dichotomy is insufficient in order to appropriately describe infrastructure-related differences in daily supply with goods and services. She points to the need for a spatially more differentiated discussion: "*Here [in rural areas] there are certain differences. The more peripheral the less available. It's as simple as that*" (I 2).

Moreover, it is difficult to derive valid spatial-related impacts on the entire collective of older women with regards to spatial health assessment and quality of life. The latter is why quality of life is something very individual and also depends on the respective demands and expectations. In this context some interviewees relate the questions concerning quality of life more to their own person than to the target group of women in old age.

6.2.3. The Significance of Spatial Planning for Public Health and the Merit of Integrating Spatial and Planning Sciences Scholars in the Debate on Older Women's Health

Only one out of the eight interviewees was already professionally in contact with spatial planning experts prior to the workshops of the expert group on "Women in Old Age".

The working context referred to various projects of designing public spaces and accessibility, as well as of fall and accident prevention. The author of this article was already well known to two of the interviewees.

The significance of the professional discipline "spatial planning" is perceived as strategic and project-oriented as well as object-related and is often associated with the terms "housing" and "public space".

Some of the experts assign strategically important competencies to spatial planning by saying that spatial planning:


Good spatial planning, in the eyes of the experts interviewed, is therefore: "*That one considers how an environment has to be designed so that people can cope with everyday life to a reasonable extent, regardless of their level of education and financial resources*." (I 2) This also includes providing opportunities "*in order to enable unplanned communication*" (ibid.).

Thus, those interviewees who took part in the workshops found the interdisciplinary dialogue with a spatial planning scholar enriching on the one hand and challenging, exciting and promising on the other hand.

Nonetheless, the following should always be borne in mind: "*Each expert speaks for him- or herself or rather for one's own profession*" (I 5).

The discussion of spatial references was also complicated by the fuzzy use of terms. Although the interdisciplinary discussion sharpened the subjective perception of space and place and helped to create (more) awareness of the importance of spatial planning as a focal subject, the experts do not remember any specific spatial planning-related arguments, or rather key messages.

6.2.4. Explanations for the Degree of Spatial-Related Abstraction of the Impact Goals and Measures and Its Determining Factors

The experts' judgements of the extent to which the integration of spatial planning expertise during the workshops influenced the way in which spatial references were taken into account in the impact goals and measures, and how the quality of their spatial relevance should ultimately be assessed, are controversial. Some of the interviewees were not surprised by the high level of spatial abstraction, as they know this from other contexts of work, and say: "*The spatial reference is not considered in the action plan, which is a pity.*" (I 4)

According to another expert, spatial references serve at best as justification for the impact goals. Another expert believes that the author could be pleased that the terms "*space and region are addressed directly in the Action Plan*" (I 7).

Addressing the high level of provision and quality of both social and health care facilities in international comparison, an interviewee comments as follows: "*In Austria we discuss [health] at a relatively high level. Nevertheless, there is still so much to be done.*" (I 2)

Moreover, there is an agreement on the need to pay more attention to spatial-related aspects in the context of the health of older women. However, in many cases, these aspects have been neglected in public affairs and politics:


#### 6.2.5. Recommendations for Spatial Planning Scholars

Established lobbies and power structures determine the implementation of inter- and transdisciplinary knowledge. Cross-cutting issues are nodded off, so they fall out of the prioritization phase. This could be alleviated by problem-centered evidence and ideas and strategies that address various spatial levels of action. An expert puts it as follows: "*You should approach from two directions: case study based from below; in the general view from above*" (I 6).

Furthermore, modifications are needed in the handling of knowledge production and knowledge transfer. There is disagreement among the interviewed public health experts on the merit of public consultation processes. The spectrum of opinions ranges from "*each consultation is useful*" (I 2) to thinking that consultation processes are a pure formality.

One expert recommends the following: a clear positioning and identification of content-related interfaces with other disciplines as well as investments in networking, which will help to increase the perception of spatial planning as a cross-cutting discipline. Furthermore, it must always be kept in mind that spatial conditions and critical events both determine the pressure for political action. Therefore, spatial planning can contribute to creating appropriate framework conditions for healthy ageing. That is why further in-depth discussions with spatial planning scholars are welcome.

#### **7. Discussion**

#### *7.1. Methodological Strengths, Challenges and Limitations*

This qualitative research is characterized by the application of mixed-methods and a retrospective multi-perspective reflection on the opportunities and limits of anchoring spatial aspects of women's health using the example of the making of a specific strategic policy paper. This research design can therefore be assigned to action research [51].

The crucial methodological challenge with regard to the realization of the research design was the development of a conceptual evidence-based framework [52] in order to be able to capture the spatial-relatedness of the health of older women with special regard to the Austrian situation, which was to serve as a basis for the analysis of the AAPWH as well as for the expert survey. At that time, in German-speaking countries the spatial planning scientific debate on its contributions to public health has just begun. In early summer 2017, in Potsdam (Germany) the first relevant congress entitled "Anchoring Health in Spatial Planning" was organized by the Academy for Spatial Development in the Leibniz Association.

That is why the author of this article decided to base the conceptual framework on her own empirical findings. The suitability of the conceptual framing was proven during the in-depth discussions with the experts.

After having completed the eight interviewees it became apparent that despite the small sample size, content saturation has occurred. This was due to the comprehensiveness of the described space–health nexus as well as to the explanations relating to the degree of spatial abstraction of the impact goals and measures. The latter is probably also related to the fact that the interviewed experts are outstanding professionals employed in the Austrian public health scene and have various (academic) educational backgrounds. Moreover, some of them also held and still hold leading positions in the making of the AAPWH and its implementation.

Related to the analysis of the material—namely, the qualitative content analysis of the AAPWH and the transcripts of the expert interviews—it can be critically noted that the content analysis was carried out exclusively by the author of this article. Thus, this article does not claim to speak for the whole spatial planning community in Austria, but intends to fuel the discussion among spatial planning theorists and practitioners on the reasons for the lack of involvement in defining gender-related health policy impact goals and measures.

#### *7.2. Considerations on the Validity of the Questionnaire and Reliability of the Findings from the Expert Survey*

Validity is a much-discussed topic in qualitative social research, especially with regard to the question 1. of whether the information obtained in this way is right or wrong, 2. what significance can be assigned to the findings and 3. how they can be put into the larger, or rather international, context [53].

Regarding the validity of the questionnaire applied for this research, it should be noted that it was suitable for capturing the complexity of the topic. This was proven by (1) the fluency of the interviews, (2) the ability of the interviewees to put themselves back to the year 2015 very quickly, (3) staying close to the topic during the entire interview and (4) the lack of critical comments on the methodological approach chosen to reflect the making of the AAPWH and its results, as well as on the guiding questions for the interviews.

With regard to the reliability of the results of the expert survey, it should be noted that (1) the statements are in line with the empirical evidence of spatial science research in Austria on the spatial-relatedness of health in old age and (2) the explanations of the degree of the spatial-related abstraction of the impact objectives and measures of the AAPWH are logical and conclusive. The experts' pragmatic attitude towards the predefined number of impact objectives and measures can be explained by their function in the AAPWH preparation process and is therefore perfectly understandable.

Looking at the applied research approach and the reliability of the expert judgements, it must be critically noted that more than two years passed between the third workshop of the expert group on "Women in Old Age" and the first expert interview. Whether and to what extent this time span had an effect on the quality of the content and the level of detail of the retrospective assessments cannot be assessed ex post. On the contrary, it should be emphasized that the willingness of the experts to reflect on the AAPHW in greater depth can be interpreted as a sincere interest in a cross-disciplinary discussion of health issues, including the spatial planning perspective.

#### *7.3. The Merit of the Interdisciplinary Discussion within the Expert Group on "Women in Old Age" Including the Spatial Planning Perspective*

The cross-disciplinary reflection on the impact goals and measures defined in the AAPWH has stimulated public health experts: (1) to take a different look at issues of (older) women's health; (2) to reflect on the principles of informed political decision making and the feasibility of taking into account the spatial-related complexity of challenges and problems, taking into account a predefined number of impact goals and measures; and (3) to become aware of the similarities and differences of the objectives and differences in the approaches of public health and spatial planning.

7.3.1. Identifying Health-Relevant Spatial Aspects and Dealing with Spatial-Relatedness of (Older) Women's Health: Similarities of and Differences between the Two Professions

Public health experts assign great importance to spatial aspects for the health protection and health promotion of women in different stages of life, or rather life situations above all the accessibility as well as the availability and quality of health care and nursing facilities, as well as counselling services for those seeking information and advice.

It is interesting to note that the focus here is on the provision of social and healthrelated infrastructure facilities, and that the experts pay little attention to the importance of green spaces for maintaining health. With regard to the AAPWH this may be due to the fact that green space planning is not within the competence of the ministries responsible for the AAPWH. A follow-up of the cross-disciplinary dialogue on the importance of green infrastructure including the spatial planning perspective might perhaps lead to the involvement of other ministries in the AAPWH in the long run.

It was shown that public health experts ascribe a great importance to spatial planning with regards to health protection and health promotion (cf. Figure 2). Therefore, it is surprising that the interviewees have had little professional contact with other representatives of the discipline of spatial planning, or rather have not actively sought contact with them. Intensive cooperation between public health experts and spatial planning experts would be a good thing; both professions are dealing with cross-sectional issues [54,55], address important social and socio-political questions, put general interests at the center of their considerations, are used to working in a system- as well as target group-oriented manner and take the function of policy advisors. In addition, both professions, public health and spatial planning, call for a comprehensive discussion of health issues with particular attention to area-wide measures and equal access to infrastructure.

On the other hand, there are differences between public health and spatial planning experts with regard to dealing with spatial levels of action and the complexity of spatialrelated inequalities of health in old age. When it comes to health and infrastructure

disparities, spatial planning professionals think beyond urban–rural dichotomies and, within the framework of spatial research, draw attention to the importance of the functional interactions between different spatial archetypes in terms of the question of for what purposes people spend time in particular places and where health infrastructure should be located. The so-called multilocal lifestyle is becoming more and more important in this context [56].

#### 7.3.2. Explanations for the Level of Spatial-Related Abstraction in the Impact Goals and Measures of the AAPWH

Despite the public health experts' basic understanding of the relevance of spatialrelated aspects for the health and well-being of women of all ages, they do not think beyond distinct spatial categories, or rather the so-called urban–rural dichotomy. Moreover, they do not mind the absence of a clarification of the so-called "regional reference level of action" with regard to the defined impact goals and measures.

On the contrary, from the public health experts' perspective, the lack of a more precise spatial differentiation should be considered less a failure than a proof of the logic of dealing with cross-cutting and cross-sectional socio-political topics.

The experts' explanations of the standards and particularities of the preparation of inter-ministerial strategic policy—formal specifications such as the predefined number of impact objectives and measures as well as the length of the policy paper on the one hand, and the basic challenge of implementing the requirements of cross-cutting disciplines such as spatial planning on the other hand—can be interpreted as an important limitation of knowledge transfer in the context of evidence-based policy making [57]. A strategic argument for this may also be the political "*desire for a feasible plan*" (I 5) aiming at a win-win-situation for all involved stakeholders [27], which requires to include measures, which build on existing measures or rather be suitable to be integrated into existing actors' and supply landscapes as well as projects and initiatives. This also explains the political approval of the AAPHW and guarantees planning continuity. Despite all criticism, it should be noted that politically speaking "*it is not easy to get everything together*" (I 7).

Moreover, one expert recommends taking the AAPHW for what it is: a living interministerial paper addressing the national level of and expressing the political commitment to the relevance of the subject of women's health in Austria without a defined expiry date and thus, serving as a strategic and operational anchor point for the implementation of changing focal topics in public administration units which are responsible for health provision (planning) at different spatial levels of action [58], encouraging the inter-sectoral and cross-disciplinary networking of experts within the framework of so-called focal points on selected, or rather emerging health topics.

#### *7.4. Some Considerations on the Fit of Findings into an International Perspective*

The results from this research are in line with the findings from other recent studies from the Netherlands [24,27,59] as well as from United Kingdom [28,60] on the general anchor and sticking points in inter-sectoral public health policy making, including the spatial planning perspective.

More generally formulated, the findings from Austria most likely may fit into the perspective of other high-income welfare states where (1) the creation of equivalent living conditions is a supreme political imperative, (2) the public sector takes a major role in the provision of services of general interest and (3) spatial planning is a public responsibility.

Furthermore, it is necessary to point out that the discussion on health-in-all-policies as well as the need for and the potential of inter-sectoral collaboration in order to protect and promote health—with particular regard to older women—in the Global North differs much from the Global South, since in the latter the basic (spatial-related) requirements for good health and well-being such as nutrition, sanitation, housing, security and medical care are still not met. Particularly, this situation limits the international transferability of the inferred conclusions of this research presented below and moreover, underlies the

challenge of creating a geographically and socio-culturally overarching global mindset on public health in the foreseeable future [61].

#### **8. Conclusions and Outlook**

Conclusion 1: The degree of the spatial-related abstraction of the impact goals and measures can be explained by the fact that in Austria spatial planning as a cross-cutting discipline—as stated by Storm et al. [24]—has not yet been included in the strategic discourse on health protection and health promotion.

Conclusion 2: Both professions, public health and spatial planning, have similar ideas about the complexity of the space–health nexus and the importance of (governing) values in planning [58,62]. Thus, the joint dialogue in the expert group and the reflection on the impact goals and measures were experienced as fruitful on the part of both sides. The spatially differentiated approach and the way of reasoning in spatial planning can thus enrich the interdisciplinary discourse on women's health issues. For this reason, some of the interviewees also expressed their wish to keep in touch with the author of this article. This has already happened—for example in the context of network meetings or targeted information about current publications.

Conclusion 3: The need for a closer cooperation of public health and spatial planning as claimed amongst others by Tomlinson et al. [63], by McKinnon et al. [60] as well as by Hendriks et al. [64] in general or Lowe et al. [65] for the urban context in particular emerged during the expert discussions. A concrete thematic starting point for a further dialogue between the two disciplines could be the issue of long-distance caregiving, a topic still neglected in public health in Austria [45,66]. The main challenge of including "new" issues or target groups in the AAPWH is to integrate them into the right impact goal(s) and already existing measure(s).

Conclusion 4: It would be valuable to analyze the AAPWH in the context of an intertextual content analysis [67] with regard to the consideration of the spatial relatedness of the impact goals and measures for the two other target groups "Girls and Young Women" and "Women of Working Age" and subsequently—as shown here for the expert group of "Women in Old Age"—to reflect them in an interdisciplinary manner. An in-depth and continuous dialogue between public health and spatial planning experts may reveal cross-connections between target group-specific needs for action and thus perhaps bring to light the new cross-target group's priority themes, impact goals and measures including explicit and implicit spatial references.

Conclusion 5: Spatial and planning scholars must learn to understand that a change towards a comprehensive, or rather holistic, approach to health issues including the "spatial dimension" takes time. At the academic level, key representatives of other relevant disciplines (including, above all, public health) must be introduced to the mindset of spatial planning; at the political level, much effort is still needed to raise awareness and to sensitize all relevant stakeholders to the space–health nexus as a main reason for inequalities in (women's) health and to take ownership of the discovered interrelations [68,69]. Therefore, especially against the backdrop of demographic ageing, climate change and the impact of pandemics, as in line with the claim for more "evidence-informed public health policy" [26], spatial and planning scholars are encouraged to:


**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Ethical review and approval were waived for this study, due to the character of this research. It does not count as investigative medical research.

**Informed Consent Statement:** Informed consent was obtained from seven out of eight involved experts in this research. Therefore, this article only relates to findings from experts who gave consent.

**Data Availability Statement:** No new data were created or analyzed in this study. Data sharing is not applicable to this article.

**Acknowledgments:** The author would like to thank all interviewees for their support and openness. The open access publishing was supported by the BOKU Vienna Open Access Publishing Fund.

**Conflicts of Interest:** The author declares no conflict of interest.

#### **References**


## *Article* **Economic and Social Factors That Predict Readmission for Mental Health and Drug Abuse Patients**

**Quang "Neo" Bui \* and Emi Moriuchi**

Rochester Institute of Technology, 105 Lomb Memorial Drive, Rochester, NY 14623, USA; emoriuchi@saunders.rit.edu

**\*** Correspondence: qnbbbu@rit.edu; Tel.: +1-585-475-4411

**Abstract:** According to the United Nations, curtailing the rise of mental illness and drug abuse has been an important goal for sustainable development of member states. In the United States, reducing readmission rates for mental health and drug abuse patients is critical, given the rising health care costs and a strained health care system. This study aims to examine economic and social factors that predict readmission likelihood for mental health and drug abuse patients in the state of New York. Patient admission data of 25,846 mental health patients and 32,702 drug abuse patients with multiple visits in New York hospitals in 2015 were examined. Findings show that economic factors like income level and payment type impact readmission rates differently: The poorest patients were less likely to get readmitted while patients with higher incomes were likely to experience drug relapse. Regarding social factors, mental health patients who lived in neighborhoods with high social capital were less likely to be readmitted, but drug abuse patients in similar areas were more likely to be readmitted. The findings show that policy-makers and hospital administrators need to approach readmission rates differently for each group of patients.

**Keywords:** readmission; social capital; economics; mental health; drug abuse

"The inclusion of noncommunicable diseases under the health goal is a historical turning point. Finally, these diseases are getting the attention they deserve. Through their 169 interactive and synergistic targets, the Sustainable Development Goals (SDGs) seek to move the world towards greater fairness that leaves no one behind."

Dr Chan, 2013 WHO Director-General

#### **1. Introduction**

In 2013, the 66th World Health Assembly adopted a comprehensive plan to curtail the rise of mental illness and drug abuse worldwide [1]. The then World Health Organization (WHO) director, Dr. Chan, called this a "historical turning point" that moved the world toward a more sustainable future. Since then, treating mental health and drug abuse has always been integrated into the United Nations (UN) platform to promote sustainable development among member states [2]. Responding to this, various studies have focused on factors that can predict and curtail mental illness and drug abuse in various contexts [3–7].

In the United States (US), mental health and drug abuse patients have steadily increased in recent years. According to the 2017 National Survey on Drug Use and Health, nearly one in five US adults suffered from a mental health condition, and one in eight US adults struggled with both alcohol and drug use disorders [8]. The Mental Health America institute estimated that youth mental health is worsening with an increase of 4.3% over five years for youth age 12–17 [9]. The opioid crisis, which has killed 128 people every day due to overdose [10], is an example of severe drug abuse issues in the US. In addition, with the Covid-19 pandemic in 2020, it is estimated that there will be even more people suffering from the psychological impacts of lock-downs due to emotional stress and financial distress [3].

**Citation:** Bui, Q.N.; Moriuchi, E. Economic and Social Factors That Predict Readmission for Mental Health and Drug Abuse Patients. *Sustainability* **2021**, *13*, 531. https://doi.org/10.3390/su13020531

Received: 17 November 2020 Accepted: 6 January 2021 Published: 8 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

In addition, there is a sustainability crisis in the US public health care system [11]. It has a declining tax base and diminishing social values that are encouraging more private sector choices, and there are two views on addressing the sustainability of the US public health care system [12,13]. First, while there may be economies in more efficient administration of the public health care system that will address sustainability concerns, more scrutiny is needed of how funding is provided. Those who maintain that the system is unsustainable would argue that public funding and administration is part of the problem. The solution may be that Americans have to accept less-comprehensive public health insurance, with more services being paid for out-of-pocket or by private insurance. In this situation, supporters of this view would believe that the private-for-profit insurance companies are the source of the health expenditure increase. A second view is based on the rising cost of health care, which is threatening to overwhelm the public health care system. Thus, a major structural reform of the system is required to encourage better public management as it has the opportunity to provide greater efficiency in the form of faster service and greater choice for Americans [14]. Furthermore, the use of public funds to provide health care suggests that when the expenditure for health care increases, either taxes must be increased or public services reduced. Thus, to avoid such negative effects, public health care needs to maintain quality while addressing the individual's health needs accurately (e.g., differentiating illness accurately).

Against this backdrop, this study seeks to identify economic and social factors that predict readmission rates of mental health and drug abuse patients in the state of New York. Understanding these factors will help policy-makers and health administrators pursue high-quality health care and improve public health in a sustainable manner without exhausting limited resources.

This study's focus on the state of New York is motivated by the state's high readmission rates, with 93% of hospitals estimated to be penalized for such high rates [15]. In 2008, it is estimated that 15% of all hospital stays in NY result in readmission, costing \$3.7 billion per year [16]. Reducing readmission rates has become a priority for state public health officials. In addition, the state has a high growth of mental health and drug abuse patients, ranking fifth in the nation [17]. This makes it more critical to understand the factors that impact readmission rates for mental health and drug abuse patients in the state of New York.

#### **2. Conceptual Foundation**

#### *2.1. Readmission Rates in the US Health System*

Compared to other developed countries, hospitals in the United States have higher readmission rates [18]. Given the rapid rise of health care costs and rampant inefficiencies in the health care system, reducing readmission rates has been a crucial goal for quality health care and sustainable development in the US [19]. To combat this issue, in the US, since 2013, Medicare reimbursement has been linked to hospital 30-day readmission rates for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PN). Subsequently, reducing readmission rates has become an important indicator of hospital performance in the US, and a growing number of studies have examined factors that contribute to reduced readmissions. To date, prior studies have suggested a plethora of factors, ranging from hospital and treatment characteristics [20–22] to patient-level social and economic factors [23–25]. These studies make it clear that readmission is a complex issue dependent not only on hospital-related factors but also on out-of-hospital factors such as social support [25], economic means [20,26], community factors [24], or even county-level characteristics [23].

Because readmission rates historically grow out of the concern from the Centers for Medicare and Medicaid Services (CMS) for AMI, HF, and PN patients, most readmission rate research has focused on patients with chronic conditions [20,22,25]. Recent studies have investigated readmission rates for all medical care services [23,24], for insured patients [21,26], or for patients with recent surgery or with pneumonia [27]. However, hospital readmissions for other different types of patients, including mental health and drug abuse

patients, are still understudied. Studies have found that mental health and drug abuse patients indeed have a higher risk of readmission compared to other groups [28,29]. Thus, the objective of this study is to examine factors that contribute to readmission rates among mental health and drug abuse patients in the state of New York.

In addition, while some studies have looked at readmission rates for mental health and drug abuse patients [5,27,30,31], they often aggregate findings for both types of patients as one category. This study argues that such analyses can be incomplete as mental health patients possibly differ from drug abuse patients in terms of demographic factors, service utilization patterns, and diagnostic services [4]. Thus, this study aims to compare results for each type of patient to understand specific influential factors for their readmission rates.

#### *2.2. Conceptual Model*

Building on prior research, it is hypothesized that the likelihood of readmission for a patient will be explained by three groups of factors: Hospital treatments, economic factors, and social factors. First, the nature of treatments that patients receive can determine how likely they are to relapse and be readmitted. This is especially important for mental health and drug abuse patients [32], and prior studies have pointed out that those two types of patients utilize treatments and service hours differently [4]. Specifically, drug abuse patients had more treatments and longer stays than mental health patients [4], while prior diagnostic history was a strong predictor for readmission rates among mental health patients [30,33]. Thus, this study hypothesizes that these differences in hospital treatments will explain the readmission rate for mental health versus drug abuse patients.

Second, various studies have attributed readmission rates to economic factors, especially whether patients have access to economic means to afford hospital visits [20,21,25,26]. For mental health and drug abuse patients, economic factors can also reflect their neighborhood living conditions as low-income patients often cluster in poor areas and are more prone to mental health problems or entrenched drug usage. For instance, several studies have associated homelessness with a higher risk of readmission for mental health patients [30,34,35]. Thus, it is hypothesized that economic factors can help explain the readmission rates for mental health and drug abuse patients.

Finally, recent studies have posited that social factors also predict patient readmission rates [23,24]. Instead of relying on economic factors as a proxy for social impacts, these studies explored the supporting characteristics of patient living environment or community health system factors. For example, several researchers have suggested that follow-up in community hospitals can reduce readmission rates for mental health patients [30,31]. Others have identified stronger community support can reduce readmission rates in general [24,33,36]. Compared to hospital treatment factors (hospital-controllable) and economic factors (patient-controllable), these community factors are uncontrollable for hospitals and patients. Thus, they can provide a complete understanding of readmission rates. In this paper, it is hypothesized that social factors play a critical role in readmission rates of mental health and drug abuse patients because those patients need a wide range of social support to overcome their issues.

In sum, this study's conceptual model (Figure 1) uses hospital treatments, economic factors, and community factors to explain readmission rates for mental health and drug abuse patients. One model is built for each type of patient, and the findings are compared and contrasted to unveil insights that can inform policy-makers on how to reduce readmission rates for those patients. Next, data sources and variables used in the analysis are discussed.

**Figure 1.** Conceptual model.

#### **3. Materials and Methods**

*3.1. Data Sources and Variables*

The study used 2015 discharge data from the New York State Inpatient Databases (SID) by the Healthcare Cost and Utilization Project (HCUP), and Agency for Healthcare Research and Quality [37]. The original dataset contained 2.29 million records. The unique patient identifier was used in the dataset to filter out patients with multiple hospital visits in 2015, and patients who received services classified as mental health and drug abuse services. This left 120,140 patients.

To distinguish mental health versus drug abuse patients, the ICD-10 Procedure Coding System (ICD-10-PCS) was used to identify specific services received by patients during their hospital visits. Specifically, mental health patients were those who received service coded 218 (psychological and psychiatric evaluation and theory) while drug abuse patients were those who received service coded 219 (alcohol and drug rehabilitation/detoxification). Excluding patients with missing data, the final datasets contained 25,846 mental health patients and 32,702 drug abuse patients.

From the patient's visit date and length of stay, a calculation was made as to whether the visit was readmission within 30 days of last visit discharge. This gave a primary outcome variable for hospital readmission (1/0 indicator). The independent variables come from three groups: Hospital treatments, economic factors, and social factors, with demographics as control variables.

Hospital treatments included the number of diagnoses received during a visit, the number of procedures received during a visit, the number of external causes of injury, and length of stay [5,30,31,38]. Following prior studies [5,21], economic factors, such as the median household income for a patient's zip code, and insurance type of primary payer were used. Insurance payment method includes private (self-pay, group insurance) and public (Medicare, Medicaid). Income was determined based on New York State's income categories (e.g., 1 = under poverty <\$12,760 for an individual, <\$24,600 for a family of four). Demographics included sex and age [30].

Social factors came from the social capital index for a patient's county [39]. The social capital index is developed by the Northeast Regional Center for Rural Development (http: //aese.psu.edu/nercrd), and it is calculated using an array of individual and community factors to measure the socio-economic growth of a community. Prior studies have suggested that social capital within a community will impact patient readmission rates [23,24,40]. For this study, the social capital index is particularly relevant as supporting communities are likely to help reduce readmission rates for mental health and drug abuse patients [24,36].

#### *3.2. Statistical Analysis*

To estimate the likelihood of readmission for patients, a multinomial logistic regression was used. This type of regression uses maximum likelihood estimation to predict the probability of category membership on a dependent variable based on multiple independent variables. Goodness of fit analyses were conducted to accurately specify the model, in which a gamma distribution and log link were selected.

The sample size was split into two groups: Mental health patients with a total of 25,846 cases, and drug abuse patients with a total of 32,702 cases. The assumption of multinomial logistic regression in this study was that readmission due to a diagnosed illness is not related to the readmission for another diagnosed illness. In our mental health patient sample, a majority of the patients were hospitalized based on the occurrence of an emergency (82.1%), and 61.1% identified as male. The average age was 42 years old, and the average length of stay was 14 days. On the other hand, 52.4% of drug abuse patients were hospitalized due to an emergency, and 75.4% identified as male. The mean age was 43 years old. The average length of stay was 6.5 days.

Our model is as follows. The dependent variable Y = 0 if not readmitted, Y = 1 if readmitted.

$$y\_i = \begin{pmatrix} y\_{i1} \ y\_{i2} \ \dots \ y\_{ir} \end{pmatrix}^T$$

Readmission = length of stay + number of diagnoses + number of procedures + number of external causes + median household income + payment type + social capital + sex + age.

#### **4. Results**

#### *4.1. Mental Health Patients*

A multinomial logistic regression was performed to model the relationship between the predictors and type of illness in the two groups (mental health issues and drug abuse). The traditional 0.05 criterion of statistical significance was employed for all tests. For mental health patients, the final model showed a good fit between model and data, χ2(14, N = 25,846) = 199.70, Nagelkerke R<sup>2</sup> = 0.01, *p* < 0.001. Goodness-of-fit results showed that Pearson Chi-square was insignificant (*p* = 0.41), which indicated that the model fit the data well. Table 1 shows that for mental health patients, their readmission rates are significantly predicted by length of stay, number of diagnoses, number of procedures, median income, payment type, social capital, gender, and age.

Consistent with prior studies [22], our results showed that for mental health patients, their readmission likelihood was predicted by hospital treatments: Specifically, by the number of diagnoses and the number of procedures, but not the number of external causes. While the more diagnoses a patient had, the lower the readmission odds (negative coefficient), the more procedures a patient had, the higher the readmission odds (positive coefficient). A one-unit increase in the number of diagnoses will lead to a 0.02 decrease in the relative log odds of being readmitted as a mental patient, while a one-unit increase in the number of procedures will result in a 0.025 increase in the readmission odds. Length of stay appears as a significant predictor, but its beta was small, indicating its low impact.

Economic factors showed positive impacts on readmission likelihood, but the effects varied. Specifically, median income level as a whole showed a negative significant impact (*p* < 0.05) on readmission, and for every unit increase in the income level, there was a 0.08 unit decrease in readmission odds. However, among different income categories, patients who fell under the poverty line tended not to be readmitted (β = −0.79, *p* < 0.05), while the effects of income were negated for other income levels. On the other hand, the payment method (i.e., insurance coverage) also had an impact on readmission odds, but only among patients who used Medicare or Medicaid (β = −0.30, *p* < 0.05; β = −0.39, *p* < 0.01, respectively).


**Table 1.** Findings from Multinomial Logistic Regression for Mental Health vs. Drug Abuse Patients.

\* *p* < 0.05, \*\* *p* < 0.01.

Additionally, the results showed that a one-unit increase in the social capital status is associated with a 0.04 decrease in the relative log odds of being readmitted to the hospital as a mental health patient. The beta shows a negative result, which suggests that patients who have higher social capital (which reflects the social environment of their neighborhood) will have lower readmission odds. This finding is aligned with prior studies that have suggested the positive impacts of social influences on readmission rates [24,25].

Age was also found to have a significant positive impact on readmission odds (*p* < 0.05). Income level showed a negative significant impact (*p* < 0.05) on readmission. For every unit increase in the income level, there is a 0.08 unit decrease in readmission odds.

#### *4.2. Drug Abuse Patients*

Table 1 shows that for drug abuse patients, significant predictors for their readmission were length of stay, number of diagnoses, income level, payment method, social capital, gender, and age. The model has a good fit, χ2(15, N = 32,702) = 791.54, Nagelkerke R2 = 0.04, *p* < 0.001. Goodness-of-fit results also show that Pearson Chi-square was insignificant (*p* = 0.28), which indicates that the model fits the data well.

Among hospital treatment factors, only length of stay and number of diagnoses were significant predictors for readmission odds among drug abuse patients. Interestingly, every day remaining hospitalized was associated with a 0.04 increase in the readmission odds (β = 0.04). On the other hand, a one-unit increase in the number of diagnoses was associated with a 0.017 decrease in relative log odds of being readmitted (β = −0.017, *p* < 0.01).

As with mental health patients, economic factors also had a significant impact on readmission odds for drug abuse patients. At the aggregate level, income level had a significant negative impact on readmission odds. A one-level increase in the variable income level is associated with a decrease in the relative odds of being readmitted. While only income below the poverty line impacted readmission odds for mental health patients, higher income brackets also impacted drug abuse patients (below the poverty line, between \$24,000–\$35,000, and above \$35,000). More interestingly, patients below the poverty line saw reduced readmission odds while patients with higher income levels saw a higher chance of being readmitted as a drug abuse patient.

Payment through Medicare, self-pay, and private insurance, but not through Medicaid, had a positive impact on readmissions. Specifically, every one-unit increase in the use of Medicare is associated with a decrease of 0.24 units in readmissions odds (Exp(β) = 0.79). On the other hand, every one-unit increase in self-pay is associated with a 0.28-unit increase in readmission and a 0.36-unit increase for private insurance payer.

Social capital had an opposite effect on drug abuse patients compared to mental health patients. The results showed that a one-unit increase in social capital is associated with a 0.05 increase in being readmitted as an opioid patient (β = 0.05, *p* < 0.01). Combined with the findings related to the income level above, this further confirms that drug abuse patients who have more disposable income and live in good neighborhoods are more likely to relapse.

For demographics, every year increase in age is associated with a 0.01 decrease in the relative log odds (Exp(β) = 1.01) of being readmitted. Moreover, males were more likely to be readmitted than females.

#### *4.3. Patients with Both Mental Health and Drug Abuse Issues*

The number of patients who have been diagnosed to have two types of illness (drug abuse and mental health) is 4226. Similar to the two models above, a multinomial logistic regression was performed to model the relationship between the predictors and their readmission likelihood. The traditional 0.05 criterion of statistical significance was employed. The final model showed a good fit between model and data, χ2(15, N = 4226) = 1021.23, Nagelkerke R<sup>2</sup> = 0.01, *p* < 0.001. Goodness-of-fit results showed that Pearson Chi-square was insignificant (*p* = 0.54), which indicated that the model fit the data well. Table 2 shows that significant unique contributions were made by all the factors except the number of external causes and age.


**Table 2.** Findings from Multinomial Logistic Regression for Patients with both Mental Health and Drug Abuse.

\* *p* < 0.05, \*\* *p* < 0.01.

When the data were analyzed based on a patient who has been diagnosed with two types of illness, length of stay had a positive significant impact on readmission odds. For every day in the hospital, there is an increase of 0.01 units in relative log odds of being readmitted as a patient. The number of procedures had a positive significant impact on readmission odds. For every increase in the number of procedures done on the patient, there is an increase of 0.12 units in the odds of being readmitted as a patient. Being a patient

who fell under the poverty line had a significant negative impact on being readmitted. For every increase in income bracket, there is a decrease of 0.22 units of being readmitted as a patient. The odds ratio is Exp (β) = 0.81. For dual illness patients, the results showed that payment method did have an impact on their readmission odds. Additionally, social capital did not have an impact on patients' readmission odds. The results showed that males were more likely to be readmitted than females.

#### **5. Discussion**

This study examines contributing factors that predict readmission likelihood for mental health and drug abuse patients. Readmission rates in the United States have been high for many years [8,18], and many institutions have been faced with financial penalties for high readmission rates [27]. To address these concerns, hospitals are seeking a new path forward to reduce readmissions. Several studies have reported different ways of reducing the readmissions rate. These include improving patient safety at hospital discharge [41], enhancing medication reconciliation [42], and improving the transition from inpatient to outpatient setting [43]. However, these prior studies often focus on chronic diseases [20,22,25] or combine mental health and drug abuse patients into one group [5,27,32]. This study separates these two groups of patients to discern differences in factors that impact their respective readmission odds. The findings showed some similarities and differences. For both groups, hospital treatments, economic factors, and social factors played significant roles in predicting readmissions. However, their effects varied across the two groups. Specifically, the number of procedures was a significant predictor for mental health patients' readmissions, but not for drug abuse patients, while length of stay was a significant predictor for drug abuse patients' readmissions but not for mental health patients. This is a contrasting finding with prior studies as researchers have associated lower length of stay with lower readmission rates for mental health patients [31,32]. Interestingly, for mental health patients, the number of procedures had a positive impact, and the number of diagnoses had a negative impact. Prior research has often associated the number of procedures with lower readmission rates [22,38], but has not scrutinized the number of diagnoses. Future study is needed to explain the underlying reason.

In terms of economic factors, the findings confirm prior studies that economic factors matter [5,34], however, prominent differences between the two types of patients were found. Patients in higher income levels were likely readmitted for drug abuse issues but not mental health issues, and private insurance and self-pay significantly predicted readmissions of drug abuse patients but not mental health patients. Relating back to the Affordable Care Act, mental health is covered by Medicare and Medicaid, so this finding that only Medicare and Medicaid patients are frequently readmitted makes sense. This finding also suggests that healthcare accessibility through economic means has different effects on different types of patients. This seems to indicate that patients with more disposable income are likely to relapse to drug abuse. This finding is also related to social capital impact when an opposite effect is observed: High social capital locations were associated with a higher chance of readmission for drug abuse, but lower readmission odds for mental health patients. This is a surprising finding given prior studies have often associated well-off communities with lower admission rates overall [23,24,31,36,43]. Thus, the findings indicate that communitybased support should be strategically allocated for each type of disease, as found in this current study.

The findings have several implications for societal sustainability. First, this study illustrates the importance of healthcare accessibility to the reduction of readmission rates for mental health and drug abuse patients. It echoes prior studies and suggests policymakers pay greater attention to economic inequality as a direct influencer on community well-being [23,24]. For instance, given patients with high income levels who live in neighborhoods with high social capital actually have higher readmission odds for drug abuse issues, community leaders in well-off areas can consider incorporating rehabilitation facilities to address the issue. Second, the findings inform hospital administrators of various factors that can be used as an indicator of potential readmission among mental health and drug abuse patients. By identifying potential relapse, hospitals can reduce inefficiencies and get closer to a more sustainable healthcare system [19]. Finally, the findings show differences between mental health and drug abuse patients, which suggests the need for different policies to reduce readmission rates for each group of patients. For instance, patients with lower socio-economic means are likely to suffer from mental illness, thus governmental-level support is needed to help this population (e.g., extending coverage for mental illness).

The study is not without limitations. The data focused solely on New York State, thus, the findings are generalizable only to states that have similar demographics and populations. However, nation-wide data would be more comprehensive in addressing the shortfall for this study. A second limitation is that this data set is focused only on one year of readmission data. A longitudinal data set would benefit the study of these readmission rates in a time series manner. In other words, other determinants, such as a change in public policy due to a change in political parties, could be used to compare the difference in insurance cost and how that will impact readmission rates.

#### **6. Conclusions**

Hospitals in the United States are financially penalized for having a higher than expected thirty-day readmission rate among patients who have comorbid mental health diagnoses or other symptoms. Traditionally, hospitals have been categorizing readmission rates between drug abuse patients with mental health patients [5,27,31,32]. It is also unknown what the effect of distinguishing the readmission data into its respective disease could have on readmission rates. While many patients are comorbid patients, this study found that although the effects vary in each group, it is important to have different and separate policies to reduce readmission rates for patients with different types of diseases. Prior studies argued that providing support to mental health patients after their discharge helps with reducing physical health readmissions [36]. In addition, prior studies found that alcohol dependence and other mental disorders are associated with inpatient admission or emergency department (ED) visits [5]. In that same study, the authors found that insurance types were predictors of readmission. This study contributes to the existing literature by utilizing hospital discharge data from the state of New York to understand predictors for readmission rates of mental health and drug abuse patients. This study investigated not only the hospital-controllable and patient-controllable factors (i.e., hospital treatments and economic factors) but also uncontrollable factors, such as social determinants, to predict the readmission odds of mental health and drug abuse patients. Considering the high rate of readmissions and ED use in the United States [18], and the concomitant spending by patients, such efforts to address these knowledge gaps could improve patient outcomes and reduce readmission rates, which leads to a reduction of health care costs in a sustainable manner.

**Author Contributions:** All authors have contributed equally to the conceptualization, analysis, writing, and preparation of this study. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** This study uses 2015 discharge data from New York, State Inpatient Databases (SID), Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality (https://www.hcup-us.ahrq.gov/).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Inequalities and the Impact of Job Insecurity on Health Indicators in the Spanish Workforce**

#### **Raúl Payá Castiblanque \* and Pere J. Beneyto Calatayud**

Department of Sociology and Social Anthropology, University of Valencia, Av. dels Tarongers 4b, 46022 Valencia, Spain; Pere.J.Beneyto@uv.es

**\*** Correspondence: raul.paya@uv.es; Tel.: +34-650-157-401

Received: 15 June 2020; Accepted: 7 August 2020; Published: 10 August 2020

**Abstract:** In a context of high job insecurity resulting from social deregulation policies, this research aims to study health and substance abuse inequalities in the workplace from a gender perspective. To this end, a transversal study was carried out based on microdata from the National Health Survey in Spain—2017, selecting the active population and calculating the prevalence of the state of health and consumption, according to socio-occupational factors (work relationship, social occupational class, time and type of working day). Odds ratios adjusted by socio-demographic variables and their 90% confidence intervals were estimated by means of binary logistic regressions stratified by sex. The results obtained showed two differentiated patterns of health and consumption. On the one hand, unemployed people and those from more vulnerable social classes showed a higher prevalence of both chronic depression and anxiety and of hypnosedative and tobacco use. On the other hand, the better positioned social classes reported greater work stress and alcohol consumption. In addition, while unemployment affected men's health more intensely, women were more affected by the type of working day. The study can be used to design sustainable preventive occupational health policies, which should at least aim at improving the quantity and quality of employment.

**Keywords:** job insecurity; health and consumption indicators; gender inequalities; sustainable preventive policies

#### **1. Introduction**

More than a decade has passed since the financial crisis and the stagnation of the global economy in 2008 (the great recession) began and austerity policies (the great aggression) imposed by the Troika (formed by the European Commission, the European Central Bank and the International Monetary Fund) based on a political exchange of "neoliberal intergovernmentalism" that forced the member states of the European Union with economic difficulties, especially the countries of the South (Spain, Greece and Portugal), to deregulate the labor market and labor relations [1,2] with the "conditionality" of obtaining financial aid and bank bailouts [3]. These policies have led to a radical transformation of industrial relations models and the breakdown of the fragile balances achieved during decades of social dialogue by deregulating the three historical collective mechanisms that have acted in the defense and protection of workers: protection of legality, trade union intervention and business coverage [4]. This aggression has meant a great regression that, on the one hand, has led the most disadvantaged social classes to even worse living conditions, and, on the other hand, has slipped the middle classes into economic fragility, extending economic and social vulnerability to large social strata [5].

In this regard, with regard to the extent of social and economic vulnerability in the field of health, a study by Stuckler et al. [6,7] in post-communist countries found that massive privatization programs in the health system increased short-term adult male mortality rates by 12.8% among the most disadvantaged social classes. On a national level, the impact of austerity policies on the health of the Spanish population has been the subject of numerous investigations from the public health field, concluding that the effects on the population are heterogeneous and controversial, endangering the sustainability of the national health system [8–12]. In particular, important differences have been identified between the Autonomous Communities in terms of the management of the economic crisis and austerity policies. While the government of the Basque Country did not implement austerity policies during the crisis, in La Rioja, Madrid and the Balearic Islands, privatization policies were implemented in the health system [8]. In addition, the study conducted by Del Pozo-Rubio et al. [10] showed how the co-payment of dependency introduced in Spain through the Resolution of 13 July 2012 meant, on the one hand, an unequal application of co-payments between the Autonomous Communities, with Andalusia, Valencia and Catalonia having the highest levels of co-payment, and, on the other hand, how the co-payment went from 20% in the national average before the reform to 53.54% after the reform. Thus, these studies would confirm both the inequalities between social classes and between the different regions of Spain. In addition, there is empirical evidence on problems of technical quality (misdiagnosis or inappropriate diagnosis) and interpersonal quality (poorer treatment and communication) in the public health system related to cuts in the health workforce, which affects the whole population, but especially people with fewer resources and immigrants [12]. These cuts and privatizations could explain how since the beginning of the economic crisis in Spain, there has been a significant slowdown in the reduction of the cancer mortality rate [11], which would be related in some way to the studies by Stuckler and others [6,7]. This context of economic crisis jeopardizes the sustainability of the welfare state, as the protection system (healthcare and social benefits) is financed in most European countries through employers' and workers' contributions to work performance. As a result, austerity measures reduce state revenues, which are largely dependent on full employment and decent wage policies, leading to severe cuts in public health and other social expenditures [13]. High unemployment rates in turn erode the bargaining power of workers and their class organizations, feeding back into the spiral of deregulation and deterioration of working conditions and occupational health [14].

In this context and for the purposes of this research, there has been an increase in job insecurity both in the European Union (EU) in general and in Spain in particular [15]. Job insecurity can be defined as a perceived threat to the continuity and stability of employment as currently experienced [16] or the loss of well-being resulting from job uncertainty [17]. Different domains or facets of job insecurity can be drawn from these definitions. The first would be uncertainty in a threefold dimension: (a) uncertainty about whether or not employment will eventually be lost; (b) uncertainty about when it will occur, i.e., when the job will be lost; and (c) uncertainty about the consequences of the loss of employment [18]. The second domain would be the threat, since the uncertainty of loss of employment is comparable to the severity of the threat [19]. That is, depending on the possibilities of finding new employment and the degree of dependence on wages for survival, the degree of threat will be greater or lesser, and therefore, is related to some theories of human needs (for example, Maslow's pyramid) [20]. Finally, there is a third dimension that refers to the powerlessness or absence of strategies available to workers to resist the threat of dismissal [21–23]. Thus, the lack of protection (trade unions, working without a contract, unemployment benefit systems, and so on) makes it more or less likely that they will resist the threat of unemployment [22]. In the scientific literature, various ways can be found to study and make job insecurity operational in order to measure it. On the one hand, there are studies that focus on the analysis of perceived or subjective insecurity [24], understood as an interpretation or evaluation by the worker of a series of external signs that have to do with job continuity [25]. On the other hand, there are studies that focus on the analysis of attributed insecurity [26], that is, on the objective signs of insecurity (contractual situation, position of the worker in the company, working conditions, etc.) that do not depend on the worker's perception [24]. Although there are various ways of studying job insecurity, the fact is that perceived or subjective insecurity and attributed or objective insecurity are related, insofar as, although perceived insecurity depends on personal and contextual factors, which may lead some workers to overestimate the probability of losing their jobs,

the fact is that there is empirical evidence that correlates the subjective dimension with the objective one [27]. In fact, it has been shown that temporary contracts are associated with greater self-perceived insecurity [28–30] and the transformation of temporary contracts into permanent contracts with a greater perception of security [31]. The focus of this research is on attributed insecurity and therefore requires further analysis. In this regard, the previous literature has identified four types of studies related to objective insecurity: (a) research that focuses on studying insecurity dynamically through unstable employment trajectories [32,33]; (b) insecurity produced by closures, restructuring or downsizing, including those workers not affected by layoffs [34,35]; (c) job insecurity from the point of view of the type of contract or contractual relationship (temporary or permanent) [36,37]; and (d) research that studies job insecurity from a multidimensional point of view not only based on the contractual relationship but also including other elements such as occupational social class or working time, approaching the concept of job insecurity [38–41]. From the classification given, this study is part of the proposals for measuring insecurity attributed from a multidimensional perspective. It should be noted that this holistic perspective is related to labor precariousness, since this construct is broader and contemplates insecurity but not in an isolated manner [25].

Focusing on the health effects of perceived job insecurity, previous studies have identified how a subjective perception of insecurity leads to an erosion of job satisfaction [42], increased feelings of anxiety [15], or high levels of stress comparable to those who are unemployed [16]. However, the psychological health effects of job insecurity can be modulated by subjective employability [43]. In other words, subjective employability differs from objective employability (observable contextual conditions such as contractual conditions [44]) because it focuses on people's belief that they can easily find a new job based on their genuine skills, such as work experience or educational level [45], and that previous studies have shown that it is associated with lower levels of psychological risk when unemployment is addressed proactively [45–47].

With regard to the health effects of attributed or objective job insecurity, it has been shown that unemployment exposes individuals to greater psychological risk [48,49]. Specifically, unemployment has been associated with a worse self-perceived general health status [50], increased mental illness such as anxiety and depression [51,52], psychosomatic and sleep disorders [43,49], the use of hypnosedatives [50,53], addictive behaviors such as the consumption of alcohol, tobacco or drugs [50,53,54], and even family conflicts and suicides [48]. If we focus on the multidimensional perspective of this study, previous research has observed elements of job insecurity attributed to the increase in psychosomatic disorders and unhealthy habits, such as the use of hypnosedatives and addictive substances that erode people's health [55–60]. Specifically, with regard to contractual status, working conditions and occupational social class, it has been identified that (a) people on temporary contracts use health services less frequently for fear of being absent from work and dismissed [55]; (b) working long hours has been associated with higher levels of alcohol consumption [55]; (c) night work has been associated with regular smoking [56]; (d) high levels of occupational stress have been associated with a higher prevalence of alcohol consumption [57,58] and the use of hypnosedatives [59]; and (e) the more vulnerable manual social classes have been associated with poor mental health [60] and regular tobacco use [55].

From the findings of the previous literature, it is possible to observe multiple and complex bilateral and multilateral relationships between socio-professional factors, on the one hand, and the spiral of constant health deterioration on the other. For example, work-related stress has been associated with depression or anxiety [60,61]. These mental disorders are in turn linked to the use of hypnosedatives [59] and addictive behavior [57,58]. Even among the most disadvantaged manual workers, alcohol consumption has been found to be associated with an increased likelihood of losing their job [62], which deepens the feedback between social vulnerability and health impairment.

Given the complexity and current occupational vulnerability in Spain and the scarce specific and partial studies that study the associations between mental health, the use of hypnosedatives and the consumption of addictive substances in the workplace, it is necessary to carry out more extensive analyses to explore possible patterns of relationships between all the aforementioned variables of health and consumption of the active population in the labor market, in order to establish sustainable health and employment policies that reverse the health emergency situation caused, to a certain extent, by the economic crisis management policies themselves. Therefore, the main objective of this research is to explore, holistically and jointly, the possible patterns between the main occupational factors of attributed or objective job insecurity (type of contract or employment relationship, occupational social class, working time and type of working day) and the various health factors (general and mental state, consumption of hypnosedatives, tobacco and alcohol) in the Spanish active population. In addition, several studies indicate that the different gender roles in the area of reproductive tasks [63,64] and the precarious working conditions that affect working women most intensely [65,66] make them more likely to refer to psychosomatic disorders and to consume more hypnosedatives [67–69]. In light of these findings, it is considered relevant to address the objective of this research to explore health and consumption patterns by stratifying the working population sample by sex.

#### **2. Materials and Methods**

#### *2.1. Sample and Study Population*

In order to achieve the proposed objectives, the use of the microdata from the questionnaire for adults from the National Health Survey (ENSE, 2017) [70], carried out by the Ministry of Health, Consumption and Social Welfare of Spain, was considered the most suitable source for carrying out the study, since for each Autonomous Community, an independent sample was designed, which allowed for having a large and representative sample of the entire country [71]. The sample carried out was a polytopic one. In the first stage the census sections were selected and in the second stage the main family dwellings were selected. In each dwelling, an adult person aged 16 years old or over was selected to carry out the adult questionnaire. The fieldwork was extended between the months of October 2016 and October 2017, for the purpose of collecting data that might be affected by seasonality. The total size of the ENSE survey for adults in 2017 was 23,089 persons, with a high response rate of 95%. Information was collected through personal interviews. The same was complemented, in exceptional cases, by means of a telephone interview. For the present study, only the active population was selected. The active population was considered to be those persons who were of working age (16 years old or over in Spanish legislation) and who carried out a professional activity, as well as those persons of working age who were unemployed and who were actively seeking employment [72]. Thus, the sample for this research was 12,260 persons between the ages of 16 and 64 years old. Specifically, the study included a sample of 6299 men (5163 (82%) employed and 1136 (18%) unemployed) and 5931 women (4610 (77.3%) employed and 1351 (22.7%) unemployed) (Table 1).


**Table 1.** Sociodemographic characteristics of the active population in Spain, health status, consumption of hypnosedatives, tobacco and alcohol, according to sex.

**Table 1.** *Cont.*



**Table 1.** *Cont.*

<sup>a</sup> *n* = number; % = percentage of total sample; <sup>b</sup> *p* value = sex differences calculated using Chi-square test, with 95% confidence level. DK/DA= Does not know/does not answer.

#### *2.2. Dependent, Independent and Covariant Adjustment Variables*

Nine dichotomised dependent variables were used. The general health status was evaluated on the basis of two questions: (a) "In the last twelve months, would you say your health status has been very good, good, fair, bad, very bad?" This question was dichotomized into 0 = Bad health (fair/bad/very bad) and 1 = Good health (very good/good); and (b) "When was the last time you consulted your general practitioner or family doctor for yourself?" The variable was dichotomized into 0 = No (12 months ago or more/Never) and 1 = Yes (Within the last 4 weeks/Between 4 weeks and 12 months). It is worth mentioning that the self-perceived health variable was dichotomized following common practices in public health studies [73–75]. In addition, we studied the relationship between the self-perceived health variable constructed with twenty-five indicators of pathologies diagnosed by health professionals, finding in all cases a statistically significant relationship that shows how people with good self-perceived health present a lower frequency of being diagnosed with pathologies (Table A1, Appendix A) and, therefore, demonstrate the validity of the constructed variable.

With regard to the state of mental health, three variables were used. Two of them refer to whether the person interviewed suffered from depression in the last 12 months (0 = No; 1 = Yes) or chronic anxiety (0 = No; 1 = Yes). The third variable, corresponding to work stress, was measured through the following question: "Globally and taking into account the conditions in which you carry out your work, indicate how you consider the level of stress of your work according to a scale from 1 (not at all stressful) to 7 (very stressful)". The question was dichotomized by the median which was 4, with 0 = No (from 1 to 4) and 1 = Yes (from 5 to 7). The consumption of hypnosedatives was measured through two questions referring to whether in the last 12 months the person interviewed had consumed tranquilizers, relaxants and/or sleeping pills (0 = No; 1 = Yes), or whether he/she took antidepressants and/or stimulants (0 = No; 1 = Yes). Finally, addictive behaviors were measured through two questions: (a) "Could you tell me if you smoke?"—the question was dichotomized into 0 = No (I don't currently smoke, but have smoked before/I don't smoke or have never smoked regularly) and 1 = Yes (Yes, I smoke daily/I do smoke, but not daily); and (b): "During the past 12 months, how often have you had alcoholic beverages of any kind?"—it was dichotomized by the median; this resulted in 0 = No (Never/No in last 12 months/3 days per month to less than 1 day in a month) and 1 = Yes (Daily or almost daily/6 to 1 days per week).

The independent variables correspond to the main socio-labor characteristics present in the ENSE survey itself. These are: the type of contract or employment situation, the socio-labor category, the working time and the type of working day. It is worth noting that it was not necessary to transform any of the four independent variables, since the ENSE already provided them in an adequate manner to carry out the study. The socio-demographic adjustment variables were age, nationality, marital status, level of education, the income of the family household, type of family life and family care work, following the guidelines of previous studies with similar characteristics [54,56,76]. These variables were selected because they interact predictably with gender roles and can affect men and women differently and act as confounding variables [59–62]. In fact, to avoid selection problems in the female labor force, these studies incorporate family status and care work as adjustment variables, since the reproductive and productive spheres are interconnected [60]. However, in order to verify the presence or absence of selection bias in the female labor force derived from their lower level of participation in the labor market, the Heckman two-stage model was used. The results obtained (Table A2) show that there is no selection bias in any of the nine dependent variables derived from the fact that the correlation coefficients of the error terms of the two equations (Rho) are close to zero and are not significant. Therefore, the likelihood test carried out to verify the null hypothesis of independence between the equations is not rejected. In addition, it can be seen how the coefficients of each variable show how women who are married or live with a partner and in households where there are care jobs have less participation in the labor market. These findings would reinforce the robustness of the results of the present study. Finally, it should be mentioned that the answers "don't know" and "don't respond" were eliminated from the statistical analyses.

#### *2.3. Statistical Analysis*

First, a descriptive analysis of the absolute and relative (%) frequencies of all the variables used was performed, and the differences between men and women were recorded using the chi-square test (*p* < 0.05) (Table 1). Secondly, before stratifying the sample by sex, in order to compare differences between health and consumption indicators between men and women, adjusted odds ratios (aOR) were calculated for all socio-labor and demographic variables and their 90% confidence intervals, using logistic regression models, establishing men as the reference category (Figure 1 and Table 2). Third, once the comparison between both sexes was made, the sample was stratified between the male and female labor force to find associations between socio-labor factors and health and consumption indicators. To this end, as in the previous case, logistical regressions adjusted for all socio-labor and demographic variables were carried out for both the male (Table 3) and female labor forces (Table 4). The regression models were based on the most favorable socio-labor categories (Employment status = Entrepreneur; Socio-labor category = Manager with more than 10 workers; Working time = Full time; Type of workday = Start), following the criteria of favorability used in previous studies with similar characteristics [54]. The goodness of fit of the models was evaluated using the Hosmer–Lemeshow test. The calculations were performed with the SPSS program (version 24) which allows the analysis of complex samples.

**Figure 1.** Logistic regressions between health and consumption indicators by sex. OR: adjusted odds ratio for the four socio-labor variables included in the table (type of contract or employment situation, occupational category, household income, working time and type of working day) and the demographic variables (age, nationality, marital status, level of education, type of family life, family care work, monthly household income) with men as the reference category; IC90%: confidence interval; \* significance level of the *p*-value < 0.10.


*Sustainability* **2020**, *12*, 6425

**Table 3.** Adjusted logistic regression between socio-labor determinants and health and consumption indicators in the male labor

 force.


a b g

Significance

 level value of *p* < 0.01.



a b g

Significance

 level value of *p* < 0.01.

Reference category; c Insu

fficient sample size for analysis; d

Indicators measured only in employed persons; e

Significance

 level value of *p* < 0.1; f

Significance

 level value of *p* < 0.05;

#### **3. Results**

The descriptive analysis showed, on the one hand, that working women presented a worse state of self-perceived health in the last 12 months (24.9%), visited their family doctor more frequently (82.1%), suffered from a higher prevalence of depression (9.5%), chronic anxiety (10.9%), occupational stress (51.8%), and consumed tranquilizers (10.1%) and antidepressants (5.1%) more frequently. On the other hand, the consumption of tobacco and alcohol was higher in men (34.1% and 66.6%, respectively) (Table 1).

The regression models (Figure 1), would confirm the associations found in the descriptive analyses, to the extent that women were 1.47 times more likely to report poor health perceived by themselves than men (aOR = 0.68; IC90%:0.62–0.74) and 1.93 times more likely to visit the family doctor (aOR = 1.93; IC90%:1.77–2.1). In addition, women had a worse mental health status as they were 2.54 times more likely to suffer from depression (aOR = 2.54; IC90%:2.13–3.02), 2.23 and 1.26 times more likely to remit chronic anxiety and stress, respectively, compared to men. A similar situation occurred in the consumption of hypnosedatives, since women were more likely to consume tranquilizers (aOR = 1.51; IC90%:1.29–1.76) and antidepressants (aOR = 2.35; IC90%:1.85–2.99). However, men were 2.63 times more likely to consume alcohol (aOR = 0.38; IC90%: 0.35–0.41).

In order to deepen the analysis of the differences between the male and female labor force, regressions of the nine health and consumption indicators were carried out with the interactions between gender and the type of labor relationship and the occupational social class. The results obtained (Table 2) show that the gender differences found in Figure 1 increase both in work situations and in more vulnerable social classes. Specifically, women working without a contract were found to be 4.03 times more likely to suffer from depression (aOR = 4.03; IC90% = 2.22–7.3), 3.96 times more likely to report chronic anxiety (aOR = 3.96; IC90% = 2.2–7), or 4.01 times more likely to take antidepressants (aOR = 4.01; IC90% = 1.92–8.3) than men working without a contract. Similar situations were identified in unemployed women who were more likely to suffer from depression (aOR = 3.19; IC90% = 2.71–3.2), chronic anxiety (aOR = 3.15; IC90% = 2.69–3.6) and antidepressant use (aOR = 2.95; IC90% = 2.36–3.69) compared to unemployed men. With regard to the social occupational class, the most relevant gender differences were also found in psychosomatic pathologies and the consumption of hypnosedatives among both qualified and unqualified manual technicians, although these associations were less intense.

#### *3.1. Relationships between Socio-Labour Characteristics and Consumption Indicators in the Male Labour Force*

Regression analyses on the male labor force (Table 3) found how unemployment correlated with poorer health and consumption standards, as unemployed workers were 1.75 times more likely to report poorer self-perceived health (aOR = 0.57; IC90%: 0.38–0.89), 5.53 and 5.92 times more likely to suffer from depression and chronic anxiety, respectively (aOR = 5.53; IC90%:2.49–12.26; aOR = 5.92; IC90%: 2.83–12.42, respectively), compared to employers. In addition, unemployed workers were also more likely to use tranquilizers (aOR = 3.09; IC90%: 1.60–5.95), antidepressants (aOR = 6.75; IC90%: 1.91–23.83), and tobacco (aOR = 1.75; IC90%: 1.28–2.41). However, the employers presented greater probability of suffering labor stress with respect to the rest of labor situations, arriving to present 1.56 times larger probabilities of referring to stress than the workers without a contract (aOR = 0.64; IC90%: 0.51–0.79). In addition, employers were more likely to consume alcohol than unemployed workers (aOR = 0.51; IC90%: 0.37–0.71).

In reference to occupational social class, both skilled and unskilled manual technicians were associated with worse health standards (general and mint) and consumption of hypnosedatives compared to managers with more than 10 employees. Nevertheless, the highest differences were found in unskilled manual workers who were 1.75 times more likely to have worse self-perceived health status (aOR = 0.57; CI90%: 0.48–0.69), as well as being more likely to suffer from depression (aOR = 2.39; IC90%: 1.67–3.44), chronic anxiety (aOR = 2.40; IC90%: 1.71–3.42) and to take tranquilizers (aOR = 1.92; IC90%: 1.44–2.56), antidepressants (aOR = 2.30; IC90%: 1.42–3.71) or tobacco (aOR = 2.62; IC90%: 2.19–3.13). However, managers with more than 10 employees were 1.96 times more likely to

report job stress (aOR = 0.51; IC90%: 0.40–0.63) and 1.47 more likely to consume alcohol (aOR = 0.68; IC90%: 0.58–0.80) compared to unskilled manual technicians.

Finally, the most noteworthy results regarding working time were that, on the one hand, part-time workers reported a smaller likelihood of suffering work stress (aOR = 0.60; IC90%: 0.49–0.7) and, on the other hand, that those who worked shifts appeared more likely to report a worse state of self-perceived health (aOR = 0.82; IC90%: 0.52–0.99), visiting their family doctor more (aOR = 1.25; IC90%: 1.07–1.40) and reporting chronic anxiety (aOR = 01.67; IC90%: 1.21–2.3) compared to those who worked split shifts.

#### *3.2. Relationships between Socio-Labour Characteristics and Consumption Indicators in the Female Labour Force*

In reference to the female labor force, the results obtained (Table 4) showed similar findings to those identified in men regarding the labor situation, insofar as unemployed women presented a lower probability of referring to a good state of self-perceived health (aOR = 0.56; IC90%: 0.39–0.79) and a higher probability of suffering from depression (aOR = 2.70; IC90%: 1.59–4.58), chronic anxiety (aOR = 2.00; IC90%: 1.23–3.26) or taking tranquilizers (aOR = 2.29; IC90%: 1.30–4.00) and tobacco (aOR = 1.92; IC90%: 1.03–3.61) with respect to female entrepreneurs. However, the highest probabilities were found in women working without a contract, who were 1.92 times more likely to suffer from depression (aOR = 1.92; IC90%: 1.12–4.1) and 2.60 times more likely to smoke tobacco (aOR = 2.60; IC90%: 1.01–6.70) than female entrepreneurs. Furthermore, coinciding again with the results for the male workforce, associations were observed between female managers and work stress or alcohol consumption, insofar as women working without a contract were less likely to suffer work stress (aOR = 0.62; IC90%: 0.36–0.8) or consume alcohol (aOR = 0.56; IC90%: 0.3–0.98) than female entrepreneurs.

In reference to occupational social class, again, coinciding with men, both qualified and unqualified manual techniques were associated with worse health and consumption of hypnosedatives compared to managers with more than 10 employees. The largest differences were found in unskilled manual workers who were 1.61 more likely to have worse self-perceived health (aOR = 0.62; CI90%: 0.53–0.72), as well as a higher probability of suffering from depression (aOR = 1.82; IC90%: 1.46–2.27) or chronic anxiety (aOR = 1.76; IC90%: 1.41–2.20) and of taking tranquilizers (aOR = 1.52; IC90%: 1.24–1.87), antidepressants (aOR = 1.69; IC90%: 1.28–2.23) or tobacco (aOR = 1.96; IC90%: 1.64–2.33). However, managers with more than 10 employees were 1.39 times more likely to report job stress (aOR = 0.72; IC90%: 0.58–0.89) and 2.00 times more likely to consume alcohol (aOR = 0.50; IC90%: 0.43–0.57) compared to unskilled manual technicians.

Finally, with reference to the working day, unlike the male working population, women working the afternoon shift or irregular days were those who presented the most significant associations with general and mental health indicators. Women working continuous afternoon shifts were 1.56 times more likely to report self-perceived ill health (aOR = 0.64; IC90%: 0.48–0.84), 2.38 times more likely to suffer from depression (aOR = 2.38; IC90%:1.63–3.4) and 1.95 times more likely to suffer from chronic anxiety (aOR = 1.95; IC90%: 1.35–2.8) than women working split shifts. On the other hand, workers with irregular working hours also presented a higher likelihood of reporting depression (aOR = 1.56; IC90%: 1.14–2.1), chronic anxiety (aOR = 1.35; IC90%: 1.01–1.8) and consumption of tranquilizers (aOR = 1.59; IC90%: 1.16–2.1). In terms of working time, part-time workers were more likely to suffer from depression (aOR = 1.50; IC90%: 1.22–1.8) and less likely to report job-related stress (aOR = 0.59; IC90%: 0.40–0.61).

#### **4. Discussion**

The results obtained (Table 1 and Figure 1) confirm some results of previous studies, as the prevalence of poor self-perceived health, mental disorders and hypnosedative use is higher in women [77,78], while alcohol consumption is higher in men [77–79]. Furthermore, as shown in Table 2, the differences between men and women increase in the most unstable employment situations (working without a contract or unemployed) and in the most vulnerable occupational social classes (skilled and

unskilled manual workers) to the extent that the odds ratios identified in these categories are higher both in psychosomatic pathologies and in the consumption of hypnosedatives.

Likewise, it is confirmed for both sexes that unemployment is related to worse self-perceived health, the fact of suffering from depression and the consumption of hypnosedatives [51,54,77]. However, as noted in the introduction, subjective employability could influence as a possible moderator the relationship between job insecurity and negative mental health outcomes. It would be interesting in future studies to have measurement variables of subjective employability to observe their interaction with attributed job insecurity and health and consumption outcomes.

Continuing with the analysis, it is worth mentioning that the impact of unemployment is greater among the male labor force for several reasons. First, because unemployed workers are more than twice as likely to suffer from depression as employed women. Second, while the unemployed have had a high prevalence of chronic anxiety and antidepressant use compared to actively working men, no differences in antidepressant use have been found between currently working and unemployed women, and the relationships identified for chronic anxiety are much smaller than those of men. This situation could be explained by the division of family roles and responsibilities between men and women, as previous studies have shown [60,80]. However, these hypotheses merit specific analysis in future studies, rather than the multidimensional analysis sought in this research, since, while there are important differences between the probabilities of the female and male workforces, when the sample is stratified by sex, they are no longer comparable.

Previous research has found that temporary workers and those with job instability make less frequent visits to the family doctor [55] and have a higher prevalence of mental disorders [53]. However, in our study we found that self-employed workers are the least likely to make medical visits and the most likely to suffer from depression and chronic anxiety. Despite the divergences between the results and the complexity of the relationships, there may be a pattern derived from the stronger perception of distress among precarious workers when they perceive high job insecurity [53], which may lead them not to absent themselves from work for fear of being fired and, consequently, not attending the doctor and opting to self-medicate. In fact, the Sixth European Survey on Working Conditions 2015 [81], identified that 44% of workers with permanent contracts declared that they had worked while sick during the last year, while among self-employed workers the rate was 50%, which could confirm that the productive need makes the self-employed worker not absent from work, even if he is sick. On the other hand, a study conducted in public hospitals identified that professionals with temporary employment contracts were more likely to self-medicate [82], which would explain why people who feel a high degree of job insecurity, whether as self-employed or temporary workers, tend not to be absent from work when they have health problems, and opt for self-medication. However, these hypotheses should be evaluated in future research. We could also consider that the precarious working conditions to which temporary workers are subjected may mean that they do not have sufficient financial resources to take out private health insurance and therefore go to the doctor less often. However, Spain has a universal health system, so this hypothesis for the Spanish case would be ruled out.

Since the aim of this research is to explore and describe, as a whole, the associations between factors of job insecurity and the different health and consumption indicators, we can observe different relationship patterns, depending on the work situation and the occupational social class. On the one hand, it has been identified that unemployed people, who belong to the most vulnerable social classes, suffer more frequently depression and chronic anxiety. These mental disorders, in turn, are associated, as shown by previous studies, with the increased consumption of hypnosedatives [58] and tobacco [83]. This would explain, to a certain extent, the patterns and associations of social vulnerability with mental disorders, consumption of hypnosedatives and tobacco obtained in our study.

The occupational classes with the highest status in Tables 3 and 4, on the other hand, have reported greater stress than manual occupational classes, and both male and female managers with more than 10 employed people have also reported greater job stress, which would refute the findings identified in previous studies [84]. The greater occupational stress of these groups could be derived from the intensification of work in the most qualified "knowledge" jobs as they are more intensely exposed than manual workers to psychosocial risk factors such as emergencies to perform tasks, time pressure, speed or short term in the execution of work, role dysfunction, self-management, etc. [85–90]. The fact that the occupational classes with the highest status are also associated with the highest alcohol consumption would in turn confirm other previous findings [54]. There are two hypotheses that could explain the higher alcohol consumption in the better-positioned occupational social classes. The first is that differences in consumption across classes are explained by cultural patterns and by reduced access to such substances by blue-collar workers [54]. The second hypothesis is that higher consumption of alcohol by these groups is associated with a greater need to combat stress [57]. Both hypotheses could explain the relationships found in this study between occupational social classes with higher status, work stress and alcohol consumption.

We can also see the influence of the relationship between professional situation and social class on working time. On the one hand, if we consider (albeit with certain nuances) that part-time work is part of precarious employment [74], the results obtained show that this partiality is associated with a higher prevalence of depression in women. On the other hand, the results show that full-time work is associated with greater job stress in both sexes. Previous studies record similar results, insofar as this research has associated a higher number of working hours with higher occupational stress [55].

Finally, with regard to the type of working day, the results obtained show significant differences between the sexes. Although few associations have been identified in men, with shift work being the most damaging to general and mental health, multiple associations have been identified in women. In particular, it should be noted that the continuous afternoon shift is the one with the highest prevalence of depression and chronic anxiety, while the irregular shift is also associated with the highest probability of suffering from depression, chronic anxiety and the use of tranquilizers. These results differ from those of previous studies associating mental disorders with shift work [56,91], and further research is needed to improve the understanding of this relationship. However, it was observed that while the work situation or occupational social class affected men more, the type of working day affected women more. This could again be explained by the division of gender roles, which implies a greater workload for women in the family setting [60,80].

#### *Limitations*

The study presents some common limitations of the use of this type of survey. Firstly, the most important limitation is that there may be a risk of reverse causality, and therefore the findings identified should be considered as associations rather than causal relationships. This is a common limitation in this type of study [56–61]. Secondly, except for Figure 1 and Table 2, in which differences between men and women can be compared, the results obtained from the separate regressions for the male and female labor force (Tables 3 and 4), as explained in the discussion, do not have comparable parameters, since the variance-covariance matrix is calculated separately. This situation would also occur in work of a similar nature [56–61]. However, it should be remembered that the objective of the research is to look for patterns that will allow more concrete comparative analyses to be carried out in future studies. Thirdly, there could be information and response biases of complacency on the part of participants, or of responding to what is considered socially acceptable. In this sense, more favored social classes and men, associated with stronger and more powerful roles, may be unwilling to acknowledge certain health problems because of social stigma. This attitude may result in an underrecording of mental pathologies or substance use. In fact, this situation could explain the low number of affirmative responses about mental disorders (depression or chronic anxiety) or the high number of unanswered cases about the use of hypnosedatives. On the other hand, there could be selection bias, for example, in the most vulnerable

occupational classes due to the possibility of dropouts, or the increase in the number of unanswered questions. All this may favor the underregistration of pathologies and consumption. In addition, the underrepresentation of some categories (e.g., non-contract work, shift work, night work) prevents some more comprehensive stratified analyses. In the future it might be interesting to stratify the analysis by sex and age simultaneously or by a more disaggregated occupational social class, but the sample size would only allow a subset of analysis in those cases, leading to a reduction of possible analyses. The impossibility of having socio-occupational variables (e.g., number of working hours, production rates, social support) can also act as a confounder. It would therefore be useful to include them in future editions of the ENSE survey. Finally, it is worth mentioning that the associations found cannot be evaluated as "causal", since this is a transversal study.

#### **5. Conclusions**

In conclusion, we believe that the study is relevant, since the exploration of health and consumption patterns can serve as a reference for the planning of sustainable preventive occupational health policies, both in labor and health institutions and in companies. These programs should focus, at least, on the unemployed, those who belong to the most vulnerable occupational social classes and considering the gender differences described. Specifically, two patterns of health erosion have been identified as a result of high rates of job insecurity. On the one hand, the most vulnerable people (unemployed and manual workers) suffer with a higher prevalence of depression, chronic anxiety, hypnosedative use and tobacco consumption, and therefore active employment policies should be promoted to reduce the high unemployment rates that still exist in Spain. On the other hand, more qualified people and, above all, managers have reported greater work stress and alcohol consumption. The problem of these groups does not lie so much in sustaining employment, but rather in the deregulation of working conditions which has led to an increase in the intensification of employment, which is a determining factor in the increase of work-related stress and alcohol consumption. To all this, we should add another series of policies to reconcile work and family life (for example, reducing working hours and establishing schedules compatible with reproductive tasks), since, as we have seen, it affects working women in particular in a negative way. It seems reasonable, therefore, to call for the revival of social dialogue for the implementation of sustainable measures to improve the quantity and quality of employment, since neoliberal policies for the management of the economic crisis have caused a serious public health problem. However, it should be remembered that this study is of an exploratory nature, and therefore, rather than directly suggesting courses of action, it highlights the need to increase research into labor relations and occupational health, and then, on that basis, to implement specific labor and health policies.

**Author Contributions:** Conceptualization, R.P.C. and P.J.B.C.; methodology, R.P.C.; formal analysis, R.P.C.; research, R.P.C.; curatorship of data, R.P.C.; preparation of the original draft of the manuscript, R.P.C.; review and editing of the manuscript, P.J.B.C.; supervision, P.J.B.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Spanish Ministry of Education, Culture and Sports, grant number FPU2016/04591.

**Acknowledgments:** The authors would like to thank the Ministry of Health, Consumer Affairs and Social Welfare for providing the microdata from the National Health Survey in Spain, 2017.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**



<sup>a</sup> *n* = Number; % = Percentage over the total sample; <sup>b</sup> *p* value = Sex differences calculated using a chi-squared test, with 95% confidence level.


 

#### *Sustainability* **2020**, *12*, 6425

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **COVID-19: A Relook at Healthcare Systems and Aged Populations**

**Thanh-Long Giang 1,2,\*, Dinh-Tri Vo 2,3 and Quan-Hoang Vuong 4,5,\***


Received: 21 April 2020; Accepted: 16 May 2020; Published: 20 May 2020

**Abstract:** Using data from the WHO's Situation Report on the COVID-19 pandemic from 21 January 2020 to 30 March 2020 along with other health, demographic, and macroeconomic indicators from the WHO's Application Programming Interface and the World Bank's Development Indicators, this paper explores the death rates of infected persons and their possible associated factors. Through the panel analysis, we found consistent results that healthcare system conditions, particularly the number of hospital beds and medical staff, have played extremely important roles in reducing death rates of COVID-19 infected persons. In addition, both the mortality rates due to different non-communicable diseases (NCDs) and rate of people aged 65 and over were significantly related to the death rates. We also found that controlling international and domestic travelling by air along with increasingly popular anti-COVID-19 actions (i.e., quarantine and social distancing) would help reduce the death rates in all countries. We conducted tests for robustness and found that the Driscoll and Kraay (1998) method was the most suitable estimator with a finite sample, which helped confirm the robustness of our estimations. Based on the findings, we suggest that preparedness of healthcare systems for aged populations need more attentions from the public and politicians, regardless of income level, when facing COVID-19-like pandemics.

**Keywords:** COVID-19; healthcare systems; aged populations

#### **1. Introduction**

The rapid COVID-19 outbreak since late February 2020 has posed critical challenges for public health, politics, and medical communities [1,2]. Although old lessons (such as quarantine, isolation, social distancing, and travel restrictions) are still helpful, the roles of hospital beds, medical staff (i.e., nurses and physicians) and aging population on the severity of this pandemic has not yet been studied systematically.

Is the number of deaths related to COVID-19 the consequence of overwhelmed healthcare systems and aging populations? In Europe and USA, the healthcare systems have been restructured toward centralization and budget cutoff. Aged populations are a clear evidence of this in these countries. Since the outbreak of COVID-19, several studies found that the fatality rate has been significantly higher with an increasing profile of age [3,4]. Furthermore, concerns about the healthcare systems in such countries as Italy, Spain, France, UK, and USA currently have been a hot topic on public media. The importance of the healthcare systems has been emphasized by [5,6] and [7].

Given these concerns, this study aims to examine the factors associated with the death rates of the COVID-19 infected people, in which we emphasize healthcare systems and aged populations along with other covariates. In the next section, we present a literature review on healthcare systems, aged populations, and important factors supposed to have direct correlations with the death rate from COVID-19 and some previous epidemics. We then introduce data, research methods, and discussed empirical results. Finally, we conclude and share our perspectives on healthcare systems and aged populations.

#### **2. Literature Review**

#### *2.1. Health Systems and Pandemics*

Discussing about the roles of human resources and healthcare infrastructure, [7] argued that staffing and supplies should be critically and carefully planned because COVID-19 patients should be discharged only to designated facilities or to those already caring for such patients. Practically, however, it might be that non-institutional care systems (such as home-based) were not capable to deal with a large number of discharged patients. In addition, since healthcare workers and supplies were critically important in mitigating the outbreak, it would be also crucial to prepare supplies protecting health workers who work with infected patients, and this in turn would help reduce infection and death rates.

Reviewing the history of pandemics in 1918, 1957, and 1968 [8] showed that, in recent flu seasons, hospital emergency departments faced limits in emergency rooms and inpatient beds when the number of patients increased substantially. For the US healthcare system in pandemic, one of the most concerning issue was human resources at institutional care facilities because home-care and community-care settings did not have enough experienced nurses and managers when facing a surge of patients at communities. Healthcare workers are extremely important for fighting outbreak. In pandemics with an increasing number of patients, hospital intensive care unit (ICU) beds and ventilators would not be useful if there are inadequate numbers and types of healthcare personnel.

Also discussed the US healthcare system, [9] argued that only 15 states could be able to respond fully to emergency, while others would run out of beds or face a shortage of nurses in similar situations. More critically at national level, if the country faces a 1918-like pandemic, hospital beds would increase about twice and patients in the intensive care unit (ICU) would increase about 4.6 times. Staff shortages would exacerbate the pandemic situation because it was also possible that some healthcare workers might expose themselves to infectious patients. At the same time, facing drained resources, healthcare workers would have to make important and difficult decisions about allocating limited resources while prioritizing and triaging patients.

Developing computational models with data collected from the 2014–2015 Ebola outbreak in Guinea, Liberia, and Sierra Leone, [10] estimated the repercussions of the outbreak on the populations at risk for three diseases (malaria, HIV/AIDS, and tuberculosis). They showed that accessibility to healthcare services is important to reduce the number of deaths. The simulated results indicated that if there was a 50% reduction in access to healthcare services, the Ebola outbreak would have exacerbated malaria, HIV/AIDS, and tuberculosis mortality rates by additional death counts of 6269 in Guinea, 1535 in Liberia, and 2819 in Sierra Leone.

Using observations from various data sources and reports, [11] reviewed how countries responded to COVID-19 by combining containment and mitigation activities along with delaying major surges of patients and levelling the demand for hospital beds. This proposition was also supported by [5,6]. The success of South Korea in controlling the COVID-19 with high detection rate, which required the readiness of healthcare systems, should be a guiding reference [12]. This view was also supported by [13].

Health care workers and supplies would be critically essential in mitigating the outbreak. Preparing supplies, such as N95 respirator masks and other personal protective equipment, is important to protect health workers while working with infected patients. This in turn would help reduce

infection and death rates. More importantly, the emergency need of Intensive Care Units (ICU) could collapse the healthcare system [14].

#### *2.2. Aged Population, Health Conditions and Fatality in COVID-19*

Using demographic and health-related data of 191 COVID-19 adult inpatients (aged 18 and over) from Jinyintan Hospital and Wuhan Pulmonary Hospital in Wuhan, of which 137 were discharged and 54 died in hospital by 31 January 2020, [4] explored risk factors associated with in-hospital death. They found that 91 patients (48% of the studied sample) had a comorbidity, in which hypertension was the most common, and then diabetes and coronary heart disease. Multivariable regression showed increasing odds of in-hospital death associated with older age, higher Sequential Organ Failure Assessment (SOFA) score, and d-dimer greater than 1 g/mL on admission. The authors concluded that those risk factors could help clinicians to identify patients with poor prognosis at an early stage.

Similarly, extracting data and analyses from other studies, [11] emphasized that older people (particularly those aged 80 and over) and people with comorbidities (such as cardiac disease, respiratory disease, and diabetes) were at the highest risk of serious disease and death. As shown in the US, the authors were concerned that individuals in aged care facilities were at particular risk of serious disease when the healthcare system faced a surge in COVID-19 patients.

Exploring data from 13 January to 12 February 2020 in China, [15] analyzed data on 799 patients with confirmed COVID-19 who were transferred to or admitted in Tongji Hospital. As of 28 February 2020, 113 of the 799 patients died (a mortality rate of 14.1%) and 161 patients recovered and were discharged. The statistics showed that the median age of deceased patients was 68, which was significantly older than that of recovered patients, with a median age of 51. Of these patients, 71 persons (or 63% of patients who died) and 62 persons (or 39% of patients who recovered) had at least one chronic medical condition. Among deceased patients, hypertension, cardiovascular disease, and cerebrovascular disease were much more frequent than the other diseases.

Doing similar research with data from 138 patients with confirmed COVID-19 hospitalized at Zhongnan Hospital from 1 January to 28 January 2020 and followed-up by 3 February 2020, [16] described epidemiological and clinical characteristics of those patients. The median age was 56 years, and 54.3% were men. A total of 36 patients (26.1%) were transferred to the ICU because of complications, including acute respiratory distress syndrome, arrhythmia, and shock. Compared with patients not treated in ICU, those treated in ICU were older (median age 51 for the former vs 66 for the latter) and were more likely to have underlying comorbidities (72.2% vs 37.3%). Such a medical situation suggests that age and comorbidity might be risk factors for poor outcome. There was no difference in the proportion of men and women between ICU patients and non-ICU patients.

#### *2.3. Travelling and Other Control Measures in COVID-19*

To estimate COVID-19 outbreak size in Italy, [17] used data on non-residential travelers and their average length of stay with an assumption that the epidemic began in late January 2020. They found that the COVID-19 case exportations from Italy were larger than the official case counts.

For the case of China, [18] showed that, up to mid-January 2020, more than 95% of the daily exposing risk of CoV-19 was due to international travel. The authors also showed that the travel restrictions decreased the daily rate of exportation.

With data from 28 countries which imported COVID-19 cases, [19] argued that travel restrictions were not effective enough to prevent the global spread of COVID-19 in most airports. Their study highlighted the need to strengthen local capacities for disease monitoring and control rather than controlling the importation of COVID-19 at national borders via the airline network. Similarly, [20] argued that a lock-down along with nationwide traffic restrictions and a stay-at-home movement had a determining effect on the spread of COVID-19.

#### **3. Study Data and Methods**

#### *3.1. Data*

We manually downloaded the situation reports from the World Health Organization (WHO) from Report no.1 (21 January 2020) to Report no.70 (30 March 2020). With the extracted data, we then combined them with data from the World Bank's Development Indicators and the WHO's Application Programming Interface (API) for the selected variables. Due to the availability from data source, we took the value of the most recent year. The description of variables is presented in Table 1.


In the following step, we computed the death rate from each report and selected a sample of countries that had more than 100 confirmed cases (so we had 95 countries in the studied sample). The final panel data set consisted of 70 points of observation, in which the least minimum country-time observation of variable was 3447 (Table 2). In this table, n is the number of country-report observations; other values are at country level (such as the highest cases of 122,653 was of the US at the 70th report). At the date of the 70th report, the country with the highest confirmed number of cases was the US with 122,653 cases, and the country with the highest number of deaths was Italy with 10,781 cases. The average death rate for the whole sample was 1.44%.


**Table 2.** Descriptive statistics.

Note: HR: sum of physicians (per 1000 citizens) and nurses and midwives (per 1000 citizens); DoC: probability (%) of dying between age 30 and exact age 70 from any of cardiovascular disease, cancer, diabetes, or chronic respiratory disease. Source: Authors' calculations from the collected data.

On average, countries in the sample had 38.38 hospital beds per 1000 citizens and 8.68 medical staff (including nurses and physicians) per 1000 citizens. The average proportion of people aged 65 and over for the whole studied sample was 12.77%.

#### *3.2. Methods*

We first estimated three models with pooled estimator. Then, we compared our interested model by employing pooled, fixed-effects (FE) and between-estimator methods.

As different countries at various income levels have different healthcare systems and aged populations, for further analysis, we divided the sample into two sub-samples according to the income classification by [21]: high income countries (HICs) and middle- & low-income countries (MLICs).

We also employed Pesaran's cross section dependence (CD) test to detect cross-sectional dependence. As the results suggested the possibility of the problem, we applied Robust Covariance Matrix Estimators to check the standard errors. With the properties of a finite sample, the method provided in [22] was the most suitable estimator, compared with the White method [23].

Finally, we compared the results for the pooled estimation without and with Robust Standard Errors.

#### **4. Empirical Results**

#### *4.1. Main Results*

In Table 3, we present the results obtained from different estimations. In all models, four variables (hospital beds, human resources (HR), death due to non-communicable diseases (DoC), and population 65) showed their consistent impacts on the death rate of COVID-19 infected persons. In regard to health systems, variables "hospital beds" and "HR" implied that the better the healthcare infrastructure and human resources, the lower the death rate. Such a situation is clearly illustrated in the case of Italy, Spain, and the US during the studied period, as reported in [24].


**Table 3.** Pooled estimations with baseline and extended models.

Note: \* *p* < 0.1; \*\* *p* < 0.05; \*\*\* *p* < 0.01. HR: sum of physicians (per 1000 citizens) and nurses and midwives (per 1000 citizens); DoC: probability (%) of dying between age 30 and exact age 70 from any of cardiovascular disease, cancer, diabetes, or chronic respiratory disease. Source: Own calculations.

Factors representing demographic ("population 65") and health ("DoC") aspects also reflected the real situations: older people accounted for the majority of deaths, and most of those had various comorbidities, particularly non-communicable diseases (such as cancer, diabetes, or chronic respiratory disease) [15].

When adding logarithm of real GDP per capita (Model 2 in Table 3), its coefficient was positive and statistically significant, meaning that, given other demographic and healthcare system conditions, higher income countries experienced higher death rates than did those at lower income levels. This has been true in practice where both the number of deaths and the death rates in such high-income countries as the US, UK, Italy, and Spain were much higher than those of lower income countries.

One of the key channels for spreading out the COVID-19 pandemic has been domestic and international travels. The results from Model 3 with an addition of the variable showing the rate of passengers carried by air indicated that the countries which had a higher rate of passengers experienced higher rate of deaths. This finding reflected the real situation that the virus is transmitted from human to human, and the countries where a lot of people moving in and out for various purposes like businesses, travelling, and visiting, like Italy, Spain, the US, the UK, and China (especially in the spring holiday season), had an increasing number of people infected during the studied period. The recent social distancing and isolation in many countries and their regions have proved that less travelling and movement helped reduce the infection rates, and thus—to some extent—reduced the death rates [15,20].

A new approach in this paper was to apply the between-estimator estimation in order to explore the relationships of the model. In recent years, this method has been a new edge in evaluating the long-run effects of macroeconomic factors (see, for instance, [25–28]). This methodology, using the time-averaged data, was suitable with the dataset of this study since all independent variables were collapsed at one time.

Table 4 presents the results from pooled, fixed-effects (FE) and between-estimator methods. The results clearly showed the confirmation on the sign and significance of variables "hospital beds", "HR" and "population 65". More importantly, the R2 was at 0.599, meaning that the model was better than the other. It is worth noting, however, that we could not apply this estimator when splitting the sample for further analyses since we had limited observations.


**Table 4.** Different estimation methods.

Note: \* *p* < 0.1; \*\* *p* < 0.05; \*\*\* *p* < 0.01. HR: sum of physicians (per 1000 citizens) and nurses and midwives (per 1000 citizens); DoC: probability (%) of dying between age 30 and exact age 70 from any of cardiovascular disease, cancer, diabetes, or chronic respiratory disease. Source: Own calculations.

#### *4.2. Further Results*

Since countries at different income levels have different healthcare systems and strategies to deal with COVID-19 pandemic, we were interested in exploring how the aforementioned factors influenced the death rates of the COVID-19 infected persons in those different countries. Table 5 shows the results estimated for two groups of countries by income levels: high-income countries (HICs) and middle- & low-income countries (MLICs).

These results were different from those in the FE models for all countries as presented in Table 4. The negative coefficient for HICs was kept, meaning that healthcare system infrastructure was important to reduce the death rates of COVID-19 infected people in these countries.

Regardless of income levels, the coefficients for variable "HR" were negative and statistically significant in both groups of countries, meaning that the number of medical staff available in the pandemic has been extremely important for reducing death rates.

Except for HI countries, the coefficient of variable "DoC" for MLICs was positive and statistically significant, and this could be explained with the same reason discussed in the FE models in Table 4.

The coefficient for variable "Population 65" was positive and statistically significant in HICs, while it was not the case for MLICs. Such results could be elucidated the same as in FE models in Table 4. HICs had a higher rate of older people, who have been at highest risk of death under COVID-19.

In both groups of countries, coefficients for variable "real GDP per capita" were positive and statistically significant, meaning that higher income countries had higher death rates than those at lower income levels in the same group. This could be explained by various facts, including a higher proportion of people aged 65 and over and a higher number of air passengers—among others—in higher income countries.



Note: \* *p* < 0.1; \*\* *p* < 0.05; \*\*\* *p* < 0.01. HR: sum of physicians (per 1000 citizens) and nurses and midwives (per 1000 citizens); DoC: probability (%) of dying between age 30 and exact age 70 from any of cardiovascular disease, cancer, diabetes, or chronic respiratory disease. Source: Own calculations.

For the variable showing travelling impact (i.e., "Air passengers"), the coefficient for HICs was positive and statistically significant, showing the fact that these countries experienced huge flows of immigrants and emigrants during the studied period, and thus have experienced more infected people and more deaths. In contrast, MLICs started quarantine at the early stage of COVID-19 spreading so that they could limit the number of infected people via international and domestic travelling flows.

To check the robustness for all above estimations, we conducted different methods to see whether the standard errors of the same variables were significantly different. The results are presented in Table 6. We could see clearly a small difference in the standard errors of the same variables between estimations. These results confirmed that the main results in Table 4 were robust; that is, the correlations between the death rate and important explanatory variables (such as number of hospital beds, number of medical staffs, DoC, aged population, and air passengers) were significant strongly. Furthermore, there was only one difference in the significance but not the sign of the variable "real GDP per capita". This implied that we could not confirm strongly the correlation between real GDP per capital and the death rate. Meanwhile, among countries with different income levels, the heterogeneity did exist in some variables.


Note: \* *p* < 0.1; \*\* *p* < 0.05; \*\*\* *p* < 0.01. HR: sum of physicians (per 1000 citizens) and nurses and midwives (per 1000 citizens); DoC: probability (%) of dying between age 30 and exact age 70 from any of cardiovascular disease, cancer, diabetes, or chronic respiratory disease. Source: Own calculations.

#### **5. Concluding Remarks**

In this paper, we used the daily statistics on the death rates of the COVID-19 infected people in various countries which had more than 100 infected cases from 21 January 2020 to 30 March 2020, and explored their possible associated factors. Although the results were different when we controlled for various factors (such as income levels), we still found consistent results that healthcare system conditions, particularly the number of hospital beds and the number of medical staff, played extremely important roles in reducing death rates of COVID-19 infected persons. In addition, both mortality rates due to different non-communicable diseases (NCDs) and the rate of people aged 65 and over were significantly related to the death rates in all countries, meaning that aged populations along with prevalent NCDs would exacerbate the situation of death under any pandemics related to pneumonia like COVID-19. We also found that controlling international and domestic travelling by air along with increasingly popular anti-COVID-19 actions (i.e., quarantine and social distancing) helped reduce the death rates in all countries. Last but not least, the danger of COVID-19 has made clear that the preparedness of healthcare systems and aged populations needs more attention from public and politicians, regardless of income level, when facing COVID-19-like pandemics. In any country, timely and strong cooperation between government, civil society, and private individuals are important in building up the trust in fighting public health crisis like COVID-19 [29].

Given the nature of global research with cross-sectional data, this study could not avoid some key limitations, as follows. First, we could not disaggregate the data on death by sex and age groups since the used statistics did not cover these important indicators for all countries in the studied sample. Second, some non-health and non-demographic factors such as culture and living styles could not explore due to unavailable data; those could be studied at a country-specific level. Third, due to limited timeline for the study (up to 30 March 2020), the impact of various measures coping with COVID-19 such as social distancing and lock-down could not be explored. We hope to integrate these factors in the coming studies with a specific group of countries or a single country.

**Author Contributions:** T.-L.G. checked data analysis, drafting, editing and finalizing the article. D.-T.V. was in charge of data collection and calculations as well as drafting the article. Q.-H.V. provided comments to revise drafts. The final version was completed and agreed by all authors. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*
