Australian College of Rural and Remote Medicine.



FACRRM: Fellowship of the Australian College of Rural and Remote Medicine; FARGP: Fellowship in Advanced Rural General Practice; RACGP: Royal Australian College of GeneralPractitioners; ACRRM: Australian College of Rural and Remote Medicine.

the

### **2. Materials and Methods**

This study used 2008–2017 data (waves 1–10) from the "Medicine in Australia: Balancing Employment and Life (MABEL)" study. MABEL collected annual cross-sectional survey data from a national panel of doctors across all career stages. It commenced in 2008, with 10,498 doctors (19% of the sampling frame, minimal participation bias) completing the initial survey (wave 1) [39]. There has subsequently been an annual 70–80% study retention rate, with new doctors topping up the sampling frame. MABEL was approved by the University of Melbourne Faculty of Business and Economics Human Ethics Advisory Group (Ref. 0709559) and the Monash University Standing Committee on Ethics in Research Involving Humans (Ref. CF07/1102-2007000291).

This study only included data from clinically active GPs and excluded those currently enrolled in vocational training (equivalent to 'residency') programs. Qualification data were self-reported across all 10 waves, responding to "What GP and other specialist postgraduate qualifications have you obtained in Australia? (e.g., FRACGP, FRACP, FACRRM, diploma)" and "Please list any GP and other medical qualifications you have obtained in Australia since the last time you completed the MABEL Survey". Doctors were categorized into qualification categories, as described in the analysis below.

Geographic distribution of the main work location was the primary outcome. Rurality was defined using the Department of Health's Modified Monash Model (MMM) classification as metropolitan (MMM-1) rural (MMM 2–7) [40]. Some analyses further collapsed the rural category into MMM 2–3 (large rural/regional, >15,000 population) or MMM 4–7 (smaller rural <15,000 population or remote/frontier areas). Distribution was additionally explored by the state, due to the potential for state-based variation from both geography and state-based rural generalist support. Other key demographic factors were gender, age (<50, 50+), childhood background (at least 6 childhood years in a rural area), and place of basic medical training (Australian or New Zealand medical graduate (AMG), or overseas trained doctor (OTD)).

Measures of scope of practice were firstly defined by advanced skills area, whereby all doctors reported doing "specialized training of at least 6 months which is outside the normal scope of practice for GPs". Four groups were defined (see skills listed in Table 1): (i) practicing at least one additional skill; (ii) practicing one of the four procedural skills; (iii) having trained in an additional skill area, but not currently practicing it; and (iv) having trained in a procedural skill, but not currently practicing it. The latter two categories aimed to identify skill maintenance. These scope data were only available in Wave 10 (2017) of the MABEL survey. Secondly, scope was defined by a series of other indicators: work in a hospital, work on-call, total hours worked, direct patient hours, hours worked in community settings, and two self-nominated measures of practice complexity.

### *Analysis*

Descriptive statistics were used to analyze longitudinal outcomes of geographic distributional and scope of practice for (i) wave 1 (2008), (ii) wave 6 (2013), and (iii) wave 10 (2017). Due to multiple fellowships, some doctors were counted in more than one category. Notably, all FARGPs also had a FRACGP qualification (a pre-set requirement), 5–10% of FACRRMs also had a FRACGP, while 25–35% of FARGPs also had a FACRRM. Secondary analyses limited respondents to only those who graduated from medical school after 1995, as a proxy for the cohort doing general practice training in the period of both the ACRRM and GradDip RurGP programs emerging, thus largely removing those awarded via full RPL. The discrete qualification categories were those (i) having a FACRRM; (ii) having a FARGP or GradDip RurGP (henceforth merged as 'FARGP'); (iii) having a FRACGP and not having (i) or (ii); (iv) GPs not reporting any related qualification. Multiple logistic regression models were used to measure associations between these fellowships, other key characteristics, and the main geographic distribution outcomes. Sampling weights were used to adjust for survey non-response bias of key demographics. All analyses used Stata SE 15.1 for Windows (Stata Corp, College Station, TX, USA) and statistical significance was *p* < 0.05.

### **3. Results**

In waves 1, 6, and 10 there were respectively 3930, 2936, and 3185 clinically active GPs who completed the MABEL survey. On average, in each wave there were 274 (8%) and 63 (2%) who, respectively, indicated they had either the FACRRM and/or FARGP qualifications.

FACRRMs were 75–83% male, compared with 50–65% for all other qualification groups (Table 3). Both FACRRMS and FARGPs were more likely to have a rural background (32–38%) than the other qualification groups (18–21%), but less likely to have been trained overseas (8–15%, compared with 22–31%). Reflective of their large RPL process, most FACRRMs were aged 50+. In contrast, most FARGPs were aged <50.

FACRRMs were mostly working in rural areas (75–84%) and approximately half were in the smaller communities (41–54%) (Table 4). Those with FARGPs were also mostly working in rural communities (56–67%), though proportionally fewer were in the smaller rural communities (25–41%). Around 50–60% of both FACRRMs and FARGPs were working in either Queensland or New South Wales, reflecting the largest rural populations. Amongst recent graduates, FACRRMs were moderately biased to working in Queensland (48%).

Both FACRRMs (26–31%) and FARGPs (29–34%) were more likely to be using advanced skills in their job, compared with those without these qualifications (14–26%) (Table 5). This was more pronounced for the four main procedural skills. However, FACRRMs and FARGPs were also more likely to have an advanced skill but not use it (13–26% vs 9–16%). Recent graduate FACRRMs (>1995) were more strongly related to maintaining their advanced skills than recent FARGP graduates. FACRRMs were most likely to work in a hospital setting and do on-call. FARGPs worked the longest hours per week, though both FARGPs and FACRRMs worked longer per week in other community settings. FACRRMs and FARGPs reported using less consultation support for complex patients, which is possibly reflective of their geographic distribution. FACRRMs reported mostly seeing patients with complex problems.

After adjusting for covariates (Table 6), FACRRMs were substantially more likely to be working in a rural area compared with those with standard qualifications (OR 8.7, 5.8–13.1), including when limited to graduates > 1995 (OR 9.6, 3.4–27.0). FACRRMs working rurally were significantly more likely to be working in smaller rural communities (OR 3.5, 2.3–5.3). FARGPs were also significantly more likely to work rurally (OR 4.2, 2.2–7.8). However, rural FARGPs were not more likely than those with standard qualifications to work in smaller rural communities (OR 1.1, 0.5–2.5).



 two categories (FACRRM FRACGP, or FARGP): 65; 6 97; 10 88; (or New Zealand) Graduate; Overseas Trained Doctor; FACRRM: Fellowship of the Australian College of Rural and Remote Medicine; FARGP: Fellowship in Advanced Rural General Practice; FRACGP: Fellowship of the Royal Australian College of General Practitioners.



Rural and Remote Medicine; FARGP: Fellowship in Advanced Rural General Practice; FRACGP: Fellowship of the Royal Australian College of General Practitioners.



Those with multiple fellowships were counted in each respective category; Aggregate of Community health center, Residential/aged care facility, Aboriginal health service; "I normally consult with others in the practice about the management of patients with complex health and social problems"—% agree or strongly agree; 2 "The majority of my patients have complex health and social problems"—% agree or strongly agree; FACRRM: Fellowship of the Australian College of Rural and Remote Medicine; FARGP: Fellowship in Advanced Rural General Practice; FRACGP: Fellowship of the Royal Australian College of General Practitioners.



\* model (coefficients are not shown as they largely reflect the population dispersion across Australia's states); AMG: Australian (or New Zealand) Medical Graduate; OTD: Overseas Trained Doctor who gained basic medical qualifications an another country; FACRRM: Fellowship of the Australian College of Rural and Remote Medicine; FARGP: Fellowship in Advanced RuralGeneralPractice;FRACGP:FellowshipoftheRoyalAustralianCollegeofGeneralPractitioners.

### **4. Discussion**

This paper presents the first empirical evidence about the characteristics and geographic distribution of doctors related to rural general practice faculties compared with GPs who are not members of these faculties. It identifies GPs associated with both FACRRM and FARGP compared with GPs of standard qualifications. None significantly improved rural distribution and expanded the scope of practice. Though the faculties are structured in different ways and function relatively independently of each other, each faculty relates to members who work in rural communities at a broader scope of practice, with improved geographic distribution than those GPs who are not such faculty members.

A key finding is that the stand-alone faculty that has a specific rural mission and delivers wholly rural training (FACRRM), relates to doctors of better distribution into smaller rural and isolated communities, as well as doctors who sustain practice of their advanced skills (working in areas like obstetrics atop of general practice, as rural generalists). These findings demonstrate the value of rural faculties as a professional hub for rural doctors, enabling rural-tailored training and professional support, as a critical strategy for growing and sustaining a skilled and geographically distributed primary care workforce.

These data additionally provide a strong reminder that GPs associated with rural faculties remain a small minority of the trained general practice workforce, around 10% relative to 29% of Australians living rurally, and the 13% of Australians living in smaller rural and isolated communities (where rural generalist doctors are most indicated to be required). GPs working and living in large regional centers may not require specific professional training for their practice, and often have similar professional and personal experiences to colleagues in metropolitan areas [41]. However, strong growth of rural faculties might assist to address growing the skilled rural generalist workforce that is sorely needed in smaller rural towns. Further, most of the ACRRM fellows are older than 50 and will require replacement within 15–20 years. Currently, of around 1500 new general practice vocational training enrolments each year across Australia, there are approximately 150 FACRRM (10%) enrolments annually and around 85 FARGPs (6%). A February 2020 government announcement stated that ACRRM's training intake will increase from 150 to 250 in future years, which is likely to be a welcome expansion.

Another potential source of faculty expansion to consider is to draw on the large proportion of international doctors, both those graduating domestically as international students or those migrating as graduates to Australia from their home country (OTDs). Each of these groups face Australian regulations that require up to 10 years of rural practice if they wish to access Medicare billing opportunities in Australia. Other research identifies that FGAMS have higher odds of working as a GP than local graduates, but decreased odds of working rurally [42]. Additionally, OTDs constitute a high and increasing proportion of GPs and other specialists in large and smaller rural communities [43]. Despite this, OTDs were seen to have considerably lower rural faculty membership, and there may be ways for current faculties to attract uptake of memberships by this group (and FGAMS), in order to encourage their experience of collegial and skills-supported rural practice.

The FACRRM group are predominantly male (75–83%, or 60% for graduates >1995), despite the majority of Australia's recent medical graduates being female (around 55–60%). This may relate to ACRRM's relatively large initial recognition of prior learning process to grow the faculty at its initiation, but it may also reflect that female GPs are less likely to practice procedural skills and often desire more control of their working hours [44,45]. Flexible training options, supportive team practices with sufficient staff relief, salaried employment options, female-tailored continuing professional development topics, and robust social and professional network opportunities may be important strategies to attracting more females to this workforce [44,46,47]. Previous research has demonstrated the linkage between female GPs having children and relocations to more urban settings, with the same effect on males only occurring when the children are of secondary school age [48]. There appears to be a strong scope for rural faculties to play a role in accommodating the tailored employment and family needs of doctors.

Potentially related to their work locations, a higher proportion of rural faculty had a broader scope of work than standard qualified GPs. Notably, a higher proportion of FACRRMs who recently graduated (>1995) were using their advanced skills, whereas GPs mostly used four procedural types. This is likely capturing the strong association between the recent graduates working in Queensland where there is a specific state-based award, recognition, and remuneration for procedural rural generalist doctors [37]. This may also relate to FACRRMs, unlike FARGPs, compulsorily required to complete at least 12 months of training in smaller rural communities and 6 months of emergency medicine. Thus, their members may have greater confidence in working in more isolated communities requiring advanced skills. Maintaining advanced skills in procedural practice areas is likely to depend on matching training options to community need and ongoing job opportunities, availability of hospital departments with service gaps from other specialists, as well as employing adequate professional rewards and continuing learning support for advanced skill use [49].

A limitation of our study is that it likely has undercounted specific qualifications and advanced skills, as we relied on self-reported data. It is not possible to distinguish between incomplete (missing) data and genuine not applicable (missing) data. A further limitation of this study is that qualifications via RPL mostly cannot be distinguished from those related to completing training requirements. RPL was a major feature of ACRRM's establishment and thus results of only more recent graduates are shown. This study presents a series of cross-sectional results, thus only associations rather than causality can be identified. A strength of this study is its use of national data, without focus on a single program; however, not all characteristics of the two rural faculty programs will readily match those of other countries.

### **5. Conclusions**

This study demonstrates the value of different rural faculty models for building a skilled and qualified rural generalist GP workforce, over standard GP training. It highlights that rural faculties, whether as a standalone rural college (FACRRM) or embedded within an existing faculty (FARGP), reflect a common professional practice model. Both groups of rural faculty members related to a majority geographically distributed workforce (>50% in rural communities), practicing at a broader scope. FACRRM members, however, were more likely to work in smaller rural communities and retain use of their procedural skills. Our evidence suggests that rural faculties may better cater for a rural-ready primary care workforce with common professional practice models, providing potential gains for developing rural-specific networks, continuing professional development activities, and promoting recognition of rural practice. A key factor for future planning is maintaining objective data to evaluate further the critical design, progress, and outcomes of rural faculties against their specific missions to ensure that they remain fit for purpose. Expanding the utilization of rural faculties to sufficient capacity is likely to be a critical strategy for building and sustaining a primary care doctor workforce that meets the needs of rural communities.

**Author Contributions:** Conceptualization, M.R.M. and B.G.O.; methodology, M.R.M. and B.G.O.; formal analysis, M.R.M.; data curation, M.R.M.; writing—original draft preparation, M.R.M.; writing—review and editing, M.R.M. and B.G.O. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work is part of the Medicine in Australia: Balancing Employment and Life (MABEL) longitudinal study of Australian doctors. MABEL is funded by the National Health and Medical Research Council (Health Services Research Grant 2007–2011 and Centre for Research Excellence in Medical Workforce Dynamics: 2012–2017), with additional funding from the Commonwealth Department of Health (2008) and Health Workforce Australia (2013).

**Acknowledgments:** The authors acknowledge the initial project completed in 2017 by Akil Islam (FACRRM, FRACGP), undertaken as part of his Advanced Specialist Training in Academic Medicine with ACRRM. His project, which explored basic FACRRM member outcomes, was a strong inspiration for this study. We thank the other members of the MABEL team for their support and input and especially acknowledge the doctors who give their valuable time to participate in MABEL.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*

### *Article* **Workers' Healthcare Assistance Model (WHAM): Development, Validation, and Assessment of Sustainable Return on Investment (S-ROI)**

**Lilian Monteiro Ferrari Viterbo \*, André Santana Costa, Diogo Guedes Vidal and Maria Alzira Pimenta Dinis**

UFP Energy, Environment and Health Research Unit (FP-ENAS), University Fernando Pessoa, 4249-004 Porto, Portugal; decovirtual@yahoo.com.br (A.S.C.); diogoguedesvidal@hotmail.com (D.G.V.); madinis@ufp.edu.pt (M.A.P.D.)

**\*** Correspondence: lilianmferrari@gmail.com; Tel.: +351-225-071-300

Received: 21 March 2020; Accepted: 29 April 2020; Published: 30 April 2020

**Abstract:** The present study aimed to present and validate the Worker´s Healthcare Assistance Model (WHAM), which includes an interdisciplinary approach to health risk management in search of integral and integrated health, considering economic sustainability. Through the integration of distinct methodological strategies, WHAM was developed in the period from 2011 to 2018, in a workers' occupational health centre in the oil industry in Bahia, Brazil. The study included a sample of 965 workers, 91.7% of which were men, with a mean age of 44.9 years (age ranged from 23 to 73 years). The Kendall rank correlation coefficient and hierarchical multiple regression analysis were used for the validation of WHAM. The assessment of sustainable return on investment (S-ROI) was made using the WELLCAST ROI™ decision support tool, covering workers with heart disease and diabetes. WHAM can be considered an innovative healthcare model, as there is no available comparative model. WHAM is considered robust, with 86% health risk explanatory capacity and with an 85.5% S-ROI. It can be concluded that WHAM is a model capable of enhancing the level of workers' health in companies, reducing costs for employers and improving the quality of life within the organization.

**Keywords:** Workers' Healthcare Assistance Model (WHAM); patient-centred care; integrated care; interdisciplinary; sustainable return on investment (S-ROI); economic sustainability; WELLCAST ROI™

### **1. Introduction**

More than ever, life, as we know, will never be the same. The world is currently experiencing the coronavirus pandemic (COVID-19) [1], an unforeseeable health development that is affecting the entire global population, and consequently healthcare assistance models across the globe. There is now an urgent need to look at human health through the "one health" lens [2], to design and implement programs, policies, legislation, and research in a cooperative manner among all sectors of society to achieve better public health outcomes.

In addition to the recognition of the success of the current healthcare models in the relief of pain and the treatment of multiple pathologies, several criticisms are gaining support, pointing out the limitations relating to the attention to patient health. These issues include approaches that take an undifferentiated view of the individual, which is focused exclusively on the part of the body that is sick; the focus on the curative actions of diseases, injuries, and damages; the advancement of medicalization; and the generalization of hospital care using technology. In the past, if a medical doctor was seen as a figure possessing the knowledge necessary to cure the patient, nowadays that figure is seen as one part of a team, with the patient being the final decision-maker in their health outcomes.

The World Health Organization has chosen to strengthen people-centred care and integrated health services as priority strategies to transform health services to meet the health challenges of the 21st century [3]. This favours the emergence of integrated care models, which are seen as possible solutions to the growing demand for improvement in the patient experience, especially in patients with chronic conditions.

Considering economic sustainability in the search for integral and integrated health, this study aims to present and validate a model of workers' healthcare, the Workers´ Healthcare Assistance Model (WHAM), which embraces an interdisciplinary approach towards health risk management.

In light of the literature review, the following three research hypotheses were formulated:

**Hypotheses (H1).** *WHAM promotes integral and integrated care;*

**Hypotheses (H2).** *WHAM is robust and has greater explanatory capacity for workers' health risks;*

**Hypotheses (H3).** *WHAM is economically sustainable and provides a significant return on investment.*

### **2. Literature Review**

A review of the literature in the field of occupational health highlights discussions relating to "assistance models", a term that varies based on the conceptualization, which can include "assistance modalities or technological models" [4,5]; "ways to promote health" [6]; "assistance models" [4,6,7]; "technical, techno-assistance, and technical assistance models" [4,8]; "modes of intervention" [7]; "attention models" [9–11]; and "care models". The result of this diversity of terms is the already identified difficulty in conceptualizing assistance models. Healthcare assistance models are understood as technological combinations with different purposes, which are used to solve problems and meet needs within a given context and population and in a given territory (individuals, groups, or communities), to organize health services or to intervene in situations, depending on the epidemiological profile and investigation of health problems and risks [12]. These logical systems organize the functioning of care networks, articulating the relationships between network components and health interventions. In turn, these are defined according to the prevailing view of health, demographic and epidemiological situations, and social determinants of health at a given time and in a given society and place [13].

According to Campos [5,6], the conceptualization of an assistance model, technological model, or assistance modality must go beyond mere organizational and technical design, showing a new way of producing assistance actions anchored in the organization of the state.

According to Silva [14], biomedicine has become the hegemonic model in the provision of health services in Brazil and other countries around the world, influenced by accumulated knowledge and the paradigm of science. In this process, the daily requirements in the health sector stand out, such as the relationships between people; the involvement and co-responsibility of managers, health professionals, and patients in healthcare; as well as the bond, reception, and humanization of healthcare assistance practices [15]. From a technological point of view, there is a predominance of the use of the so-called "hard technologies" (equipment), to the detriment of light technologies (professional-patient relationships) [8,16]. Thus, diagnostic tests are a priority, but patients are not necessarily considered in terms of their suffering. This approach has been the target of criticism at the international level, starting from the 1970s and gaining greater importance in the second half of the 1980s [11,17]. In terms of the biomedical model, there is a certain neglect of the importance of the determinants of the health–disease process; that is, the focus on the disease and not on the elements that contribute to health promotion, underestimating that cultural, ethical, and social aspects condition lifestyles and that these are also determinants in the same process [13,14,18].

Merhy [8] contributes to the debate about the need to change the hegemonic assistance model, arguing that it is necessary to impact the core of care. In this sense, it is necessary to invest in relational-type light technologies, focusing on the needs of users and reversing the investment in hard or light-hard technologies, which can be translated into standards, equipment, and materials. Thus, light technologies are used and combined with people and resources to achieve certain objectives, which are gathered in an organized manner and consolidated as essential elements of health services [19].

Regardless of the scope, health services are always complex. The processes are standardized by regulatory bodies, service providers, and class representatives, among others. They have highly specialized and qualified workers who, belonging to different class councils, have interests that do not always converge [20]. Team composition characteristics in health services must be highlighted, recognizing these team members as the main actors responsible for the implementation of technologies aligned to a healthcare assistance model. Faria [21] draws attention to the fact that actions performed in a given place to deal with a certain problem may not apply to other situations, considering the historical-political context that influences a situation. Therefore, the use of healthcare assistance models invariably requires the selection of certain constructs that support them. Thus, they can be used in an alternative or adapted way, as long as they enable the achievement of similar results. To incorporate new health needs, healthcare assistance models can be considered to have influenced the organization of care models, being more focused on specific populations, such as the chronically ill. A comprehensive care model defines how health services are offered, providing the best care and service practices for a person or population group as they evolve through a condition, injury, or event, aiming for people to receive the right care, at the right time, by the right team, and in the right place [22].

The field of occupational health is a fertile environment for the development of interdisciplinary practices [23–26], as it encompasses knowledge from different disciplines, requiring constant and complex interactions between professionals in the fields of epidemiology, the environment, engineering, and healthcare, among others. The framing of occupational health in a biomedical healthcare assistance model favours the development of disjointed and ineffective interventions regarding the needs presented by workers, while the biopsychosocial model is often used in their work environments. According to Annadale [27], the biomedical healthcare assistance model only focuses on the physical processes, i.e., the pathology, biochemistry, and physiology of a disease, neglecting the roles of social factors or individual subjectivity.

In this context, it is necessary to discuss a model of assistance in occupational health that is capable of reviewing the central characteristics of the biomedical healthcare assistance model, including: (i) organization of practices focused on the identification of signs and symptoms and the treatment of diseases, with health promotion not being a priority; (ii) assistance is organized based on individual spontaneous demand, with an emphasis on specialization and the use of hard technologies; (iii) the work is developed in a fragmented, hierarchical manner and with inequality across different professional categories; (iv) difficulty in implementing the integrated care due to the lack of understanding of the individual as a multidimensional human being, as well as the lack of communication and integration between the services involved; (v) health planning is seldom used as a management tool; (vi) the training of health professionals is specialized, based on the hegemony of scientific knowledge; and (vii) themes such as interdisciplinary, people-centered care, attachment, and welcoming are not prioritized. Another aspect of great relevance in the current global context of scarcity of resources, particularly in the current context of COVID-19, is the prioritization of investments ineffective, integral, and integrated interventions, which can be achieved through a model that contemplates the management of occupational health risks, considering the social health determinants [28,29], global disease burden [30], environmental aspects [31,32], sustainable development goals [33,34] and in particular, working conditions that affect an individual's health [35].

In the current context, the effectiveness of a healthcare assistance model must include economic sustainability in addition to health gains, to know how much the company has earned due to investments made in a certain area, with the sustainable return on investment (S-ROI) being a very important metric for this assessment. Measuring the S-ROI [36–38] of preventive programs is not an easy task, due to the large number of variables that influence this calculation. The main variable is patient health, which can improve or worsen unpredictably. Analyzing the S-ROI in preventive programs identifies the financial impact a program generates concerning the amount invested, which must be

considered in the long term. Disease prevention actions bring future returns, mainly to the reduction of healthcare assistance costs. If the individual participates in preventive programs, the probability of developing diseases or discovering them in advanced stages decreases. Over the past 20 years, several studies [39–47] have addressed this issue and there is growing evidence that workplace programs can generate acceptable financial returns for employers investing in them. A study of Johnson and Johnson employees [39] showed a difference in the increase in the average annual costs of internment between workers involved and not involved in lifestyle improvement programs and changes in the workplace, representing \$43 and \$76, respectively, thus representing a considerable increase in percentage terms. The study by Munir et al. [45] aimed to conduct a cost-benefit analysis of the stand more at work (SMArT) workplace intervention, designed to reduce sitting time. A net saving of \$2.18813 (95% CI; \$−4.3804; \$4.8143) per employee was found as a result of productivity increase. Peik and others [46] applied the Research and Development (RAND) Europe model, a program designed to expand access to up to 40 evidence-based clinical preventive services for all employees and eligible family members, as part of a unique global health initiative at the country level to estimate the return on investment over a five-year timeframe. The study concluded that this program generates a global return of \$4.28–\$11.88 (after investment cost). Gao and co-workers [47] assessed the economic performance of a workplace-delivered intervention to reduce sitting time among desk-based workers. The incremental cost-efficacy ratios ranged from \$6.28/minute reduction in workplace sitting time to \$8.45/minute reduction in overall sitting time. The intervention was cost-effective over the lifetime of the cohort when scaled up to the national workforce, and provides important evidence for policy-makers and workplaces regarding the allocation of resources to reduce workplace sitting.

### **3. Materials and Methods**

### *3.1. Study Design*

The present study was carried out from 2011 to 2018, in a workers' occupational healthcare centre in the oil industry in Bahia, Brazil. It involved the integration of distinct methodological strategies for the development of WHAM, such as the development of a conceptual model, action research, statistical validation, and S-ROI analysis. The study involved two experts who had been working in the field of occupational health for fifteen years, with an emphasis on ergonomics and health management, an interdisciplinary approach, and a database composed of a population group and sample of workers, numbering 1275 and 965 individuals, respectively (Table 1).



### *3.2. Data Analyses*

Data analyses were carried out using SPSS version 25 for Windows (IBM Corporation, New York, NY, USA). Diagnostics and intervention prevalence were presented as absolute and relative frequencies. Correlations among modifiable health risk factors and health outcomes were performed through the Kendall rank correlation coefficient. Correlations among health indicators and the interdisciplinary risk coefficients were also performed using the Kendall rank correlation coefficient. Hierarchical

multiple regression analysis was used to calculate the independent contributions of occupational medicine interdisciplinary, dentistry interdisciplinary, physical education interdisciplinary, nursing interdisciplinary, and nutrition interdisciplinary risk coefficients, to provide an estimate of incremental variance accounting for the Workers' Health Risk Index (*WHRI*) [48]. This index had already been published, resulting from the classification of workers into three risk ranges—"low", "moderate", and "high". The Durbin–Watson test was applied to detect the presence of autocorrelation at lag 1 in the residuals (prediction errors), through which the hierarchical multiple regression analysis multicollinearity was verified. To lead the application of the WHAM, the "Guidelines for Implementing the Workers' Healthcare Assistance Model (WHAM)" were developed, which are presented in the Supplementary Materials (Word S1).

### *3.3. Model Development*

The "Workers' Healthcare Assistance Model" is understood as the organization of the conditions necessary to carry out a person-centred care process, about the method, staff, and instruments. The term "process" used in the context of healthcare makes it possible to identify, understand, describe, explain, and predict the needs of a person, family, or community at a given moment in the health and disease process, demanding professional care by health specialists. Therefore, WHAM presupposes a set of actions, through certain means of action, regulated by a course of thinking; that is, through a conception of workers' health, WHAM's origin and its potential to transform itself or to be transformed.

To compose the WHAM, the Interdisciplinary Workers' Health Approach Instrument (IWHAI) [49], a tool that had already been published, was used as a data collection instrument, aiming to collect data from 43 health indicators. To map the diagnoses, the health taxonomies were used, while the *WHRI* [48] was used to prioritize the health risks of the workers. Figure 1 shows the main stages of integrating the WHAM.

**Figure 1.** Phases in the Workers' Healthcare Assistance Model (WHAM).

### 3.3.1. Data Collection

The data collection stage aimed to identify health problems, as well as the efficient and targeted recording of the workers' needs in its broadest sense. For this, the IWHAI [49] was chosen. It allows structured data collection, covering the disciplines of medicine, dentistry, nursing, nutrition, and physical education, as well as environmental, occupational, behavioural, personal, and metabolic factors. It is composed of in 5 dimensions with 43 indicators, totalling 215 sub-indexes with closed response coding, where zero represents non-existent or inadequate control of risk and four represents optimal control of risk, arranged in the following scale: 0 = non-existent or inadequate; 1 = tolerable; 2 = reasonable; 3 = good; 4 = excellent.

### 3.3.2. Diagnostics Mapping

For the diagnostics mapping stage, it was necessary to define taxonomies that encompass the complexity of the workers' health field, especially those related to the health, environment, and work triad. The following codes were used for medical, dental, nursing, nutritional, and physical education factors: (i) International Classification of Diseases (ICD 11) [50]; International Classification of Nursing Practice (CIPE®) [51,52]; International Dietetics and Nutritional Terminology (IDNT) [53]; and the International Classification of Functioning, Disability, and Health (ICF) [54].

### 3.3.3. Intervention Planning

For the intervention design stage, it was necessary to define classifications that encompass proposals for interventions, which include ecological and occupational care. For each mapped diagnosis, an intervention must be associated. During the attendance of the worker, priority is given to diagnoses for health indicators that are classified as control or health conditions: 0 = non-existent or inadequate; 1 = poor; 2 = reasonable.

### 3.3.4. Interdisciplinary Consensus

This consists of a discussion amongst the interdisciplinary health team to validate the perceptions [55] raised by professionals in each area during the attendance of workers, sharing the diagnoses and interventions proposed by each discipline. The IWHAI [49] was used as a guiding instrument for data collection. For support of the team decisions regarding the hierarchy of priority interventions, the *WHRI* [48] was used, allowing multidisciplinary (by dimension) and interdisciplinary (association of all dimensions) risk classifications. The classifications comprise three ranges: "low", "moderate", and "high". Since 64% of the sample age is above 40 years and the gender proportion of male to female is very high, the effects of these factors were controlled in this step by the *WHRI* [48] assessment. As the workers' ages increase, the risk indicator also increases; the same happens for male and female workers for some sex-related diseases, such as the higher susceptibility by men to develop cardiovascular diseases and alcohol abuse. For this reason, when *WHRI* [48] is applied, each worker will have two risk indicators influencing the indicators of health behaviours and outcomes: a risk indicator related to the workers' age, whereby the older the worker, the higher their risk indicator; and another risk related to their sex, whereby female or male gender will have different impacts on health behaviours and outcomes. The final *WHRI* [48] score is mediated by the workers' age and sex.

The *WHRI* [48] dimension that has the greatest weight in the interdisciplinary context is designated as the worker case manager (WCM) and will assume technical responsibility concerning care management.

### 3.3.5. Implementation of the Healthcare Plan

The care plan (CP) is an interdisciplinary document, composed of relevant IWHAI indicators with their respective diagnoses and associated interventions, in addition to the definitions of the implementation and deadline. For the implementation of the CP, the WCM must bring together the interdisciplinary intervention team (IIT), ratify the CP, and proceed with the treatment of the proposed actions through interdisciplinary assistance, group work, and collective and environmental interventions. After validation of the CP by the IIT, the workers are involved in discussing the CP and implementing it at the individual level.

### 3.3.6. Assessment

The assessment stage deals with the follow-up and monitoring of the workers to the effectiveness of the implemented health interventions. For this, it is necessary to systematically reassess the *WHRI* [48]. The attendance took place in a single period (shift) by each member of the interdisciplinary team, with an average time of 40 min for each consultation and a total time of 3.5 h for each worker in the health service.

### *3.4. WHAM Validation*

To validate the WHAM, the data collected in 2018 were used in a representative sample of the population of 965 workers, where attendance by the interdisciplinary team occurred at the same time. Through statistical tests, the intention was to identify the prevalent diagnoses and interventions, how the modifiable factors are related to health outcomes in this sample, and the impact each dimension has on the *WHRI* [48], i.e., if the joint use of these dimensions contributes to greater robustness and explanatory capacity of the WHAM.

### *3.5. Assessment of Sustainable Return on Investment (S-ROI)*

To assess the cost-benefit (*CB*) relationship of implementing WHAM, interventions directed at workers with coronary heart disease (CHD) and diabetes in the period ranging from 2011 to 2018 were analyzed. The effectiveness of the intervention was based on the results of epidemiological studies over this period. Brazilian national data were used to estimate the average annual benefits of preventing direct medical costs for diseases.

The analytical tool WELLCAST ROI™ [56], developed to justify the approval of disease prevention and management programs, was used to calculate the S-ROI. For this, the following steps were taken: (i) determine the incidence of the pre-program disease; (ii) determine all costs associated with the disease, either medical costs (for CHD patients, the Framingham model [57] was used to calculate incidence pre and post-program for a period of 10 years, assuming changes in Low-density lipoprotein (LDL) cholesterol, and systolic and diastolic pressure risk factors; for patients with diabetes mellitus, the reduction in the progression of diabetes comorbidities over 10 years was calculated, based on the reduction of glycemia, considering the retinopathy, kidney disease, neuropathy, and microangiopathy comorbidities) or economic costs (monthly salary data, loss of daily productivity, medical inflation rate, among other rates estimated by WELLCAST ROI™); (iii) define the program and its cost; (iv) determine the effectiveness of the program in reducing costs; (v) subtract post-program costs from pre-program costs to determine reductions; and (vi) apply the concepts of net present value (*NPV*), internal rate of return (*IRR*), and *CB* to determine the S-ROI.

### *3.6. Ethical Approval*

In all stages of the study, the recommendations and guidelines of Resolution 466/2012 [58] of the Brazilian Ministry of Health on ethical aspects regulating research with human beings, approved by the Research Ethics Committee of the Bahia School of Medicine and Public Health and Certificate of Presentation for Ethical Consideration (CAAE) 84318218.2.0000.5544, were followed. All subjects gave their informed consent for inclusion before participating in the study.

### **4. Results**

The prevalent diagnoses and their respective interventions by dimension are presented in detail in Table 2.


**Table 2.** Diagnosis and intervention prevalence by dimension.


**Table 2.** *Cont.*

In the physical education dimension, the most prevalent diagnosis is "regular aerobic capacity" (76.3 %), with the most prevalent intervention being "encourage thinking about starting a physical activity program, warning about the harm of physical inactivity" (84.8 %). In the field of nursing, the "impaired ability to perform leisure activities" (100.0 %) stands out as the most prevalent diagnosis, followed by the need to "promote ergonomic comfort" (99.0 %) as the most necessary intervention. In the field of medicine, "primary essential hypertension" emerges as the diagnosis with the highest prevalence among workers (87.2 %), preceded by "encourage health-seeking behaviour" (95.5 %) as the intervention with the greatest application within this sample. At the nutritional level, "excessive

alcohol intake" is the most prevalent (99.0 %), with the intervention with the greatest application focusing on the need for "adequate macronutrients" (87.6 %). Finally, in the field of dentistry, the most prevalent diagnosis is identified as "other somatoform disorders related to stressful events—bruxism" (97.1 %), with the predominant intervention being "guide to restorative treatment with external dentist" (72.9 %).

Table 3 shows the statistically significant correlations between modifiable health behaviours and health outcomes.


**Table 3.** Significant (*p* < 0.05) correlations among modifiable health behaviours and health outcomes.

Note: 1—Diabetes mellitus; 2—Dyslipidemia; 3—Arterial hypertension; 4—Musculoskeletal pathology; 5—Triglycerides; 6—Caries; 7—Periodontal disease.

Moderate correlations in Table 3 (τ*b* ≥ 0.30) are identified as follows: between diabetes mellitus and altered blood glucose (τ*b* = 0.65), energy balance intake (τ*b* = 0.48), and the level of food knowledge (τ*b* = 0.46); between arterial hypertension and the contemplation stage for physical activity (τ*b* = 0.31); between the musculoskeletal pathology and the feeling of pain (τ*b* = 0.40); between psychiatric pathology and energy balance intake (τ*b* = 0.36); between triglycerides and energy balance intake (τ*b* = 0.32); between caries and oral hygiene quality (τ*b* = 0.30); between periodontal disease and periodontal condition (τ*b* = 0.76), oral hygiene quality (τ*b* = 0.58), level of food knowledge (τ*b* = 0.31), altered blood glucose (τ*b* = 0.45), energy balance intake (τ*b* = 0.44), and simple carbohydrate intake (τ*b* = 0.33).

The results are shown in Table 4 show which indicators are most correlated with each coefficient of each dimension of interdisciplinary risk.


**Table 4.** Correlations among health indicators and the interdisciplinary risk coefficients.

Notes: \* significant correlations (*p* < 0.05).

The values presented in Table 4 make it clear which indicators are most correlated with multidisciplinary risk; the worse an indicator is, the more the risk increases. Thus, in the field of physical education, it appears that the indicator of the contemplation stage for physical activity is the one that is most strongly correlated (τ*b* = 0.59). In nursing, the physical aspects of ergonomic risks have the most significant correlation (τ*b* = 0.44). In the field of medicine, diabetes mellitus is the most disturbing indicator (τ*b* = 0.60). In nutrition, alcohol consumption presents the strongest correlation (τ*b* = 0.45). Finally, the highest correlation of all is for oral lesion on soft or hard tissue, which is the most significant indicator in the field of dentistry (τ*b* = 0.82).

Hierarchical regression analysis was applied to understand whether the variables or dimensions under analysis explain a statistically significant amount of the variance of the dependent variable to be tested—in this case, the *WHRI* [48] *(*Table 5). A comparison of stages is made by gradually adding each independent variable in each stage, to understand if the combination of the dimensions explains more than considering them separately.


**Table 5.** Hierarchical multiple regression analysis scheme.

Notes: *B* = unstandardized beta; *t* = *t*-test statistic; *R* = multiple correlation coefficient; *R*<sup>2</sup> = R Square; R2 a = Adjusted R Square; R =Step 1: Constant = 0.370, F = 496.6, *p* < 0.001; Step 2: Constant = 0.300, F = 511.7 *p* < 0.001; Step 3: Constant = 0.231, F = 638.5 *p* < 0.001; Step 4: Constant = 0.101, F = 911.1, *p* < 0.001; Step 5: Constant = 0.035, F = 1141.3 *p* < 0.001. Durbin–Watson = 1.506. All predictors are significant at 0.05 level. No multicollinearity was identified.

It can be observed that as the dimensions under analysis are added, the model becomes more robust and has greater explanatory capacity for the dependent *WHRI* [48] variable. Thus, when comparing the first stage (step 1) with the last stage (step 5), an increase of 52% in the explained variance of the *WHRI* is observed with the 5 analyzed dimensions, showing values of 34% (*R*<sup>2</sup> = 0.34) and 86% (*R*<sup>2</sup> = 0.86), respectively. Medicine is the dimension with the most significant impact on the model (*B* = 0.205; *t* = 35.03; *p* < 0.05) and nursing has the least impact on the model (*B* = 0.168; *t* = 20.76; *p* < 0.05). The model's final expression is as follows:

$$\begin{array}{l} \text{WHRI} = 0.035 + (0.205 \times \text{Medicine}) + (0.194 \times \text{Nutrition}) \\ + (0.179 \times \text{Physical Education}) + (0.168 \times \text{Nurs sing}) + (0.166 \\ \times \text{Denttivity}) \end{array} \tag{1}$$

After analyzing the robustness of WHAM, its economic sustainability was assessed using the WELLCAST ROI™ tool. For the analyzed time period and based on the *NPV* of USD 23,363.29/per worker, the *IRR* of 85.5%, and the *CB* of 1.85:1, the S-ROI was determined, suggesting that WHAM is economically sustainable.

### **5. Discussion**

Given its complexity, the field of healthcare requires the mobilization of specialists from different areas, with the aim of promoting comprehensive and integrated care for workers. Based on an approach aimed at changing behaviors and adopting healthier lifestyles, going beyond the mere medicalization or treatment of diseases, the interdisciplinary care on which the WHAM model is based resulted in the data presented in Table 2. In view of the most prevalent diagnoses identified for each of the integrated dimensions, an intervention was generated that promotes worker autonomy and the maintenance of healthy lifestyles and behaviors, such as physical activity, healthy eating, non-consumption of alcohol and tobacco, good oral hygiene, balanced social and environmental relations, and decent work habits [55]. At this level, hypertension or diabetes mellitus diagnosis is highlighted, suggesting healthy behaviors or healthier eating habits interventions. As Eng and collaborators [59] state, the workplace is a key space for guidance around healthy behaviors and the reduction of non-communicable diseases (NCDs), such as diabetes mellitus and arterial hypertension. Viterbo and co-authors [23] report that long-term interdisciplinary practice has had very positive and significant effects on reducing NCDs. Hochart and Lang [60] also mention in their study that the implementation of a comprehensive care program in the workplace with the aim of modifying health risk behaviors resulted in a decrease in workers in the high and medium risk ranges and in the maintenance of health for those that were in the low risk range. The same is true for the issue of oral health, a problem that is related to other serious

diseases [61,62], and which is solved through the implementation of regular programs for the adoption of oral hygiene behavior among workers, as reported by Viterbo and collaborators [63]. Supporting these results, and in order to reinforce the importance of an integral look at workers' health, Table 3 presents the results between the behaviors (modifiable factors) and the results for workers' health. An overview of these results makes the connections between behaviors and health outcomes even more evident, as well as between the results themselves. In this case, an individual look at a worker would not allow one to understand them as a whole, contributing to fragmentation. Certain associations exemplify this idea, namely between the level of food knowledge and the type of food, identified by the energy balance intake, altered blood glucose, and diabetes mellitus. A similar relationship was identified in a review by Sami and co-authors [64], in which guidance towards healthier eating practices reduced the level of diabetes and prevented associated complications. The study by Holynska and colleagues [65] showed that the level of food knowledge is effectively related to nutrient intake, as this study also demonstrated. In line with this, Breen et al. [66] argued that the level of food knowledge enhances the choice of food, thus optimizing the quality of life of people with diabetes.

Table 4 shows the results of the indicators that are most correlated with the risk of each analyzed dimension, making it possible to identify those that contribute most to the increased risk in that dimension. The strongest correlation belongs to the field of dentistry, more specifically for oral lesions increasing the health risk of these workers. According to Warnakulasuriya et al. [67], conducting screening programs using valid visual inspection method to detect potentially malignant oral disorders within a workplace is not only feasible, but also effective. In terms of physical activity, the indicator that has the strongest correlation is that of the contemplation stage for physical activity; that is, the predisposition to start a physical activity. In the review by Jirathananuwat and Pongpirul [68], the 48 studies analyzed demonstrated that the workplace can play an important role in promoting regular physical activity among workers. Ergonomic risks in the workplace are, in this context, assumed to be the most correlated with risk in the field of nursing. This has been documented in several studies, namely by Skovlund et al. [69] and Welch et al. [70]. Since workers spend long hours of their day at the workplace, an additional concern regarding workplace ergonomics must be considered, as correct adaptation will result not only in promoting the well-being of workers, but also in reducing medical costs for employers, as reported by Munir et al. [45], Gao et al. [47], and Welch et al. [70]. In terms of pathologies, diabetes mellitus is the indicator that most contributes to risk in the dimension of medicine. In the reviews by Hafez [71] and Gan [72], the workplace is an important space for effective reduction of diabetes mellitus.

### *Implications for Workplace*

Some of the results in this study will have a direct implication in the workplace context, thus a more detailed specific analysis is necessary. The results regarding the WHAM robustness (Table 5) make it clear that the combination of technical and scientific knowledge in the work context results in a better understanding of the workers' global health. This result makes it possible to effectively verify that the interdisciplinary approach translates into gains in health, and that it must be adopted as a matrix in all work contexts, particularly those referring to a higher exposure risk and greater number of employees, as already identified in the studies by Viterbo et al. [23], Clark et al. [73], and Costa et al. [74].

Considering that health promotion and prevention actions can influence the health habits and behaviors of workers, they can also reduce health costs. The literature review [38,75–77] suggests that programs based on behavior change theory and using personalized communication and individualized counselling for high-risk individuals are likely to produce a positive return on the amount invested in these programs. The assessment of S-ROI in the specific model under investigation (WHAM) corroborates other studies carried out in the workplace [41,44,45,47], showing positive financial results and reinforcing the advantages of applying WHAM, which in addition to directing investment in health strategies that are proven to be a priority, enables the optimization of financial resources, resulting in

an S-ROI of 85.5% for interdisciplinary, integral, and integrated interventions for the community of workers with a high risk level.

### **6. Conclusions**

The search for a healthcare model for workers that is oriented towards integrated care, expanded health needs, economic sustainability, and which overcomes the problems arising from the hegemony of the biomedicine paradigm, such as the excessive use of technologies and focus on curative actions of diseases, is one of the great challenges of the Brazilian health system today. This scenario is strongly present in Brazilian scientific production and is reflected in national and international policies through legislation and public initiative.

The results obtained with the practical application of WHAM in the oil industry in Bahia, Brazil, demonstrated the potential of the model, where the articulated and hierarchical management of the various indicators of workers' health makes it possible to direct practices aimed at the cause and not at the effect or symptom. At the individual level, the model presented an interdisciplinary diagnosis of the health conditions of each worker, correlating the modifiable health factors and their respective impacts. The presentation of information to individuals promoted autonomy and empowered workers to change behaviors that negatively interfere with health conditions. At the collective level, the application of the model demonstrated the correlation between health indicators and interdisciplinary risk in the studied context, encouraging the creation of strategies aimed at the most critical conditions, as well as the design of preventive interventions. The robustness of the model highlights this same potential, in addition to the related optimization of financial resources of 85.5% for interdisciplinary interventions.

The absence of a similar model in occupational health is a limitation of this study since comparative analyses in the context of this work are not possible. The application of WHAM in different healthcare contexts is suggested in future studies, as well as carrying out analyses of the model's effectiveness by comparing the population's epidemiological results and studying the S-ROI.

The different theoretical contributions to the theme of this study, as well as the results found, lead to the understanding that WHAM can be considered as a model capable of encompassing the complexity of the field of occupational health, considering the interdisciplinary approach, risk management, and comprehensive and integrated care, in addition to accounting for economic sustainability for companies investing in healthcare.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/1660-4601/17/9/3143/s1, Word S1: Guidelines for Implementing the Workers' Healthcare Assistance Model (WHAM).

**Author Contributions:** Conceptualization, L.M.F.V. and A.S.C.; methodology, L.M.F.V. and D.G.V.; software, D.G.V.; validation, L.M.F.V., D.G.V., A.S.C., and M.A.P.D.; formal analysis, L.M.F.V. and D.G.V.; investigation, L.M.F.V.; resources, L.M.F.V.; data curation, L.M.F.V.; writing—original draft preparation, L.M.F.V., D.G.V., and A.S.C.; writing—review and editing, L.M.F.V., D.G.V., and M.A.P.D.; visualization, M.A.P.D.; supervision, M.A.P.D.; Project administration, L.M.F.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors would like to thank to UFP Energy, Environment, and Health Research Unit (FP-ENAS), University Fernando Pessoa (UFP), funded by the Foundation for Science and Technology (FCT), in the scope of FCT Project UID/Multi/04546/2019. The authors would like to thank the anonymous reviewers for the insightful comments and suggestions that helped to significantly improve the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest. The facts, conclusions, and opinions stated in the article represent the authors' research, conclusions, and opinions, and are believed to be substantiated, accurate, valid, and reliable.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*

### *Article* **The Challenges of Public Health, Social Work, and Psychological Counselling Services in South Korea: The Issues of Limited Support and Resource**

### **Luis Miguel Dos Santos**

Woosong Language Institute, Woosong University, Daejeon 34514, Korea; luismigueldossantos@yahoo.com or luisdossantos@woosong.org; Tel.: +82-010-3066-7818

Received: 1 April 2020; Accepted: 16 April 2020; Published: 17 April 2020

**Abstract:** Public health, social work, and psychological counselling professions in South Korea are facing challenges of human resource shortage and shortage of professionals who can provide multilingual services. The purpose of this study was to explore and understand why public health, social work, and psychological counselling services degree graduates and professionals with multilingual skills in South Korea decide to leave their professional field to the hospitality and business industries, particularly for those who completed their initial training at one of the international universities. Based on the approach of the Social Cognitive Career Theory, individuals' self-efficacy, outcome expectations, interests, and goals were examined and considered. The data were collected from 12 participants with the methodology of interpretative phenomenological analysis. The general inductive approach was employed to categorize the themes for reporting. The results indicated that public health, social work, and psychological counselling services-related positions are not available, modelling from peers, and lack of career development skills are the primary difficulties of public health, social work, and psychological counselling services graduates. The completion of this study provides clear recommendations to educators, policymakers, school leaders, human resource planners, and university administrators to improve their curricula and school counselling for public health, social work, and psychological counselling services graduates and the next generation.

**Keywords:** human resource shortage; mental counselling; psychological counselling; public health; shortage; social cognitive; social work; workforce management

### **1. Introduction**

### *1.1. Background of the Study: The Current Public Health, Social Work, and Psychological Counselling Services in South Korea*

The shortage of health and social caring professionals is significant in many regions and cities, including those in South Korea. Public health, social work, and psychological counselling services are some of the foundational services [1] that can help social minorities, the elderly, refugees, vulnerable people, female residents, and even sexual minorities [2] to overcome some social, cultural, and financial difficulties in society [3]. Although the South Korean government has established some governmental agencies and foreign resident support centers with multilingual services, such centers cannot offer long-term, gradual, and follow-up services to some minority groups in the community [4]. Nowadays, nearly a million foreign residents are living in South Korea for various reasons, such as education, family reunion, work, and even giving political support [5]. However, there are only a few organizations that can provide public health, social work, and psychological counselling services to these minority groups.

As a result, after completing their secondary school qualifications, many students decide to pursue undergraduate degrees, graduate degrees, and initial licenses in the fields of public health, social work, and psychological counselling services [6] at colleges and universities outside of South Korea with multilingual and inter-cultural training. Besides joining government agencies, large non-governmental organizations (NGOs), and non-profit organizations (NPOs), establishing their start-up NPOs may provide targeted services to particular groups of people in their specialization (e.g., refugee service, girls' rights in rural communities, orphanage for abandoned girls, etc.) [7]. However, such NGOs and NPOs are rarely found in the current South Korean environment.

Moreover, although South Korea is an international region with multi-disciplinary services, industries, and business options, the South Korean government tends to invest resources in the hospitality, tourism, and service management sectors [8]. Hotels, restaurants, entertainment, and similar industries employ many local and international residents. Although there are no official statistics regarding human resource management and occupational background, many local people and residents work in these industries. Based on the current employment trends of the region, many graduates with majors other than business and hospitality may not be able to work in the industries of their backgrounds, particularly for South Korean graduates who completed their education at an international university. Such unhealthy environments may limit the diversification potential of South Korea.

### *1.2. Purpose of the Study*

The purpose of this study was to explore and understand the factors contributing to the career decisions and decision-making processes of recent public health, social work, and psychological counselling services graduates in South Korea, particularly for those with degrees from overseas universities who are working in industries other than public health, social work, and psychological counselling services. Based on the approach of Social Cognitive Career Theory (SCCT) [9], individuals' self-efficacy [10–12], outcome expectations, interests, and goals [13,14] were considered and examined.

First, studies [15] indicate that recent graduates tend to enter industries corresponding to their academic majors and personal interests. Although financial consideration is a critical element in career development, public health, social work, and psychological counselling services professionals tend to consider personal interests and outcomes developments as their priorities [16–18]. One report [19] indicates that public health, social work, and psychological counselling services graduates in western societies usually start their centers or join NPOs after graduation. However, recent public health, social work, and psychological counselling services graduates in South Korea are not interested in starting businesses or NPOs. Therefore, the researcher aimed to understand the reasons for the career decisions and decision-making processes of recent public health, social work, and psychological counselling services graduates [20].

Second, although the South Korean government has financially and administratively supported many types of start-up NPOs and NGOs, unlike other professionals, individuals in the fields of public health, social work, and psychological counselling services have no interest in establishing businesses or NPOs of any kind in the public health, social work, and psychological counselling services industries. It is essential to understand the underlying reasons for this [21].

Third, one study [19] indicates that public health, social work, and psychological counselling services professionals tend to start their centers or NPOs after graduation, particularly with support from government agencies. Although the South Korean government has established planning for supporting such endeavors, public health, social work, and psychological counselling services professionals tend to give up and leave the public health, social work, and psychological counselling services industries to further develop their careers in other areas. The study aimed to explore this unique behaviour [22].

Fourth, career development theories have been based in large part on studies with university students, career changers, and working adults. Underrepresented populations have different career desires and face difficulties concerning their social status, academic majors, networking, and career development [14,16]. However, there are only a few research articles concerning the career pathways

and development of public health, social work, and psychological counselling services graduates, particularly in South Korea. Most research articles in public health, social work, and psychological counselling services examine how to incorporate public health, social work, and psychological counselling services into the curriculum, and how to enhance practices for patients. This is because public health, social work, and psychological counselling services educators tend not to feel strongly about the further development of graduates. Therefore, a large gap in career counselling and development is found for public health, social work, and psychological counselling services graduates. An emerging area of research focusing on career development has concentrated primarily on hospitality workers' perspectives and human resources shortages [23]. Graduates with academic majors other than business-related subjects represent a significant proportion of the region's human resources, particularly graduates with public health, social work, and psychological counselling services degrees from overseas. Yet, questions remain about the career perspectives and career planning of these groups of residents [1].

### *1.3. Theoretical Framework*

This study employed SCCT [9,13,14] to explore the career decisions and decision-making processes of recent public health, social work, and psychological counselling services graduates, particularly those working in industries other than public health, social work, and psychological counselling services. SCCT [24–26] is a famous theoretical framework for vocational and academic predictors of interests, career choice options, and performance. However, only a few studies explore the career and vocational perspectives of residents in South Korea.

More importantly, research on the views and behaviors of public health, social work, and psychological counselling services professionals are absent. Therefore, the result of this study may increase the social attention paid to public health, social work, and psychological counselling services professionals. The outcome of this study may lead to recommendations for policymakers, human resource professionals, secondary school staff, parents, career counsellors, and university administrators in planning career developmental plans for the next generation.

SCCT [9,14] was developed to understand and explore career perspectives and intentions. It aims to explain, describe, and explore academic and vocational decisions and performance, and the persistence of educational and vocational goals. This framework examines how people apply personal factors in the occupational and career development procedures and how personal elements increase, decrease, or otherwise impact personal agency. SCCT also indicates the significance of Social Cognitive Theory and triadic reciprocal causality [23]. Triadic reciprocal causality is an inter-influential point which impacts the connections, interactions, and significances among people, behaviors, and environmental factors. Activities, thinking, and behaviors of people are not the results of inter-connected events between people and environmental and social elements; instead, behaviors act as inter-connected elements by impacting and influencing results, thereby impacting the personal, emotional, intrapersonal, and further movements, activities, decisions, and behaviors of people [9,14,17,27,28].

Given SCCT's notions about how people, behaviors, and environmental elements could impact and influence the career decisions and decision-making processes of people, in this case, recent public health, social work, and psychological counselling services graduates, this study explored the factors that may be related to their career decisions and decision-making processes. The SCCT works as an effective framework when seeking to address the gaps in the current debate relevant to making the career decision to work in an industry other than that relevant to one's educational goals and interests.

### **2. Materials and Methods**

Based on the structure of the SCCT [9,14] and the significances, one research question guided the direction, which was: Why do recent South Korean public health, social work, and psychological counselling services degree graduates with overseas degrees decide to work in the fields other than public health, social work, and psychological counselling services? The purpose of this chapter was to outline the research methods, including research design, participants, instrumentation, data collection procedures, and data analysis procedures.

The researcher decided to employ the Interpretative Phenomenological Analysis (IPA) in understanding how lived stories and in-depth personal understanding associated with social, education, financial, and personal goals influenced the career decision and decision-making process of recent public health, social work, and psychological counselling services graduates who are working in the industries other than public health, social work, and psychological counselling services [29,30]. Previous Interpretative Phenomenological Analysis studies have shown how individuals and participants make sense of their personal and social world and experience. As the aims of this study tended to collect in-depth understanding and rich lived stories from the participants, the Interpretative Phenomenological Analysis was appropriate as the methodology. The researcher employed the qualitative research method with purposive sampling for recruitment.

### *2.1. Participation*

The sample involved 12 participants. First, the Interpretative Phenomenological Analysis seeks to understand the in-depth understanding and lived stories. Each represents a lived story that may be extremely meaningful. The data collection and analysis may highly satisfy. Second, the general phenomenological analysis may recruit at least ten participants and up to 200 participants. Large sample size may eliminate and cover up the rich lived stories from each participant. Also, as this study primarily focuses on the in-depth understanding of career development for South Korean residents who completed their public health, social work, and psychological counselling services degree overseas, a small sample size was more reasonable [31].

A purposive sampling strategy [31] was employed to invite 12 students who graduated overseas. Participants' demographic information, such as name, age, gender, year of experience, educational background, etc. was collected. However, the identifications of participants were pseudonym to protect the privacy.

The participants of this study satisfied the following requirements,


The researcher contacted each potential participant by email invitation. The invitation letter provided the information including the nature, objective, aim, and methodology, requirement of participants, and purpose of the study with a declaration about their voluntary participation or non-participation. If the participant agreed with the research, the participant responded to the email for further actions. For detail about the participants' information, please refer to Table 1.

### *2.2. Data Collection*

The researcher was the primary tool for data collection and analysis. Participants usually shared personal background, lived stories, and career decision with people who they can believe. To establish a solid relationship, the researcher designed two rounds of a semi-structured and one-on-one interviews. The member checking interviews were conducted after the completion of data analysis. Therefore, both met each other for at least three interview sessions in total. Each semi-structured interview was between 60–90 min. The member checking interview was hosted for about 30 min. Pseudonyms were assigned to shadow their privacy [32].


**Table 1.** Biography of the participants.

### *2.3. Data Analysis*

Themes, patterns, and groups that were categorized during the interview sessions were individually mapped. The general inductive approach was employed to analyze. The general inductive approach allowed researchers to explore the interview transcripts. First, the researcher followed the general inductive approach to reduce the large-size interview transcripts into the first-level themes by employing the open-coding strategy based on the direction of the grounded theory approach. Due to the detailed and in-depth interview sessions, the researcher captured 700 pages of interview transcripts. After the open-coding strategy, the researcher categorized 19 themes and 28 subthemes for the first-level reporting [33].

Second, based on the first-level themes and subthemes, the researcher reduced the interview transcripts into the second-level themes and subthemes with the axial-coding strategy. Several qualitative researchers [31,33,34] have indicated that the axial-coding strategy allows the researcher to narrow down the first-level themes and subthemes based on the research question. Therefore, three themes and four subthemes were categorized.

### *2.4. Human Protection and Ethical Consideration*

The study involved 12 participants in South Korea. The protection of human subjects was important to this study, particularly given the study's focus. Therefore, the researcher made every effort to protect the identities of all those involved, allowing them to remain anonymous to any parties in society. In the report, each person was identified solely by their role.

All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Social Caring Center (Summer/2019).

### **3. Results**

During the interview, the participants answered the same general open-ended questions about their educational background, ideas regarding public health, social work, and psychological counselling services in the current South Korean environment, expectations to do with their degrees, personal goals, the social expectations of the public health, social work, and psychological counselling services professions, governmental policy toward the public health, social work, and psychological counselling services industries, and so on. Although all had similar interests in the fields of public health, social work, and psychological counselling services, some of their shared experiences, lived stories, and financial considerations were not the same. The situation in South Korea is unique. As a small region with a large population, people's lifestyles, family structures, conceptions, and understandings may have many common grounds while also having many differences due to geographic elements. To answer the research question in a structured order, this section is categorized into three themes

and four subthemes based on the interview transcripts and information from the participants. Table 2 outlines the themes and subthemes of this study.



### *3.1. Public Health, Social Work, and Psychological Counselling Services-Related Positions are not Available*

The high percentage of hospitality employment reflects that most available positions are in a single industry. Professionals from other fields are less likely to expand their expertise in their own fields. In the current study, all participants were in the field of public health, social work, and psychological counselling services, where employment is less likely to be found in the business and non-profit sectors.

All participants had public health, social work, or psychological counselling services degrees from overseas institutions, but none of them were able to work in public health, social work, and psychological counselling services-related positions due to the shortage of opportunities. All except Participant#8 and Participant#9 believed the government did not emphasize the public health, social work, and psychological counselling services industry enough. They referred to the current public health, social work, and psychological counselling services environment as a "desert" to reflect the lack of support. Participant#9, who had a psychological counselling degree, but was working as a restaurant servant, asserted the dead-end nature of the counselling services environment, saying, "the hotels kill all other professionals. The tourism industries are the big dragon. No one in the city even supports psychological services." Participant#2 echoed this negativity, saying, "I tried to ask the Government Department for support for the support of sexual minorities' services for foreign residents in Seoul ... not a hard request, but only negative news." Participant#10 also expressed that there were no opportunities for recent graduates, saying,

... *for those who want to join the field, there are only two ways, either start your own non-profit or enter the government. Public health, social work, and psychological counselling are not easy jobs* ... *in South Korea, even if I graduated from a top-tier university, there are no positions* ... *it depends on internships and networking* ... *at least I don't have savings to start my own [organization]. Even if I do have millions, I cannot recruit all graduates* ...

Participant#12's academic background was similar to Participant#10's (e.g., psychological counselling), but Participant#12 worked in the surveillance office due to his video-related skills (i.e., academic minor). Although Participant#12's working position was related to video, the job specifications and responsibilities were not the same as for public health, social work, and psychological counselling services, as Participant#12 explained, saying, "Looking at the camera for potential cheating is not the same as making meaningful and enjoyable videos for the minorities in my country. I cannot say I entered into the public health, social work, and psychological counselling services, this is hospitality ... " Participant#3 asserted that many orphanages and religious churches are operating in South Korea. Although a few places recruit people without networking and connections, as Participant#3 said, "South Korea is an international region ... but the governmental agencies and large-size non-profit organizational heads recruit their own team members ... for us ... there are no openings." Participant#5 and Participant#6 were hotel front line attendants at the same company. Both used the statement "no future, no dream for our public health, social work, and psychological counselling services career" as

an ironic pun on the hotel's slogan. Participant#7 somehow used his professional skill to make fun of his current position saying,

*I wish I could leave this meaningless position in valet parking. But I am glad that I can laugh in front of all customers all the time. At least I learnt social work skills. Even if I don't want to work, I must work for basic living.*

Participant#4 continued to seek openings as a professional in the fields of public health, social work, and psychological counselling services in the near future, but his expectation of potential opportunities in public health, social work, and psychological counselling services was weak, as he explained, saying,

*The government said there are rooms to open our center. But it has been four years. I want to go back to public health, social work, and psychological counselling services. But can I come back? I am afraid six years later I will not have the courage to leave my position in this hotel.*

Participant#8 did not express much negative thinking about her position, but believed professionals in the field of public health, social work, and psychological counselling services should able to take hardship, saying,

*I understood there were no opportunities once I graduated. This was my own choice* ... *I cannot work as a professional in the field of public health, social work, and psychological counselling services now. But I can enjoy chatting with my customers in the hotel. I can help my customers and other co-workers at my workplace* ... *But this is very stupid* ... *I have my professional skills in the field of public health, social work, and psychological counselling services* ... *But I cannot use my multilingual skills and professional skills to help the correct minorities and people who are su*ff*ering from pain* ...

Participant#11 worked in the marketing department. The department relied on Participant#11's photographic and art therapy skills to put pictures in emails. She said, "my responsibilities are 20% related to my public health, social work, and psychological counselling services profession, particularly art therapy. All footnote pictures at the bottom of emails in my department were my art. But I am only responsible for typing and responding to emails between my company and customers." Participant#1 was the only participant who was not working in the hospitality and hotel industry. Her viewpoint was slightly different from the others', who were working in the hospitality industry, saying,

... *I enjoy working as an administrative assistant. I can coordinate some printing workshops with the center users and members. I don't think I can open my center because I don't understand how to operate a counselling center. I would rather use the money for my down payment* ... *for my apartment and unit* ... *perhaps my children in the future?*

All participants were working in industries other than public health, social work, and psychological counselling services. Recent graduates with public health, social work, and psychological counselling services degrees may apply their professional skills and abilities to other professional environments. For example, Participant#11 applied her photographic and art therapy skills in her marketing department and email designs. However, most of the participants worked in fields that were totally removed from public health, social work, and psychological counselling services. Many expressed negative comments due to the mismatching of career expectations and personal development. For example, Participant#3 worked as a ticket seller in a hotel's box office. This mismatching may further create a high level of turnover due to dissatisfaction.

### *3.2. Modelling Peers*

Starting a career pathway is not an easy step for recent public health, social work, and psychological counselling services graduates without much working experience or connections. First, unlike their counterparts in business schools or vocational training institutions, students in public health, social work, and psychological counselling services programs may not need to complete a business-oriented, practical internship for their professional year. Second, all participants completed their academic degree at a university overseas. Most were therefore unable to work in organizations outside the university setting without visa sponsorship. The visa requirement limited their opportunities to seek appropriate working experience during their academic career. Third, South Korean students do not have the right of abode overseas. Most had to leave their host countries within a certain period after graduation. Even if they had built up strong connections and good networking, they had to leave after graduation. Therefore, the absence of social and vocational connections within the professional field in South Korea was disadvantageous for this group of public health, social work, and psychological counselling services graduates.

### 3.2.1. Modelling and Referral from Classmates

All participants worked in the hospitality industry except for Participant#1. One of the strongest reasons why graduates with public health, social work, and psychological counselling services degrees joined the hospitality industry was peer influence. All participants were originally from South Korea, where they completed their secondary education. Therefore, most of their peer connections and career recommendations were in South Korea. Participant#2, Participant#5, Participant#6, and Participant#8 said that they entered the hospitality industry due to suggestions from their secondary classmates. Participant#2 said,

*Several of my classmates studied hospitality and tourism management. They were able to seek their first full-time position right after their internships. Therefore, they referred me to their department supervisor and recruited me. I am so fortunate to have a full-time job after I came back.*

Participant#5 echoed a similar expression about referral from classmates, saying,

*In South Korean culture, references from others are key to finding opportunities. My classmates in the business administration program told their bosses to recruit me. I applied for many positions* ... *but no responses* ... *most of my friends were doing well in the hotel, so I wanted to join and try too.*

Participant#6 elaborated on peer influence and her first position in the hospitality industry, saying, "my secondary classmates are successful in the hotel. I seriously don't think the government is going to support my center. To survive, I follow my classmates' footsteps and make some money for living."

### 3.2.2. Modelling and Referral from Cousins

Participant#4, Participant#7, and Participant#9 were influenced by their same-age cousins who were in the hospitality industry. Participant#4 entered the reservation department due to the peer influence from his female cousin in a similar position, saying, "my cousin is a reservation assistant supervisor and says the workload is okay. I keep writing poems and storybooks during the days off." Participant#7's cousin was promoted to a supervisor position about two years ago. Participant#7 was able to secure his position due to the referral of his cousin, saying,

*I sent out my applications and CV [curriculum vitae] at the beginning of my last year of university. No responses or interviews. I knew my cousin was working in the parking department, so I sought him out for help. I cannot say I like it, but I know I must survive.*

Participant#9 also sought her first position in a different field based on the referral from a cousin, saying,

*Public health, social work, and psychological counselling services are not a trend* ... *the biggest companies are in France, the United Kingdom, Italy, the United States and other western countries.* *I understand the direction has been switched. I cannot wait for a lottery* ... *after 15 months of unemployment, my cousin helped me to send out my CV to her department head in a hotel restaurant* ...

It is important to note that modelling peers and classmates reflects the central element of Social Cognitive Career Theory. Scholars [9] have further advocated that modelling other people's success stories may highly influence individuals' career choices and behaviors. More than half of the participants switched their career direction from public health, social work, and psychological counselling services to hospitality due to the strong influence of their peers. These participants may be going against their own principles. However, peer influence changed their points of view about long-term and short-term career pathways.

### *3.3. Lack of Career Development Skills*

Most public health, social work, and psychological counselling services programs do not provide vocational and career-oriented training and preparation for seeking opportunities in the business environment [21]. In fact, many organizations want to recruit business professionals to increase the image of their departments, particularly for marketing advertisements. However, public health, social work, and psychological counselling services graduates usually do not understand how to apply their professional and counselling skills in a business environment. Therefore, most public health, social work, and psychological counselling services graduates are unable to apply their professional skills in appropriate directions.

### 3.3.1. Afraid to Start Own Centers and Non-Profit Organizations

Three participants expressed that they had planned to start their own centers or NPOs during university. However, these participants stated that they did not understand how to begin, maintain, promote, operate, and continue such endeavors. Therefore, after consideration, they terminated their plans. Participant#3 shared his experience of establishing a center, saying,

*During university, the only public health, social work, and psychological counselling services professionals that I could encounter were my lecturers. They were very successful social workers, counsellors, and health professionals. But I could not learn ideas from them about establishing my own center. Most of them never started their own centers, so how could they have taught me?*

Participant#1 expressed another idea about the absence of career development, saying,

*I don't know how to attract residents and tourists to my center* ... *if I start in a small community. The program curriculum does not have such courses* ... *they only trained us as a professional service provider. But professional service providers also need money to survive.*

### Participant#6 further emphasized the feelings from Participant#1 saying,

*I know how to promote sexual health, elderly service, youth service, and women's issues* ... *During the last year of university, we had to learn how to serve multi-cultural and social disabled people from countries with political unrest* ... *but I wanted to start my center. But I somehow didn't know how to start the center. One or two counselling professionals may operate many of the centers in the market. But these centers don't hire outsiders. We have to start our own. But I didn't have the business sense* ...

### 3.3.2. Lack of Interdisciplinary and Practical Skills

Based on the interviews, the researcher noted that almost all the participants did not understand how to apply their valuable skills in the practical and professional environment outside of the public health, social work, and psychological counselling services professions. In other words, most of the participants only had skills in their own public health, social work, and psychological counselling services subjects, and no other professional skills. Due to the absence of interdisciplinary and practical skills required from potential employers, they were unable to expand their horizons to the next stage. For example, Participant#2 expressed her hardship in seeking employment, saying,

*Many international hotels are using technology to design their art products. But I am still in the 70s. I know how to draw and paint in watercolors with art therapy. But how to use a computer to print and how to use the computer to assist* ... *I don't know.*

Participant#9 also applied for the wardrobe and linen department at a hotel. Participant#9 should have applicable skills in clothing, so such a position should have been appropriate. However, Participant#9 expressed that her skill was not transferable, saying,

*My skill is in counselling and visual counselling. I know how to use a di*ff*erent color. My interests are all about color, cutting, and fitting. But in the wardrobe and linen department, they mainly focus on washing the clothes. I never learned that at university. I wish I understood, but I don't want to lie to the manager.*

Participant#4 should be a good writer and even speaker for documents and advertisements, but did not know how to apply this skill to a business environment, saying,

... *I applied for a newspaper journalist position. I wanted to write some articles for the forums. But they only recruited from the traditional section. I never studied, so I was refused an o*ff*er. For now, I apply for other contract writer and advertisement writer positions. Their requirements and expectations are not in my expertise* ...

Participant#5 and Participant#7 both expressed the same sentiment, saying, "how can I apply my professional skills into a business environment ... a good smile is okay."

All public health, social work, and psychological counselling services graduates should be experts in critical thinking and problem-solving. However, most were close-minded in other professional areas, particularly business. Public health, social work, and psychological counselling services elements are not hard to find in luxury hotels and shopping centers. Most expressed that their public health, social work, and psychological counselling services skills should apply in their particular direction. While some participants may have taken jobs in different industries due to money issues, resistance to applying their professional skills was also obvious.

### **4. Discussion**

The purpose of this section is to discuss the themes shared by participants and implications for the career decisions and development of public health, social work, and psychological counselling services university graduates, using SCCT as a theoretical lens. Based on the idea of SCCT, personal factors (i.e., personal beliefs, biological elements), human behaviors, and external environmental elements (i.e., social movement, society) are the central elements influencing the career choices and selections of individuals [9,17,35]. However, based on the findings of this study, external environmental elements strongly impact career selection.

First, the researcher sought to understand better why public health, social work, and psychological counselling services graduates with overseas degrees plan to work in an industry other than their academic major. Individuals usually do not select and enter careers in which their self-efficacy is lower than average [9]. However, the study discovered that all participants needed to enter an industry in which they had no experience. All sought public health, social work, and psychological counselling services-related employment opportunities but none of them were successful. Environmental elements strongly impacted their career decisions, more than human behaviors and personal factors. External environmental and ecological barriers in society can influence the career decision making of

individuals [16]. One of the most significant contextual barriers to career decisions was the limited public health, social work, and psychological counselling services-related positions in the region. The researcher was interested in understanding whether the outcome expectations of participants would also influence their career decisions, as in another study [14]. Although outcome expectations were considered a lower priority in the current study, several of the participants applied part of their public health, social work, and psychological counselling services skills in their workplace.

Second, it was surprising that almost all were working in the hospitality and service management industry, particularly the field of hotel management. Although there are also other industries, such as small business, government, transportation, and even finance, most decided to enter the hospitality and service management industry. Based on sharing from the participants, most were influenced by their social peers, classmates, family members, and the social situation. In line with SCCT [9,14], it was clear that modelling was one of the most important elements impacting individuals' career decisions and development out of the external and environmental factors. For example, Participant#2 said that most of her friends and classmates were able to achieve promotion to a reasonable position after their graduation due to the growing hospitality and service management industry, saying "regardless of my major, as long as I am willing to work in service management, I can work and potentially ... career promotion and advancement ... " In the current situation, external and environmental factors highly influenced how individuals selected their careers and their career development, regardless of their university major and background [22].

Third, other evidence to supporting the importance of external and environmental factors came from the influence of families and cousins. Families' and cousins' recommendations are the strongest factors influencing individuals' career decisions and development, particularly for individuals with a South Korean background. SCCT [9,14] asserts that individuals are influenced by external and environmental factors differently. In this case, individuals tended to be significantly influenced by external and environmental factors. For example, Participant#7 indicated that his cousin transferred his CV. Also, Participant#9 indicated that as there were no openings in her profession, she used her cousin's connections to get her first full-time employment. SCCT argues that individuals' social background, social consideration, and the current social environment always influence individuals' career decisions and development [9,13,14]. In this case, although the participants gained essential skills and abilities from their university education and training, they tended to enter and start their career and its development based on external and environmental factors [21].

Fourth, some claimed that they did not have the practical skills and understanding to establish a center with limited resources [21]. Unlike in other countries and cities, recent graduates usually do not have enough resources. Therefore, graduates with limited resources usually cannot afford the operating costs. Furthermore, some explained that the university curriculum did not offer any training in establishing a center. Therefore, self-efficacy [36] and personal intention [24] were low. As a result, most were influenced by external and environmental factors as their surrounding society and environment always encouraged them to follow social expectations and trends [9,13,14].

In short, individuals' career decisions and career development may be explored and described using SCCT [9,13,14] with personal factors (i.e., personal beliefs, biological elements), human behaviors, and external environmental elements (i.e., social movement, and society) as the key elements. In the case of public health, social work, and psychological counselling services university graduates, most were highly influenced by external and environmental factors due to South Korean cultural expectations and limited career opportunities. Unlike other countries and cities with reasonable rental fees, space, and career opportunities, South Korea does not provide additional support and opportunities for graduates to exercise their skills and practices with limited resources, regardless of their majors and skills, particularly for overseas graduates. Therefore, regardless of university majors and skills, most of the participants in this case expressed that their only employment opportunities after university graduation were in the hospitality and service management industry [1].

### **5. Limitations, Future Research Directions, and Conclusion**

### *5.1. Limitations and Future Research Directions*

Some may argue that the number of participants was too small. However, the in-depth and two-round interview sessions were used to overcome this limitation. First, this study employed the Interpretative Phenomenological Analysis methodology to capture in-depth understanding, lived stories, and sharing from the participants. The research study tended to capture how individuals made sense of and understood their social world. Thus, the results of this research study allowed a rich and detailed understanding of the background of the social problem.

Second, some may argue that the background of the participants was limited to the fields of public health, social work, and psychological counselling services. Each research study needs to focus on a group with a focused direction. Therefore, future research studies and projects may expand the current research to additional groups of people, such as medical professionals. As social problems, limited career opportunities, and human resource shortages may influence all industries and businesses as a general problem in South Korea, larger and wider studies and projects would be beneficial to all residents and government leaders.

Third, this research provides a blueprint to government leaders, human resource managers, school administrators, policymakers, university curriculum heads, and general residents for a background understanding of the shortages in the fields of public health, social work, and psychological counselling services in South Korea. This research study gathered feedback and sharing from a group of public health, social work, and psychological counselling services graduates who needed to enter other industries due to the limited career opportunities in the region. Such feedback and sharing should be an indicator to all government leaders to reconsider the region's social policy and management for the coming fiscal years. Therefore, interested researchers and government leaders should take this research study as a starting point for redesigning the social policy regarding the public health, social work, and psychological counselling services industries and related small organizations and companies in the region.

Fourth, countries, cities, and regions with similar situations may use this research to polish their current social policies and activities [1]. The situation in South Korea is extreme and hard to solve due to the chaotic management of the last few decades. Government leaders, policymakers, and human resource managers should have acted to improve the situation before this point.

### *5.2. Conclusions*

This study discovered three critical social situations in South Korea. First, South Korean society is unique to other similar countries and cities internationally. Unlike in other larger countries and cities, limited career opportunities, lands, and industries always prohibit skilled professionals from excelling in their areas of interest and skill based on their university majors and training. A large number of residents in South Korea are employees in the hospitality and service management industry. Although the government has established plans and agendas for additional industries and small businesses, most of the residents do not intend to start their businesses due to limited knowledge and environmental factors [1].

Second, the participants in this study tended to be influenced by external and environmental factors, in accordance with SCCT [9]. The results indicated that all had low levels of self-efficacy and confidence about starting their centers due to the social environment and expectations. Therefore, although most of the participants had the passion to start their own businesses in the fields of public health, social work, and psychological counselling services, most tended to enter the hospitality and service management industry based on social influences either from society or their peers.

Third, South Korea should spend additional resources on and give more consideration to industries and small businesses other than the hospitality and service management industry. Although the revenue and tax incomes of South Korea highly rely on the hospitality industry, the government should not neglect the development and promotion of other industries, such as public health, social work, and psychological counselling services, and even medical tourism [8].

**Funding:** This research was funded by Woosong University Academic Research Funding 2020.

**Conflicts of Interest:** The author declares no conflict of interest.

### **References**


© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*

### *Article* **The Future of Careers at the Intersection of Climate Change and Public Health: What Can Job Postings and an Employer Survey Tell Us?**

**Heather Krasna 1,2,\*, Katarzyna Czabanowska 2,3,4, Shan Jiang 1, Simran Khadka 1, Haruka Morita 1, Julie Kornfeld <sup>1</sup> and Je**ff**rey Shaman <sup>1</sup>**


Received: 30 December 2019; Accepted: 12 February 2020; Published: 18 February 2020

**Abstract:** Climate change is acknowledged to be a major risk to public health. Skills and competencies related to climate change are becoming a part of the curriculum at schools of public health and are now a competency required by schools in Europe and Australia. However, it is unclear whether graduates of public health programs focusing on climate change are in demand in the current job market. The authors analyzed current job postings, 16 years worth of job postings on a public health job board, and survey responses from prospective employers. The current job market appears small but there is evidence from job postings that it may be growing, and 91.7% of survey respondents believe the need for public health professionals with training in climate change may grow in the next 5–10 years. Current employers value skills/competencies such as the knowledge of climate mitigation/adaptation, climate-health justice, direct/indirect and downstream effects of climate on health, health impact assessment, risk assessment, pollution-health consequences and causes, Geographic Information System (GIS) mapping, communication/writing, finance/economics, policy analysis, systems thinking, and interdisciplinary understanding. Ensuring that competencies align with current and future needs is a key aspect of curriculum development. At the same time, we recognize that while we attempt to predict future workforce needs with historical data or surveys, the disruptive reality created by climate change cannot be modeled from prior trends, and we must therefore adopt new paradigms of education for the emerging future.

**Keywords:** climate change; health workforce; workforce planning; competencies; public health education

### **1. Introduction**

Climate change is acknowledged to be a major threat to public health [1,2]. Just as public health practice must constantly adapt to emerging viral outbreaks, non-communicable diseases, or other health threats, it must also be prepared for the diverse threats to human health posed by climate change. Several reports and large-scale commissions [3–10] point to the need for training for the health workforce, including the public health workforce, in skills and content to help lead efforts to mitigate and manage the impacts of climate change on health.

A 2008 report by the Association of Schools and Programs of Public Health (ASPPH) mentioned climate change as a key, new area of public health education [11]. The 2016 Council on Education in Public Health (CEPH) competencies for public health education include areas of focus, which allow public health professionals to protect human health from climate change impacts, such as analyzing data, discussing structural bias, assessing "population needs, assets and capacities that affect communities' health" and "applying systems thinking" [12].

Many competencies required for environmental health science students, such as "approaches for assessing, preventing and controlling environmental hazards that pose risks to human health and safety" [13] are applicable to climate change. However, knowledge of climate change specifically is not yet a core competency of public health degrees in the United States. New initiatives exist, such as the Global Consortium on Climate and Health Education, which now has 193 members [14] and recently proposed a set of Core Climate and Health Competencies for Health Professionals [15]. Additionally, the Association of Schools of Public Health in the European Region (ASPHER)'s 2018 Competencies does list climate change as a competency within "Population Health and Its Material-Physical, Radiological, Chemical and Biological-Environmental Determinants" [16] and the Council of Academic Public Health Institutions Australia (CAPHIA)'s Foundation Competencies for Public Health Graduates in Australia include "identify and describe the impacts of climate change and implications for ecologically sustainable development" and "climate change theory" [17].

To further identify existing research on the skills, competencies, and job market for individuals with training in both public health and climate change, we conducted a brief narrative review of the literature, primarily focusing on a keyword search of Google Scholar of "climate change" AND "public health" AND "workforce", which yielded 28,100 results, and "climate change" AND "public health" AND "jobs", which yielded 86,000 results; we also conducted a search of Pubmed.com for "public health education" AND ("climate change" OR "global warming"). Inclusion criteria included a focus on expected hiring needs for professionals with training in both climate change and public health. Articles that did not include information related to issues with workforce or training needs were excluded.

To identify competencies needed in a future workforce, and to ensure training aligns with labor market demand, it is accepted practice to rely on input from public health employers and organizations. Many ASPPH competencies are based on "blue ribbon panels" of employers [18], as are the Core Competencies for Public Health Professionals developed by the Council on Linkages Between Academia and Public Health Practice [19]. Similar employer input is needed to understand which skills current employers expect of public health graduates with respect to climate change. While employer surveys have been conducted in several public health workforce research articles [20–25], analysis of job postings-a potential key indicator of current employer requirements-has only rarely been used in the public health field [26–28]; this, combined with a survey of employers, can provide a fuller labor market analysis than has been conducted in the past.

Through our analyses, we can attempt to estimate current and future hiring trends for public health professionals with training in climate change-related competencies, as well as continue to identify the training needed to help address the threat of climate change. For those institutions creating new training programs focusing on both climate change and public health, it will be important to assess whether their graduates will be in demand in the labor market, and if so, which sectors are most interested in hiring candidates with these skills. We attempt to address the questions: Which employers currently seek graduates with training in both climate change and public health; and is the demand for such graduates likely to grow?

### **2. Materials and Methods**

In order to best discern whether there is a growing need for professionals with a combination of training in both public health and climate change, the researchers conducted an analysis of current job postings; and to create projections into the future, we conducted an analysis of 16 years worth of job postings in a public health job board. Finally, we conducted a survey of potential employers of public health graduates focusing on climate change to ask for their projections of the skills needed for this future workforce.

### *2.1. Data Sources*

### 2.1.1. Analysis of Current Job Postings on Indeed.com (Job Board Aggregator)

In order to determine what types of organizations are currently hiring candidates in the USA with a combination of skills or experience in both public health and climate change, on 14 December 2019, the authors conducted a search of Indeed.com, a job board aggregator, which "crawls" multiple job posting websites to gather millions of job postings into one, searchable database [29]. The rationale for searching Indeed.com is that job postings on the site are pulled from a broad range of thousands of job posting sites (including organizations' job sites as well as job boards), providing a snapshot of any jobs—not only within traditional public health organizations—that include a combination of relevant keywords, allowing an assessment of the scope of the existing job market and whether current jobs fit the training of public health graduates. Indeed.com allows for Boolean search operators. The authors searched for jobs with the following keywords: ("climate change" OR "global warming") AND ("public health" OR "environmental health" OR epidemiology OR "health policy"). A total of 172 jobs were found on Indeed.com; duplicates were removed, for a total of 159 positions. We then conducted a "scrape" (download) of the results using a commercially available web scraping tool called Scrapestorm [30], to identify the industries/sectors of the jobs with this combination of phrases. The Indeed.com main site primarily identifies jobs in the United States.

The resulting Excel file of organizations, job titles and descriptions, were then analyzed using the National Cancer Institute's SOCcer (Standardized Occupation Coding for Computer-assisted Epidemiological Research) system [31], "a publicly available application that was developed to assist epidemiological researchers incorporate occupational risk into their studies", to create Standard Occupational Classification [32] codes for the downloaded search results; those results with a lower degree of certainty in the automated coding system were hand-coded by the authors.

The industries/sectors of the employer organizations were also hand-coded, using a taxonomy in alignment with the new ASPPH employment outcomes data collection [33]. For context, an Indeed.com search of only the keywords "climate change" OR "global warming" conducted on December 19, 2019, found 2423 results. Thus, approximately 6.6% of the search results on Indeed.com related to climate change have an overlap with public health (159 of 2423). An Indeed.com search for ("public health" OR "environmental health" OR epidemiology OR "health policy") on 27 December 2019, found 37,490 jobs, so approximately 0.4% of public health-related jobs also mentioned climate change or global warming.

### 2.1.2. Analysis of 16 Years Worth of Job Postings on Publichealthjobs.org

The authors were provided access to 32,093 job postings posted into the free job board managed by ASPPH, publichealthjobs.org (previously publichealthjobs.net) dating from 17 July 2003–23 April 2019 [34]. This job board is frequently used by public health employers; it receives approximately 8.16% of all Internet traffic for the search terms "public health jobs" [35] and has been used for other analysis [26]. Of the 30,991 job postings for which the geographic location was known, 11.2% were from countries outside the United States. Unlike Indeed.com, which searches for job postings across numerous job posting websites throughout the Internet, the Publichealthjobs.org website requires employers to manually post their positions into the site, creating a self-selecting group of job postings that are specific to public health. The job description and requirements sections of the job postings were searched for the keywords "climate change" OR "global warming". Duplicates were removed. An analysis of the proportion of all postings that included either of the target phrases was conducted on a year over year basis from 2003 to 2019, using R coding [36].

### *2.2. Survey of Relevant Employers*

In order to assess the views of current employers who are likely to need candidates with training in both public health and climate change, the authors created an online survey using Qualtrics [37]. The survey questions were created through consultation with experts in both climate change and public health education, and included both closed-ended and open-ended questions (see Supplementary Material S1 for survey questions). Questions regarding specific competencies were based on the current curriculum of Columbia University's Climate and Health Certificate program. The survey and outreach methods were approved by the Columbia Human Subjects Review Board. Respondents were identified by the Columbia University Mailman School of Public Health Office of Career Services, which utilized its existing job posting database, a directory of approximately 5900 contactable employers who had posted a job or internship with Columbia University School of Public Health, or otherwise engaged with the career center, since 2012. These records are maintained using a secure vendor software hosted by the GradLeaders [38] company, and are accrued in a variety of ways: career services staff members conduct ongoing, targeted outreach by attending conferences and events such as the American Public Health Association conference, career fairs (including those focused on environment and sustainability), professional association memberships, online directories, leveraging faculty connections, and connecting to recruiters and alumni via LinkedIn.com and other social media platforms. Staff focused employer outreach efforts using input from ongoing surveys of students and engagement with academic departments and student organizations. A subset of 450 employer contacts from the jobs database was identified based on past job postings with keywords such as "climate change", as well as by targeting employers in industries and sectors related to environmental health.

Additional, new contacts were identified by using specific keyword searches on LinkedIn such as: Job Title search for (sustainability OR resilience OR mitigation OR adaptation OR carbon) and the general keywords of "Climate Change" AND "health"; and attempts were made to diversify industries of respondents. This allowed the authors to identify 100 new contacts; of these contacts, 12 were directly contacted via LinkedIn "InMail" messages and 51 by using publicly available information; 37 could not be contacted directly. Twenty-one alumni of the Columbia School of Public Health's Climate Change and Health Certificate program were also surveyed. Three contacts were referred by faculty at Columbia. A total of 537 active contacts were identified from all sources; contacts were primarily based in the USA.

The survey was distributed in January, 2019, with two reminders sent, once in January and once in March, 2019, and the survey was closed on 9 April 2019. Survey respondents were offered an opportunity to win a \$50 gift card as an incentive for responding to the survey, and they were also encouraged to forward the survey to others in their network. Ninety seven individuals responded. Ten respondents were excluded because they were current graduate students or postdoctoral researchers, as opposed to professionals employed in the field. In addition, the survey was forwarded to other contacts in many cases, and a link to the survey was also posted on several online discussion boards including the Planetary Health Online Community and Planetary Health Education Subgroup on Hylo [39]. Contrasting the survey recipients with responders, we found that 75 respondents came from our survey outreach and 12 were not on the survey distribution list. Of the 87 respondents, five were US-based international non-governmental organizations, one was a multilateral government organization, one was an international consulting firm, one was a US government agency focusing on global health, and seven were NGOs and corporations based in other countries including China, Mexico, the UK, Kenya, Haiti, and Ecuador. Thus 15 of 87, or 17% of the respondents were international.

A statistical analysis of the responses was conducted. To evaluate the perceived usefulness of skills among employers in the public health field, we designed a mixed version of questions in which the responses are ordinal consisting of seven levels or text. The survey questionnaire comprises fourteen Likert-scale items to assess the usefulness of specific skills; in the later analysis stage we removed the "other" category, so only thirteen were left for the factor construct. We used qualitative methods to analyze the information from the open-ended responses. For the ordinal Likert-scale data, we first measured the internal consistency of the questionnaire, which was performed using the whole sample with Cronbach's α values reported to be ≥0.60. Then we conducted a frequency description to identify if there was any ceiling effect or floor effect in the data. Finally, we used exploratory factor analysis and confirmatory factor analysis to identify the internal structure of the inventory. The factor analysis [40] is made up of two fundamental stages: (1) estimating the number of factors that should be extracted to represent the variability of the skillsets efficiently and (2) interpreting the meaning of the extracted factors and representing them in terms of theoretical structures that reflect the skillsets dimensions/sub-domains. In the analysis, factor loadings above ±0.40 were retained and listed in Table 4. We also assessed the trend of the annual number of public health job postings mentioning climate change or global warming as a function of year using Poisson regression. The total annual number of jobs was specified as an offset, and cross validation using continuous subsets of the total record was performed to determine if the results are unduly sensitive to a specific year or years. The descriptive and inferential statistical analyses were conducted using SPSS 24.0 [41] and R [36].

### **3. Results**

### *3.1. Literature Review*

Overall, there are many articles on the intersection between "climate change" and "public health," but relatively little on labor market projections. Several articles directly mentioned how public health nurses or health professionals can become involved in climate change response, prevention, adaptation, and mitigation, policy [42], risk management, disaster preparedness, vector-borne diseases, heat-related diseases, the evidence base for climate change adaptation, etc. [8,9,43,44]. One article focused on elements of workforce development including "undergraduate through postgraduate training" in health, professional development of existing workforce, and training of policy-makers [44]. There were three articles on the Australian response to climate change events such as bushfires, extreme heat, and poor air quality, as well as rural health services [5,7,45]. Other articles mention the training needs of governmental public health workers relating to climate change [46], or specific sub-areas of training such as nutrition [47], or the importance of communication [48], or focus generally on why climate change training is needed in public health education [49].

Several articles provided action plans related to climate change and public health, which would require workforce training [50,51]. These examples include diagnosing and investigating health problems and hazards; monitoring health status to identify and solve community health problems; focusing on disaster preparedness [4]; dealing with emerging infectious diseases influenced by climate change [4]; informing, educating, and empowering the public on these issues; evaluating the intervention effectiveness of population-based health services; and monitoring workforce strain due to climate change [52]. Overall, it is difficult to find quantitative public health employment data, but many of the articles mention the importance of training, workforce development, and education to prepare and integrate climate change into public health efforts.

### 3.1.1. Analysis of Current Job Postings on Indeed.com

The search of job postings from Indeed.com yielded the following distribution by industry:

corporation 32 (20%); nonprofit 76 (47.8%); government 17 (10.7%); and university/academia 34 (21.4%). In terms of occupational codes, the occupations with the largest numbers represented in the data set are listed in Table 1.

It is worth noting that the Standard Occupational Classifications do not include "community organizer", "grassroots activist", or "campaign organizer" as categories, so positions with these titles—the largest single group of positions in the data set—were coded as "Community and Social Service Specialists, All Other". There were a total of 17 faculty positions, 12 within schools of public health, and 5 in environmental or biological sciences. Environmental and occupational health roles—those most likely to be a fit for graduates with a Master's degree in public or environmental health—totaled 14

positions out of 159. Other common occupations included attorneys (primarily at government agencies related to environmental protection as well as legal advocacy nonprofits), public relations and fundraising, sales, and engineering roles. These data suggest that pursuing doctoral-level education, or combining a public health degree with either law or engineering, might best qualify candidates with an interest in both public health and climate change in today's job market, at least in the USA.

**Table 1.** Most common occupations in Indeed.com postings by the Standard Occupational Classification (SOC) code.


3.1.2. Analysis of 16 Years Worth of Job Postings on Publichealthjobs.org

The proportion of the 32,093 jobs from publichealthjobs.org from July 2003–April 2019, which mention either "climate change" or "global warming" consistently was a very small percentage of the total, but the percentage increased over this time period (*p* < 0.0001, Poisson regression). Cross validation found this trend to be positive and statistically significant for all 12-year or longer continuous subset time periods. The data can be seen in Table 2 and is illustrated in Figure 1.

We can observe that a salient change occurred over time on jobs related to climate change from Table 2. Overall, the total number of jobs increased since 2006, and the variability remained stable since then.


**Table 2.** Analysis of Data from Publichealthjobs.org/.net from 2003–2019.

**Figure 1.** Percent of job postings mentioning "Climate Change" or "Global Warming" in the PublicHealthJobs.org database.

### 3.1.3. Survey of Relevant Employers

As is often the case with surveys, the survey responders did not fully reflect the recipient population. In particular, government agencies and universities responded at a higher rate than the survey recipient population, while corporations, hospitals, and nonprofits responded at a lower rate (see Table 3). Comparing the survey recipients and respondents with those organizations that were actively posting positions in Indeed.com related to both climate change and public health, we can see that the populations were not quite the same; the Indeed.com search found a comparable percentage of corporations, a higher percentage of universities and nonprofits, and a lower percentage of government agency positions in comparison with the survey recipients and responses. Therefore, it is difficult to determine whether the survey is an accurate representation of the organizations currently hiring public health graduates.


**Table 3.** Survey recipients vs. responders vs. Indeed.com postings.

With this limitation in mind, we might still gather some conclusions. Fifty of Seventy three (68.5%) of the responders who answered the question, "Has your organization hired people with a Master of Public Health or PhD in Public Health in the past" responded "yes". Eighty six individuals responded to the question, "Do you expect the need to hire people with a background in climate and public health to grow in your organization in the next 5–10 years?" and of these, 33 indicated "yes", 34 "maybe", 6 "no", and 13 "don't know". Excluding the "don't know" responders, we could determine that 91.7% of respondents believed that there might be a need for public health and climate change-trained individuals in their organizations in the future.

In addition, an analysis of the thirteen-item Likert scale questions regarding skills, which would be useful to the employer organization, was conducted. See the frequency of responses in Figure 2.

**Figure 2.** Frequency of 13-item Likert responses for the question "Would any of the following skills be useful to your organization?" (%). R and SAS refer to statistical analysis software.

To standardize the questionnaire data, we considered the numeric data and text data separately. For numeric data, we found that the 13-item Likert scale response shows a high internal consistency of 0.879, which is described by Cronbach's alpha. From Figure 2, it is shown that no ceiling effect or significant floor effect was detected, suggesting it should be well-qualified as a valid measure of skill outcomes for public health employers.

A three-factor solution (all loadings ≥0.40) showed the best model fit to the survey data set. The Scree plot of the final exploratory factor analysis (EFA) solution is shown in Figure 3; we can observe that the eigenvalues of the model dropped below 1.0 when the component number reached 4, which is acknowledged as the rule of thumb cut-off point in deciding the internal structure. Thus, we set our final internal structure as a 3-factor EFA solution; this solution explained 70.16% variance by these three extracted factors and represents 13 items selected from the scale (only Likert Scale questions were included; text question and the "other" category question were filtered). In Table 4, all factor loadings were within the range of 0.456–0.928.

**Figure 3.** Scree plot of the final exploratory factor analysis (EFA) solution (three factors on 13 items).


**Table 4.** Pattern matrix of the EFA solution (three factors, 13 items).

Only two items had a cross-loading on more than one item with loadings >0.50 (Item 9 and Item 10), and we followed guidelines and discarded them in the final model. As shown in Table 4, the proposed model structure includes three dimensions and 13 items.

After the psychometric validation, we finalized the model with a 3-factor structure, with 11 items, and labeled them according to the theoretical context of each question. The first category was labeled as Population Health Exposure, included six items covering a range from population health analytical skills to general understanding of research methods, and also had a strong consistency of 0.83 (Table 5). The second category, in particular, targeted at the Climate-Related Knowledge and its intercorrelation with health status, showed a high consistency of around 0.9. The final category separates two Statistical Programming Language skills from other concrete skills, and included the two most popular statistical programming tools, R and SAS, which also retained a Cronbach's alpha value of 0.76.



The weighted sum score is calculated by using the weighted variance percentage, ranges from −1.34 to 0.63.

Open-ended comments in response to the question "What expertise or skills do you think will be needed to address the issue of climate change and human health in the next 10–20 years?" were coded using qualitative analysis methods, using categories identified by two of the authors (one with a background in higher education career services and the other with training in environmental health), and were independently coded by two research assistants to improve inter-rater reliability. Themes that emerged are listed in Table 6. Example quotes are included in Supplementary Material S2.


**Table 6.** Open-ended survey responses.

Survey question: "What expertise or skills do you think will be needed to address the issue of climate change and human health in the next 10–20 years?" (open-ended responses, coded).

### **4. Discussion**

The current state of the job market for public health graduates with training in climate change can be described as "emerging". From the Indeed.com job description data analysis, we can see there are relatively few roles—even in search results from a broad-based job board with keywords focusing on public health and climate change—currently available for a graduate with a master's level public health degree and a focus in climate change. Notwithstanding, it is likely that graduates would benefit from training in climate change-related competencies, even if the overt focus of their job is not directly related to climate change. Additionally, resonating with Wals, Corocoran, and others who frame educational institutions as change leaders, graduates with training in both climate change and public health can influence their institutions from within, to create systemic change in grappling with global warming.

The analysis of publichealthjobs.org data seemed to show that while jobs within public health that mention climate change or global warming were a very small proportion of the total, the fraction of such job postings had shown a statistically significant (*p* < 0.0001) increase over the last 16 years. This trend should be monitored by those involved in public health education and career placement of public health graduates, bearing in mind that while prior trends are often used to predict the future, they are not always the best indicator of future trends in a quickly changing world.

While "approaches for assessing, preventing and controlling environmental hazards that pose risks to human health and safety" [13] is not yet a core competency of public health degrees in the United States, the employer survey indicates that a large majority of respondents believe that there may be a growing need for graduates with training in climate change and health. The survey indicates that key skills include knowledge of climate mitigation, health equity and climate justice, an understanding of "downstream" effects of climate change, risk assessment, and technical skills in statistics, GIS mapping, and the carbon cycle. Comments from the responders indicate key themes focusing on these areas as well as communication (especially persuasive communication), finance/budgeting, cross-disciplinary collaboration and systems thinking, analytical skills, and an understanding of climate impacts on mental health, which resonate with Frankson et al.'s [53]. One Health Competency Domains including management, communication and informatics, values and ethics, leadership, team and collaboration, roles and responsibilities, and systems thinking. These skills also appear to be in alignment with

the competencies proposed by ASPHER, CAPHIA, and the Global Consortium on Climate and Health Education.

Importantly, the scope and framing of this study focused primarily on the role of educational institutions in preparing graduates to solve the problems of today, and to meet the demands of today's employers. Universities, however, not only provide education, produce research, and perform service to their communities; in addition, "higher education can play a pivotal role in turning society toward sustainability" [54]. This is an especially essential role in the face of massive and unpredictable global issues such as climate change. Universities create innovation, and can use their often privileged place in society to advocate for a sustainable future and to equip all of their graduates with understanding of their own environmental impact, both in the personal lives and in their careers. The challenges of climate change are profound enough to require an epistemological change; "sustainability is not just another issue to be added to an overcrowded curriculum, but a gateway to a different view of curriculum, of pedagogy, of organizational change, of policy and particularly of ethos" [54]. Additionally, following Scharmer's Theory U, we note that knowledge itself is not in short supply; instead, there is a "knowing-doing gap: a disconnect between our collective consciousness and our collective action" and our entire "mental and social operating system" must be upgraded from "ego-awareness to eco-awareness" [55]. Therefore, while this article focused on historical trends and current and near-term workforce needs to attempt to predict, shape, and model the need for public health students with training in climate change, the disruptive reality created by climate change likely cannot be modeled through such methods. Education should therefore help graduates develop new capacities, allowing them to deal with disruptions and lead a transformational change. The issues of sustainability are so far-reaching that it can be argued that educational institutions must reframe their full mission, using sustainability as their foundation.

### *Limtations*

There are several limitations to the analysis. Indeed.com may not capture all jobs; some jobs are never posted; and the US-focused part of the site was the only section of the site analyzed. A re-examination of these findings over a longer period of time would be helpful. The publichealthjobs.org database has a self-selection bias towards employers specifically recruiting for public health, though this is part of the reason this database was selected for analysis; and the number of job postings mentioning climate change or global warming was sparse, but is useful in indicating trends over time. The employer survey was distributed to a convenience sample of employers, with certain industries/sectors overrepresented and with a likely bias towards those in the United States (especially those based near New York City). While the response rate of 14% appears to be low, it is comparable with other employer surveys in the public health field, where studies have included rates as low as 13.4% [20] and 19.5% [23]. It is important to note, as those in public health have observed from responses to crises such as Ebola and Zika outbreaks [56], funding—and thus the need to use this funding to quickly hire highly trained public health professionals—can change quickly, if and when current events or policy priorities shift. Thus, prior trends (such as a 16 year retrospective analysis of job postings) cannot be assumed to be an accurate indicator of future job market growth. Finally, there is a need for further research in this area; competencies required for tackling climate change also require students and employers to identify and adapt to uncertainty and change, and universities have a special role to play in creating transformative change and disruption using their own critical analyses.

### **5. Conclusions**

Climate change is a growing threat to human health. While the current job market for candidates with training in both climate change and public health is relatively small, it appears to be growing; and it is likely that training in climate change competencies will increasingly benefit a range of public health organizations as climate change impacts continue to grow. Schools of public health can incorporate the skills and competencies related to climate change into their curricula and consider making them an integral/foundational part of the curriculum, if such training is not yet currently required. Employers, too, may benefit by taking note of the special intersection of skills and competencies offered by public health graduates with training in climate change-related issues. Graduates with such training can bring their paradigm-shifting lens to the work they do within any public health-related organization. Future research, including analyzing job postings, graduate employment outcomes, labor market projections, and employer surveys, could benefit curriculum development for educational institutions in countries around the world, and educational institutions could also remain at the forefront of the paradigm-shifting change that impacts the future public health workforce. By listening to the voices of current employers and assessing labor market trends, while also taking a wider view regarding the role of educational institutions in creating a sustainable world, these institutions can develop the skills and mindset needed to protect the public's health from emerging challenges such as climate change.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/1660-4601/17/4/1310/s1, Supplementary Material S1: Survey Questions: Climate and Health Jobs of the Future Survey; Supplementary Material S2: Example Quotes from Employer Survey.

**Author Contributions:** H.K.—conceptualization; conceived of the article, co-designed the survey, identified survey recipients, distributed the survey, contributed to qualitative analysis, gathered job postings data, coded the data, gathered labor market projections, and wrote and edited the article. J.S.—co-designed the survey, contributed to research design, edited the article. K.C.—contributed to article editing, research framing. S.K.—conducted literature review, coded/validated qualitative data. S.J.—coded/validated qualitative data, analyzed job postings data in R, conducted statistical analysis in SPSS and R. H.M.-assisted with coding/analyzing the survey responses, managed IRB process. J.K.—contributed to research design and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** We acknowledge Dean Linda Fried for her initial interest in this research topic.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **Readiness of Allied Professionals to Join the Mental Health Workforce: A Qualitative Evaluation of Trained Lay Trauma Counsellors' Experiences When Refugee Youth Disclose Suicidal Ideation**

**Sandra Löfving Gupta 1,2,\*, Katarina Wijk 1,2,3, Georgina Warner <sup>1</sup> and Anna Sarkadi <sup>1</sup>**

	- <sup>3</sup> Department of Occupational Health Sciences and Psychology, Faculty of Health and Occupational Studies, University of Gävle, 801 76 Gävle, Sweden
	- **\*** Correspondence: sandra.lofving.gupta@pubcare.uu.se

**Abstract:** The recent refugee crisis presented a huge challenge for the Swedish mental health workforce. Hence, innovative mental health workforce solutions were needed. Unaccompanied refugee minors (URM) are a particularly vulnerable refugee group. Teaching Recovery Techniques (TRT) was introduced as a community-based intervention utilising trained lay counsellors in a stepped model of care for refugee youth experiencing trauma symptoms. Professionals (e.g., teachers, social workers) can deliver the Cognitive Behavioural Therapy-based intervention after a brief training. A point of debate in this workforce solution is the readiness of trained lay counsellors to deal with potentially demanding situations like disclosure of suicidal ideation. This study aimed to explore the TRT trained lay counsellors' experiences of procedures upon URM's disclosure of suicidal ideation. Individual semi-structured interviews with TRT trained lay counsellors were conducted, then analysed using systemic text condensation. The analysis revealed four themes: "Importance of safety structures", "Collaboration is key", "Let sleeping dogs lie" and "Going the extra mile". Dealing with suicidal ideation is challenging and feelings of helplessness occur. Adding adequate supervision and specific training on suicidal ideation using role play is recommended. Collaboration between agencies and key stakeholders is essential when targeting refugee mental health in a stepped care model.

**Keywords:** workforce solution; mental health workforce; trained lay counsellors; unaccompanied refugee minors; teaching recovery techniques; cognitive behaviour therapy; group intervention; stepped care model

### **1. Introduction**

In 2016, Europe faced the largest single influx of refugees since World War II. This put a high demand on European countries to re-examine and find new sustainable solutions in various aspects of society, including the health care system. In Sweden, the country in the European Union with the highest number of asylum seekers per capita [1], the mental health service gap for this vulnerable group became evident. A substantial group of refugees were unaccompanied refugee minors (URM) [2], who still remain in Sweden. They have been described as the most vulnerable refugee group [3]. It has been formally acknowledged by the Swedish Social Services that existing psychiatric services do not meet the needs of this population, and an innovative mental health workforce solution is required to bridge the service gap [4].

**Citation:** Löfving Gupta, S.; Wijk, K.; Warner, G.; Sarkadi, A. Readiness of Allied Professionals to Join the Mental Health Workforce: A Qualitative Evaluation of Trained Lay Trauma Counsellors' Experiences When Refugee Youth Disclose Suicidal Ideation. *IJERPH* **2021**, *18*, 1486. https:// doi.org/10.3390/ijerph18041486

Academic Editors: Madhan Balasubramanian and Stephanie Short Received: 17 December 2020 Accepted: 1 February 2021 Published: 4 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

### *1.1. Population Need*

Numerous refugee children have been exposed to traumatic events like violence, threat, and separation in their country of origin and during migration [5]. URM, however, have a higher prevalence of traumatic events, such as torture, sexual abuse and kidnapping, compared to children fleeing with caregivers [6]. URM face the uncertainties of a complex asylum-seeking process and the stress of resettlement and acculturation hassles without the support of their caregivers [5,7]. Subsequently, trauma-related mental health problems, such as post-traumatic stress disorder (PTSD), depression, and anxiety, are particularly common among URM [8]. A recent study from Sweden [9] showed that 76% screened positive for PTSD, and a Norwegian study concluded that 43% met the criteria for a psychiatric diagnosis shortly after arriving at the host country [10]. Longitudinal studies confirm that these mental health problems are long-lasting [11,12]. PTSD diagnosis is associated with suicidality and this association is even stronger when there is comorbid depression. In Sweden, the rate of completed suicides among URM is almost 10 times higher compared to Swedish residents the same age [13]. A majority of URM live at residential homes, in absence of parents in the country, and are appointed a legal guardian. Immigration statistics indicate that the majority of URM who sought asylum in Sweden in 2015 were boys (86%) mainly from Afghanistan, Syria, Somalia and Eritrea between the ages of 13–17 [14].

### *1.2. Introducing a Stepped Care Model*

A potential solution to bridge the mental health service gap is to bring allied professionals, such as teachers, nurses or social workers, into the mental health workforce, acting as a "first line of defence" in a stepped care model. Stepped care is a system of delivering and monitoring treatments. Patients start with an evidence-based treatment of low intensity as a first step and those who do not respond adequately "step up" to a subsequent treatment of higher intensity [15].

In 2016, Teaching Recovery Techniques (TRT) was introduced to Sweden. TRT is a Trauma-Focused Cognitive Behavioural Therapy (TF-CBT) manualised group intervention aimed towards children and adolescents with symptoms of PTSD [16]. It was designed to meet the needs of low-resource and community settings. TRT comprises two sessions for caregivers and five sessions for youth and includes the following components: psychoeducation, affective modulation skills, cognitive coping and processing, in vivo mastery of trauma reminders, guided exposure and exploring plans and hopes for the future. The first two sessions focus on intrusion, the third on arousal and the final two sessions deal with exposure. Among others, techniques like positive self-talk, dual attention and relaxation are used. The focus is on symptoms and tools rather than trauma narrative and processing. In addition, normalisation of trauma symptoms in the group environment is assumed to relieve youth from shame and fear, whereas the safe environment provided by caring adults is geared to rebuild youth's trust in the adult world and provide social support. TRT in Sweden is delivered once a week during a seven-week period and two TRT facilitators co-host each session [17]. In Sweden, an introductory "getting to know each other session" prior to the core TRT sessions and a consolidating session at the end have been added. The purpose of the intervention and a brief overview of the content is discussed in the introductory session to help set expectations for the intervention. It is during this session that the TRT facilitators introduce themselves but there is no specific guidance upon how to do this. However, the importance of creating a safe environment in the group is stressed. When URM in Sweden were asked to describe their experience with the TRT intervention in a qualitative study the following six categories were revealed: social support, normalisation, valuable tools, comprehensibility, manageability and meaningfulness [9].

In order to become a TRT facilitator a three-day training workshop, run by the Swedish non-governmental organisation Children's Rights in Society, is mandatory. Another requirement is that the facilitator meets refugee youth in their professional capacity. Facilitators may be psychologists or counsellors, but also staff with no previous therapeutic experience

or specialist training in psychiatry are eligible to deliver the intervention after the workshop. Hence, some TRT facilitators could be referred to as "TRT trained lay counsellors" as they indeed have training in TRT but lack training in mental health or counselling albeit they might have professional training in other domains, such as teaching, nursing or social work.

Given its brevity (seven weekly sessions), group format and delivery by community professionals, TRT offers several potential economies over individual therapy. Therefore, if a strong evidence base for TRT effectiveness can be presented, it is logical that the intervention would form a valuable component in a stepped care model (see Figure 1). International studies have reported high acceptability and large effect sizes for decrease in symptoms of both depression and PTSD [18,19]. An exploratory trial of TRT with 46 URM in Sweden (mainly male, ages 13–18) showed a significant decrease in reported symptoms of depression and PTSD [9]. There is an ongoing nationwide randomised control trial targeting URM in Sweden to further strengthen the evidence-base; as well as investigating the overall effectiveness, this trail aims to assess effectiveness at the subgroup level [17]. The URM are screened in schools, support groups and at residential homes using the Children's Revised Impact of Events Scale (CRIES-8), where a score of 17 or above indicates high symptom burden and risk of PTSD [20,21] which makes them eligible for TRT.

**Figure 1.** Placement of teaching recovery techniques in a stepped care model for mental health provision.

Yet, a point of debate is the readiness and capability of allied professionals, such as teachers, nurses or social workers, to take on potentially demanding aspects of mental health intervention such as safety processes relating to suicidal ideation. Although a growing number of studies concerning CBT-based PTSD treatments delivered by lay counsellors [22–24] indicate feasibility and short-term effectiveness, there has been little exploration of trained lay counsellors' experiences of safety procedures.

### *1.3. Safety Aspects and the Potential Vulnerability of TRT Trained Lay Counsellors*

Although TRT is not developed for a specialist health care setting, a substantial number of URM within the TRT program report severe, high-risk psychiatric symptoms. A pilot study (*N* = 55) showed that in addition to posttraumatic stress symptoms, 83% of URM receiving TRT suffered from moderate to severe depression and 48% displayed suicidal ideation or plans [9]. A safety protocol is recommended when dealing with people at risk. Safety protocols may vary depending on location, resources and infrastructure; however, it should clearly state how to assess risk, identify warning signs, implement safety planning techniques and when and how to refer the person at risk [25]. The questionnaires used in

the TRT safety protocol have changed over time, with the Montgomery–Asberg Depression Scale MADRS [26] later replaced with the Patient Health Questionnaire PHQ-9 [27] and the Columbia Suicide Severity Rating Scale Screener, C-SSRS screener [28] introduced as a structured way to discuss suicidal thoughts, yet the overall process has remained the same (see Figure 2).

**Figure 2.** Teaching recovery techniques safety procedure.

Although the TRT facilitators are not expected to do a full suicide risk assessment or to have knowledge about risk factors for suicide, as this responsibility lays within the Child and Adolescent Mental Health Service (CAMHS), they still need to ask potentially difficult questions about suicide and help decide whether the legal guardian should contact CAMHS or not. This may leave the trained lay counsellors, with no formal training in mental health or training on how to address suicidal thoughts, in a vulnerable and potentially risky situation. Although it is important to find new health care solutions to address a need in the community, in this case by task shifting from mental health professionals to allied professionals, it is essential that the personnel are adequately prepared for the task. Previous studies regarding lay counsellors delivering trauma-focused therapy have shown that training and practicing as a lay counsellor can enhance self-esteem and lead to empowerment, whereas others highlighted the particular stress, risk of overinvolvement and even risk of indirect traumatisation lay counsellors face [29–32]. However, these studies are conducted in low-income settings and in some cases the lay counsellors are recruited within the target community of the intervention and were not sufficiently trained. The importance of regular supervision has been raised by lay counsellors working with mental health interventions in previous qualitative studies [23,33].

Dealing with suicidal ideation and conducting suicide risk assessments is a complex and challenging task that even trained mental health personnel struggle with and feel anxious about [34,35]. It is of great importance to learn about the experiences and potential struggles of lay counsellors when they are exposed to someone with suicidal thoughts or plans. This knowledge is crucial to give adequate support and training to the lay counsellors and, ultimately, to ensure the persons signalling suicidal communication are given adequate care.

Hence, the aim of this study was to examine how TRT facilitators, without formal training in mental health or counselling i.e., TRT trained lay counsellors, experience the safety procedure when participating unaccompanied refugee youth disclose suicidal ideation.

### **2. Methods**

### *2.1. Data Collection*

National recruitment of TRT facilitators was conducted through email within the Children's Rights in Society network of operating TRT facilitators (*N* = 50). The study invitation email explained the purpose of the study and the inclusion criteria: TRT facilitators without therapeutic training or formal education in psychiatry (TRT trained lay counsellors) and who had experienced URM disclosing suicidal ideation during the TRT group session.

Recruitment continued until saturation was reached and no new information was observed in the data. Ten interviews, using an interview guide (Table 1), were conducted using different forms of communication; face to face, via Skype and by phone and varied in length between 35–45 min. Of these ten interviews, two were excluded from analysis. The first was a pilot interview and the other was excluded since the TRT facilitator had formal training in psychotherapy, contrary to inclusion criteria of no formal mental health training. All interviews were conducted by the first author who is a trained TRT facilitator and a child- and adolescent psychiatrist.

**Table 1.** Interview guide.


The included respondents (*N* = 8; 7 females and 1 male) were from all over Sweden, both rural and urban areas. They included two nurses, five social workers and a child welfare-officer. They had facilitated between 1–10 TRT groups each, with an average of three groups.

### *2.2. Ethical Considerations*

After a pilot interview, it became evident that, although the respondents were interviewed in their professional capacity, which would not normally need ethical clearance according to Swedish legislation, the respondents' disclosure was of such sensitive character that we reconsidered and obtained ethical approval (dnr 2019-01427), excluding the pilot interview from further analyses.

### *2.3. Data Analysis*

All interviews were transcribed verbatim and analysed using Systematic Text Condensation (STC) as described by Malterud [36] (see Figure 3 for a description of the analytic process of STC). STC was chosen for analysis for its descriptive and inductive approach that presents the experiences as described by the participants, rather than searching for underlying meaning. First, all the transcripts were read and re-read by all authors to obtain a full comprehension of each case. This was followed by the process of decontextualisation, in which, recurrent themes regarding the respondents' different experience of dealing with URM who disclosed suicidal thoughts were recognized. Meaning units were then identified and grouped, these were subsequently sorted into categories describing different aspects of the themes. The content of each category was summarized and expressed as a single statement. The analytical text was formulated and quotes illustrating the different

categories were selected. Finally, the results were validated by rereading all eight original transcripts to see whether the themes and code groups had goodness-of-fit.

**Figure 3.** Analytical process of systematic text condensation (STC).

### **3. Results**

Four themes emerged during the analysis. The themes and their corresponding categories are presented in Table 2.

**Table 2.** Overview of themes and categories.


### *3.1. Theme 1: Importance of Safety Structures*

The TRT lay counsellors recognized the significance of having pre-existing routines at the workplace but they also emphasized the importance of finding comfort in a colleague, someone at the workplace in whom they could confide when meeting URM who disclosed suicidal thoughts. Moreover, having established a safety protocol, a step by step guide on what actions to take when a URM discloses suicidal thoughts, was also considered helpful.

### 3.1.1. Established Safety Routines at the Workplace

TRT lay counsellors working in a medical setting valued the well-defined safety structures and routines that were already established at the workplace. They stated that having a doctor available at the workplace, who was able to do a suicide assessment and refer to mental health services when needed, was very valuable. Although the doctor might not be available the same day as the URM made the disclosure, the mere knowledge that an assessment would be made by someone else was described as a relief. In some workplaces an experienced counsellor was available to assess and have regular individual follow ups with the URM in need. However, the social workers often did not have established safety structures in place as they did not have access to anyone with formal mental health training at their workplace nor did they have established pathways for referral when suicidal ideation was disclosed.

"I don't feel that we overlook anything. Since we are at a healthcare centre. I feel that we already have a safety informed way of thinking ... we are prepared to deal with this kind of (pause) [suicidal communication]. We haven't felt scared that something will happen."

(Interview 3)

### 3.1.2. Comfort in Colleagues

The TRT lay counsellors also emphasised the importance of having a close and trustful relationship with a colleague, either the co-TRT facilitator or someone else at the workplace. When reflecting on features of this colleague the TRT lay counsellor did not attribute formal training in suicide assessments as a prominent feature. Rather, that the colleague was someone the TRT lay counsellor could confide in or someone who expresses confidence in situations dealing with suicidal disclosure.

"I have learnt so much by facilitating these groups. I don't think I would have wanted to run a group with someone I did not feel secure with. The most important thing is that we know each other and that we can get through this together."

(Interview 3)

The opposite was also highlighted, that not knowing the co-facilitator beforehand and the lack of trust and co-operation between the TRT facilitators was considered particularly stressful when meeting URM who had disclosed suicidal thoughts.

### 3.1.3. Working with the Safety Protocol

Having a safety protocol was viewed as beneficial when dealing with URM who had disclosed suicidal ideation. TRT lay counsellors explained the safety protocol gave them structure and confidence. However, they also stated the safety protocol did not take into account the complexities involved in asking questions about suicide. They recall that even though they asked the suggested questions regarding suicidal thoughts/plans the answers were not clear, one URM even refused to answer the questions.

"It felt really good to have a safety protocol, it felt good to have something to lean against. It felt like a backbone. It was something we facilitators talked about beforehand. But in reality, it wasn't as straightforward. There were many factors we couldn't control . . . "

(Interview 4)

The more experienced TRT lay counsellors, who had seen different versions of the safety protocol, implied the earlier versions were too sensitive to suicidal ideation. In fact, one TRT lay counsellor reported that she consciously did not follow the initial protocol as she knew the URM and was not worried about him harming himself although he scored high on the screening.

### *3.2. Theme 2: Collaboration Is Key*

Collaborating with CAMHS and sharing the same understanding about TRT as key persons in the URM's network such as legal guardians and personnel at residential care homes was considered essential when dealing with URM who had disclosed suicidal ideation.

### 3.2.1. Closed Doors to CAMHS

The TRT lay counsellors' expectations and experiences of collaborating with CAMHS varied. Some reported their previous negative experiences led them to be concerned about this collaboration even before the TRT group started.

"That was something I thought about even during the training, you know, if we find someone with suicidal thoughts, will CAMHS agree to see them? Or will it just be nothing, limbo? And will I have time to handle it, if it ends in limbo?" (Interview 1)

A number of TRT lay counsellors reported actual experiences of not receiving the help and care they had needed from CAMHS. There was a sense that CAMHS normalised the URMs' symptoms and suicidal thoughts or did not offer an assessment as urgently as the TRT lay counsellor expected. This left the TRT lay counsellor with a sense of being alone and vulnerable.

"I felt that we did not get an adequate response [from CAMHS]. They did not take it seriously, like 'this kid has gone through so many horrible things that he is expected to have suicidal thoughts'. They just normalised it and that was not good . . . it just ended there . . . and we were left to take care of it somehow." (Interview 4)

Although some TRT lay counsellor did not report any specific encounter with CAMHS it was clear they had low expectations of what specialist mental health services could offer. One TRT lay counsellor even recommended the legal guardian to seek help within primary healthcare instead of CAMHS due to prior problems within CAMHS.

"There have been huge problems with CAMHS in my city ... it was chaos actually. Enormous waiting lists and on top of that it's my experience that they don't deal with things. So, yes, I have very low expectations."

### (Interview 2)

On the other hand, there were also TRT lay counsellors who shared a good experience of collaborating with CAMHS and felt adequately helped by CAMHS.

"I actually felt like they did a good job, it might take some time ... But if someone is really unwell it might go faster ... We had two or three who were in a really bad state and CAMHS agreed to see them."

(Interview 5)

### 3.2.2. Realising a Shared Understanding among Key Persons

The experiences of collaboration with legal guardians and personnel at residential care homes varied. Some facilitators experienced difficulties in this collaboration, for instance, practical difficulties to get in contact with the legal guardian when the URM had disclosed suicidal intention, which left the TRT lay counsellor with a huge responsibility.

"Some legal guardians were really good but it did not always work out well. They were difficult to get hold of. Maybe they did not fully understand their role? But some were really engaged. One legal guardian asked if some other teenagers that she knew could join the group. I think they appreciated that we actually did something."

### (Interview 6)

Some legal guardians and personnel for residential homes were openly opposed to and critical of TRT, since they felt TRT could potentially be harmful. One TRT lay counsellor mentioned that differences in educational level between the TRT lay counsellors and the personnel for resident care home led to misunderstanding, which was considered an obstacle.

"Some of the legal guardians had a negative reaction to what we did. Like, okay, now you have had this thing and you are just dumping it all on us. What are we supposed to do with this? We met the same reaction from the ones working at the residential care home. They felt that we stirred up too much among the teenagers and maybe we did? That we caused more harm than good."

### (Interview 8)

On the other hand, there are also examples of excellent collaboration where the key persons have been fully onboard, supportive and played a crucial role in for instance helping URM to practice TRT skills outside the sessions.

### *3.3. Theme 3: Let Sleeping Dogs Lie*

Despite the safety protocol, some TRT lay counsellors felt insecure and personally responsible for discovering suicidal thoughts. Several TRT lay counsellors did not feel equipped to meet URM with mental ill health and suicidal ideation and there were TRT lay counsellors who wondered whether the intervention actually might have a negative effect on the URMs' wellbeing.

### 3.3.1. Navigating the Boundaries of Responsibility

The safety protocol clearly stated the TRT lay counsellors role when participating URM disclosed suicidal ideation. Yet, some TRT lay counsellors described that they felt personally responsible for being the one who discovered the suicidal thoughts and it was unsatisfactory for them to refer the URM to someone else. They expressed a need to go beyond the safety protocol.

"Of course, I sometimes felt like I should have been the one to be there for them. You know, instead of just leaving it to someone else. Do you see what I mean? That I wanted to see it through all the way. And we couldn't do that ... We left these people when we should have been the ones who made sure everything turned out okay."

(Interview 5)

### 3.3.2. Readiness to Talk About Suicide

Some TRT lay counsellors worried that they might lack the competence to assess the URM mental health and that they might disregard something important that the URM was signalling non-verbally. There were times when they were worried a participant actually might harm themselves or even commit suicide and they reflected on their personal responsibility, if something like that would happen.

"I can't really pinpoint what I felt I needed, maybe more experience ... I don't know if I needed more experience of working with suicidal youth or just more life experience."

### (Interview 4)

"But it became easier with time, with more experience ... you learn from your mistakes. But of course, sometimes I was really worried, what if they were to jump in front of a train? I would have needed someone to talk to there and then, and that was difficult."

### (Interview 6)

Others did not feel any discomfort or threat by the idea that asking questions about suicide might evoke strong negative emotions. Although they had not had any formal training in suicide assessments they felt prepared and safe to ask questions about suicide.

"I believe that many adults are afraid of asking questions about suicide because they think that it is better to let sleeping dogs lie. I thought that it was liberating to realize that it wasn't the case. On the contrary, when you dare to address it the teenager actually reveals their thoughts. That is something every grown up who encounters teenagers needs to hear."

(Interview 7)

### 3.3.3. Notion of Doing Harm

Some TRT lay counsellors expressed worry and apprehension about the negative emotions that might be triggered by talking about traumatic memories. Sometimes they were challenged by a particular URM's resistance to talk about trauma. One TRT lay counsellor reflected over that, even though she knew the importance of the exposure session, she felt inhibited to fully implement since she was not sure she could manage the emotions the exposure might trigger in the URM. However, the feeling of doing potential harm seemed to be greatest when conducting their first TRT group, as TRT lay counsellor describe this feeling decreased with time.

"I remember thinking many times, this is too much for me. What have I gotten myself into? ... So, I thought, oh my God what kind of processes are we starting? Are we saying things no one else has said? We are talking about stuff and reviving their memories. What am I supposed to do with that? I don't know exactly what I felt, maybe powerless?"

(Interview 4)

Others did not feel any discomfort or threat by the fact that the TRT session might evoke strong negative emotions. On the contrary, they normalised it and viewed it as a part of the process.

"Personally, I have never been afraid of meeting people with mental ill health on the contrary I find it quite interesting to see whom they choose to tell their story to."

(Interview 5)

"I really believe that it is crucial to label things for what they are. I do not hesitate to say the difficult words."

(Interview 1)

### *3.4. Theme 4: Going the Extra Mile*

TRT lay counsellors described the need for a manual-based intervention for URM whom they had identified as a particularly vulnerable group in society. They also related to their own desire (and struggle) to find meaning and to be creative and flexible in an unstable and chaotic situation in order to deliver TRT.

### 3.4.1. Motivated by a Structured Way of Addressing a Need

The TRT lay counsellors had identified URM to be a vulnerable group in need of coping strategies and knowledge about trauma and post-traumatic stress. They expressed frustration that URM did not access proper treatment elsewhere and craved an intervention that was hands-on and manual based. This was emphasized as an important motivational factor for joining the TRT training.

"I really felt that I needed to do something for this group because, you know, people said that there is nothing we can do as long as they are asylum seekers

. . . it is better to do something than nothing."

### (Interview 6)

At the same time, a few TRT lay counsellor explained that facilitating TRT groups simply was a part of their job description and they had been asked to do the training by their supervisors and managers, rather than by their own identified need or conviction.

### 3.4.2. Fitting into the Chaos

Meeting URM in a situation of crisis, where a number of basic needs such as shelter and food are not being met, placed high demands on the TRT lay counsellor and was sometimes challenging for them. Yet, the TRT lay counsellor have shown signs of both creativity and flexibility as they described various ways and strategies in which they sought to increase motivation, session attendance and to facilitate the URMs' ability to fully engage in the sessions. This could entail seemingly small gestures like offering food during the sessions or to make wake-up phone calls to URM with sleeping difficulties. However, TRT lay counsellors reflected this "extra care", outside the manual, felt important not only for the URMs' wellbeing, but also for the lay counsellors themselves as it gave them a sense of accomplishment and meaning in a situation of chaos.

"Sometimes you need to go beyond the manual. You need to make sure that they come despite the fact that they are homeless. You need to offer fellowship, food and laughter. You need to try. It was difficult but you need to find meaning because, you know, we could offer them something."

### (Interview 1)

TRT lay counsellors also expressed great empathy and distress over the URMs' vulnerable and often uncertain life situation relating to the stress of being in the asylum-seeking process. A few TRT lay counsellors questioned whether TRT and addressing trauma actually was suitable in this situation of uncertainty.

"They were mostly guys from Afghanistan ... And they didn't know if they were going to be granted asylum in Sweden. So, they were in the middle of the asylum-seeking process. This was a major thing for them, if they were going to be able to stay or not. So, this was the dominant thing for them—not thinking about trauma . . . I feel that the timing of this intervention was wrong."

(Interview 8)

### **4. Discussion**

This study explored how TRT trained lay counsellors, without formal training in mental health or counselling, experienced dealing with URM disclosing suicidal ideation.

Although some TRT lay counsellors felt anxious and overwhelmed by the disclosure of suicidal ideation, others were surprisingly confident. Dealing with suicidal disclosures seems to be a challenging task regardless of professional training. A recent qualitative study among psychiatrists in Sweden describes feelings of fear, anxiety, uncertainty and even physical reactions in relation to suicide risk assessment [34]. Although the TRT lay counsellors' role regarding suicidal disclosure is not comparable with the role of a psychiatrist conducting a full suicide risk assessment, the experience of uncertainty, fear of making the wrong decision and sense of responsibility unites them. However, an interesting difference is while the TRT lay counsellors reflect over the boundaries of their moral responsibility, the psychiatrists also reveal being burdened by formal responsibility and the fear of malpractice litigation when assessing suicide [34] from which the TRT lay counsellor are spared. One may speculate that formally trained personnel with a legislative duty to, for instance, keep health records, might experience suicidal disclosure with regard to their professional responsibility differently than the lay counsellors.

While the TRT lay counsellors acknowledged the importance of safety structures and there was a general appreciation of the safety protocol and procedures, asking questions about suicide was not always straightforward. The TRT lay counsellors experienced that, despite having followed the instructions in the safety protocol, the URM refused to answer questions about suicide or that the URM was emotionally blunted which made therapeutic alliance difficult. There are several reasons for not disclosing suicidal thoughts, such as lack of trust, fear of hospitalisation, judgment or causing distress for the person asking the questions or even lack of empathy in the person asking the questions [37,38]. Context and proper training in giving rationale for asking questions about suicidal thoughts are important.

Furthermore, establishing interpersonal trust and setting aside screening questionnaires to strengthen therapeutic alliance has been positively correlated to greater overall disclosure [37]. Struggles to obtain suicidal disclosure is not unique to utilisation of lay counsellors or the community setting, it is also found in a therapeutic setting with formally

trained and experienced therapists [39]. Similar challenges of issues related to lack of emotional contact and credibility has been reported by trained psychiatrists and adding additional training on understanding non-verbal signs that may signal increased suicide risk has been suggested [34].

The readiness to talk about suicide varied among the TRT lay counsellors. Some reported feelings of insecurity when dealing with suicidal disclosure and suggested lack of experience (referring to both work and life experience) as a possible explanation. This is in line with a Swedish study among trained personnel working within mental health, concluding that job clarity and confidence regarding their role with suicidal patients as well as attitude towards suicidal prevention was connected to work experience as well as perception of having received sufficient suicidal prevention education [40]. Adding roleplay to training e.g., has shown both reported and observed improvement in communication with youth in distress and directly asking questions about suicide [41] in the context of a community-based suicide prevention intervention. Enhancing the present TRT training with rationale for asking questions about suicidality, training on non-verbal signs and roleplay on asking questions about suicidality could be beneficial and evaluated in future research.

The URMs' situation, their trauma narrative and helplessness in the asylum-seeking process were demanding for some TRT lay counsellors who felt great empathy but also overwhelmed and sometimes helpless themselves. However, they were also motivated by this challenging situation and experienced meaning in helping the specific URM reduce trauma symptoms and by being a part of influencing social injustice as well as addressing need. Even among experienced trauma counsellors, "providing assistance to others" both at a personal and societal level has been described as rewarding and as important factors to thrive as a trauma therapist [42]. Experienced trauma therapists also reflect over the need to modulate their own empathy and by setting boundaries and accepting the counselling intervention and their own limitation [42]. Supervision is important to support lay counsellors [23,43]; however, adequate supervision should include both management of clients and specifically inquiring about counsellors' own emotions to address the particular risk of indirect traumatization [44]. A future research direction could be to design and evaluate a dedicated supervision program for TRT facilitators.

The TRT trained lay counsellors also revealed concerns regarding the tolerability and safety of exposure and the strong negative emotions exposure evoked. This fear of doing potential harm is not limited to lay counsellors. A study among 600 mental health workers disclosed similar concerns regarding exposure although some of them used exposure in their practice [45]. Despite the strong evidence of the efficacy of exposure, even trauma experts are afraid of exposure causing symptom exacerbation and drop outs, leading exposure techniques to be under used [46,47]. Though the TRT lay counsellors received brief training in the rationale for exposure and how to conduct exposure, adding enhanced emotion-based training targeting attitude change by identifying concerns about exposure and adding video-based client testimonies may reduce concerns and enhance delivery of exposure [48]. Given the strength of opinion coming through regarding exposure, more thorough investigation of this particular topic could be warranted.

The need for a shared understanding with key stakeholders was identified by the TRT lay counsellors, as some legal guardians and personnel from residential care homes questioned the need and purpose of the intervention as they feared the URM might experience more trauma symptoms due to the intervention. However, the TRT lay counsellors who reported most apprehension and resistance from other stakeholders also reported that they themselves became TRT facilitators due to the will of their managers rather than their own conviction of a need for an intervention. One might speculate that this group of lay counsellors were more sceptic to the intervention and therefore not as well equipped to explain the benefits or reduce misbeliefs regarding the intervention to the stakeholders. Despite efforts made to reduce the knowledge gap and stigma regarding psychiatric treatment in Sweden, a study on change in public attitudes regarding mental health concluded that appreciation

of treatment of mental illness and psychiatric care remains low [49]. Hence, facing negative beliefs about mental health care is common and not unique to lay counsellors.

The TRT lay counsellors were also concerned about URM not being assessed or admitted to CAMHS. This perceived "closed door" to CAMHS could in part be due to a debated belief that trauma treatment should not commence in an unstable setting, traditionally excluding asylum seekers from accessing trauma treatment in specialist care [50]. Although CAMHS are obliged to conduct suicide risk assessments on asylum seekers, the way the profile of the population interacts with the existing service model of care is a potential source of friction in collaboration. This lack of well-functioning collaboration and accessibility to CAMHS left the lay counsellors with a sense of loneliness and vulnerability. This structural problem is in-line with a previous study emphasising "building relationships between agencies" and increasing accessibility to mental health services for refugee children are crucial for increased service utility [51] and needs to be addressed at a health governance level. There needs to be more time dedicated to identifying ways in which collaboration could be enhanced. One could look to the interdisciplinary collaboration literature [52] to look for attributes to target for instance, effective communication channels between CAMHS and TRT facilitators, shared accountability and building trust. The effectiveness of these working models in improving collaboration would need to be evaluated.

### *Methodological Considerations*

The first author is a child and adolescent psychiatrist with long experience of meeting URM and conducting suicide assessments at CAMHS. She is also a trained TRT facilitator and had, at the time of the interviews, conducted one TRT group. Although this background gave her great knowledge and experience in the field of psychiatry and suicide assessments, throughout the study there, has been an awareness of how this might impact the interviewed TRT lay counsellors, i.e., researcher reflexivity. There was an initial concern the TRT lay counsellors might feel intimidated or judged, less likely to reveal own limitations or less likely to speak freely about negative experiences of collaborating with CAMHS. In order to reduce her potential role as an "expert", or as a spokesperson for CAMHS, this issue was addressed before the interviews and by making extra efforts to create an interview environment that promoted trust and openness. In addition, the purpose of the study was stressed repeatedly and participants encouraged to speak freely and honestly.

To further strengthen credibility, a semi-structured interview guide was used. To promote transferability, interview data were collected from TRT lay counsellors with different occupations, working in different geographical areas in Sweden and a difference in number of conducted TRT groups. Furthermore, transferability was also promoted by describing both typical and atypical views expressed by the TRT lay counsellors within each theme, i.e., negative case analysis. Although the number of interviews could seem low, saturation was assessed, and all authors were in agreement that it had been reached. Dependability and confirmability were enhanced by having a clear research trail during the entire process of analysis and involving all authors in the analysis. Reflecting on possible preconceptions was an essential part in this process.

Limitations: This study adopted a qualitative methodology that intended to investigate experiences in the particular context of Sweden, hence generalizability of the findings to other international contexts was not an expected attribute; the structure of health and social care in the local context would need to be considered. The study also specifically addressed the target group of URM and not refugee adolescents in general. It is possible that knowing there is a parent available to contact and discuss with might alter the experiences of the lay counsellors. Finally, questions about suicidal ideation followed initial screening and were part of a safety protocol. Thus, the study does not cover situations where disclosure is spontaneous, not backed up with a plan. It is likely that those situations cause more apprehension, anxiety, and insecurity in lay counsellors. Future studies could therefore expand the scope of target groups to refugee or otherwise vulnerable adolescents and counselling types, to investigate how lay counsellors deal with disclosures of suicidal ideation.

### **5. Conclusions**

Dealing with suicidal disclosure is a complex and challenging task regardless of training. Both lay counsellors and experienced mental health workers struggle with feelings of uncertainty, helplessness and boundaries of responsibility; however, lay counsellors seem to be exempt from fear of professional repercussions. The motivations for becoming a TRT facilitator might also be interacting with their perceived experiences. Adding specific training on how to address suicidal throughs and talk about suicidal ideation using roleplay is recommended. Adding "attitude change" based training specifically challenging the concerns regarding exposure could be valuable, as well as adequate supervision advising on management but also targeting the lay counsellors' own emotions is recommended. Finally, collaboration with key stakeholders and building relationships between agencies is essential to facilitate working with refugee mental health in a stepped care model and new working models based on interdisciplinary collaboration is recommended. Another potential obstacle for collaboration regarding this particular population could be conflicting views about the timing of trauma treatment, which needs to be addressed. Overall, although there is room for improvement in training and collaborative working, incorporation of allied professionals in the mental health workforce appears to be a workable solution to the mental health needs of URM in Sweden.

**Author Contributions:** Conceptualization, S.L.G., K.W., G.W. and A.S.; Investigation, S.L.G.; Methodology, S.L.G., K.W., G.W. and A.S.; Supervision, K.W., G.W. and A.S.; Visualization, S.L.G.; Writing original draft, S.L.G.; Writing—review and editing, K.W., G.W. and A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research is funded by the Kavli Trust (Grant: ID: A-321629). The funder has had no involvement in the design, data collection or analysis of the study or the writing of the manuscript.

**Institutional Review Board Statement:** Ethics approval for this study was obtained from The Swedish Ethical Review Authority (dnr 2019-01427).

**Informed Consent Statement:** Both oral and written consent was obtained from all participants regarding both participation and publication.

**Data Availability Statement:** The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Article* **Transforming the Future Healthcare Workforce across Europe through Improvement Science Training: A Qualitative Approach**

**Maria Cristina Sierras-Davo 1,\*, Manuel Lillo-Crespo 1, Patricia Verdu <sup>2</sup> and Aimilia Karapostoli <sup>3</sup>**


**Abstract:** Healthcare improvement science (HIS) is the generation of knowledge to cultivate change towards improving health systems performance. Our purpose was to evaluate the experience of European nursing students after an intensive one-week summer program conducted in 2019 at the University of Alicante in Spain. The educational intervention combined theoretical and practical HIS contents, with students from different countries, educational programs, and health systems. The intervention was evaluated under a qualitative approach through the open discussion group technique based on the method of participatory action research (PAR), with a total of 25 students who reflected about their experiences and perceptions during the intervention. The responses were used to improve the program's contents, its didactics, and organization. Nursing empowerment, professional recognition, and healthcare research were some of the seven main categories identified through the systematic content analysis method triangulated by three experienced researchers. According to the students' replies, values like compassion, respect, or empathy were identified as key elements of care. Promoting international students' networking emerged as the key to creating a positive provision for change and the generation of improvement initiatives. Building a HIS culture may potentially provide future healthcare professionals with critical thinking skills and the resources needed to improve their future work settings.

**Keywords:** Europe; thinking; improvement science; nursing students; qualitative research

### **1. Introduction**

Over the period 1999–2010 the Bologna Reform in the European Union highlighted the importance of value-centered education across Europe in the field of health studies. In line with this, patient safety should be of upmost importance for healthcare professionals, while fundamental values like compassion, integrity, or human dignity, among others, are key to delivering the highest level of quality of care. However, those values are still not widely included in the training process of healthcare professionals in Europe and are not observed as part of improvement initiatives in the educational and healthcare fields [1–6].

From 2013 to 2015 the Improvement Science Training for European Healthcare Workers (ISTEW) project funded by the European Commission evidenced the gap in the provision of accredited health improvement science (HIS) education across Europe and outlined the need to improve quality of care services and related education. The most representative ISTEW outcomes were (a) the European HIS consensus definition, known as the Bled definition, (b) four HIS training modules, and (c) the Healthcare Improvement Science Evaluation Framework (HISEF) [7–9]. The Bled definition defines HIS in the European context as "the generation of knowledge to cultivate change and deliver person-centered care that is safe, effective, efficient, equitable and timely. It improves patient outcomes, health system performance, and population health" [9]. However, HIS status and understanding in other

**Citation:** Sierras-Davo, M.C.; Lillo-Crespo, M.; Verdu, P.; Karapostoli, A. Transforming the Future Healthcare Workforce across Europe through Improvement Science Training: A Qualitative Approach. *IJERPH* **2021**, *18*, 1298. https://doi.org/10.3390/ ijerph18031298

Academic Editors: Madhan Balasubramanian, Stephanie Short and Jayasree Basu Received: 6 December 2020 Accepted: 28 January 2021 Published: 1 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

non-European countries such as the United States (U.S.) remain different. Since the 1980s improvement science has been developed extensively, focusing on health outcomes from an economic and efficiency perspective. In the U.S., the Institute of Health Improvement (IHI) has been focused for decades on the creation of specific improvement education, its implementation in healthcare contexts, and dissemination in their healthcare system [8]. Across the European countries, differences among HIS understanding and practice have been evidenced. In fact, a higher level of development is observed in the English-speaking countries such as the United Kingdom and Ireland. In the European educational field specifically, the differences are even more evident. For instance, in Slovenia only 4% of the European Quality Assurance Register for Higher Education (EQUAR) courses include HIS contents, a figure similar to that of Italy (7%), followed by Poland (10%), and far behind England (27%) and Romania (25%) [8].

As stated before, based on the previous gap analysis of nursing studies in Europe conducted during the ISTEW Project there is a lack of specific training for nurses focused on the following items: development of improvement-based and critical thinking, quality improvement measurements, systems thinking, and safety practices [8]. Therefore, those items were the ones upon which the ISTEW modules and the contents of the Alicante Summer Program were based. The University of Alicante in Spain, as a partner team, promoted HIS culture and prospectively used the ISTEW outcomes by organizing an Annual International Summer Program. The "Immersion in HIS" course started in July 2016 and was repeated yearly until 2019 [7,8]. Participants were nursing students from different European universities (Scotland, Ireland, Finland, and Spain), and were therefore from different cultures, with distinct types of health system organization and professional competencies. Such international education led to a discussion on how value-centered healthcare education focusing on HIS should be considered, while analyzing the differences and similarities amongst cultures. Students had the chance during the training to propose improvement initiatives in their own real contexts and discuss what other colleagues from other cultures were doing [1]. Along the four Summer Programs, the HIS Evaluation Framework (HISEF) created throughout the ISTEW project was used as the evaluation tool which included participants' qualitative and quantitative data through different questionnaires based on Kirkpatrick's Learning Evaluation Model [10–12]. To support the data collected through the evaluation framework, new dynamics were introduced in 2019. The research presented focuses on this new section where qualitative data were collected after exploring the experience and perception of European nursing students regarding HIS after an intensive one-week summer program.

### **2. Materials and Methods**

### *2.1. Educational Intervention, Qualitative Method, and Techniques*

A practical and theoretical educational intervention regarding HIS was conducted consecutively from 2016 to 2019, focusing on the four main HIS modules developed by the ISTEW project: (a) the development of improvement-based and critical thinking, (b) quality improvement measurements, (c) systems thinking, and (d) safety practices. For our research purpose we concentrated on the qualitative data collected in the 2019 course. The educational intervention was evaluated under the scope of participatory action research (PAR), which was selected as the qualitative method. Within PAR, the subject becomes the protagonist and participates in the change itself. Citing Cassell and Symon [13], PAR enables participants to confront their experiences and existing conflicts with others, particularly in healthcare provision to the patients. The transition from object of study to subject protagonist is carried out by cyclic processes of reflection–action–reflection where the researcher continuously evaluates each intervention, interacting constantly with the target study population [14]. We understood that the inclusion of all the course users and all the educators participating in this intervention would determine the success of the implementation of HIS knowledge in the future healthcare workforce [15].

Although we used the HISEF as the evaluation tool of the HIS learning, which included open-ended and closed questions together with Likert scales, we considered this insufficient for our qualitative goal. For that reason, further qualitative research to capture students' personal perspectives and experiences was needed. In order to fill this gap, a plenary discussion and brainstorming session was conducted at the end of the intervention in 2019, providing an approach to the participants' perceptions and experiences. The session had four main topics: (a) take-home ideas, (b) values learnt, (c) previous HIS experience, and (d) initiatives that students would implement to improve their local settings and also the recently visited ones during the Summer Program.

### *2.2. Setting and Procedures*

The educational intervention for healthcare future professionals and its evaluation was conducted in July 2019 at the University of Alicante. Since the ISTEW project ended, this course has been the only implementation initiative with regard to the specific educational modules created in the project. It consisted of a one-week 50-h program divided into theory and practice. Students had the chance to visit Spanish public and private hospitals as well as primary health care centers, observing, detecting, and discussing similarities and differences with regard to their healthcare contexts and contrasting such practical experience with the knowledge achieved in the theoretical sessions. The purpose of this intervention was to develop their theoretical and practical knowledge about HIS contents and values, promoting critical thinking, developing improvement-based thinking and behavior, creating awareness, and consequently generating a HIS culture. During the course, students created their own projects designing HIS interventions in practice by using scientific HIS evidence and sources (e.g., indicators, questionnaires, interviews etc.) and presented their ideas in a dynamic environment where all students could make their input and interact to one another. For our research purpose we conducted the discussion session in the main classroom used for the course at the University of Alicante on the last day once the program had been fully completed.

### *2.3. Participants*

Twenty-five nursing students from other European Higher Education Institutions such as the University of The West of Scotland in the United Kingdom, the Waterford Institute of Technology in Ireland, the Laurea University of Applied Sciences in Finland, and the University of Alicante itself participated. All of them agreed to be part of the plenary discussion and participated in the cyclic process of reflection–action–reflection based on PAR principles in which the researchers evaluated continuously each intervention, interacting constantly with them [16].

### *2.4. Data Collection and Analysis*

Students' experiences through the course were collected from the discussion session conducted. Notes were taken manually by one researcher. Another experienced researcher moderated the session in which students and educators participated, and the other researcher was the observer. The full transcribed notes are in the Supplementary Material (Document S1). The data content analysis was the method of analysis chosen and was carried out throughout a triangulation process in which three experienced qualitative researchers participated. Content analysis is a systematic analysis method that makes inferences in this case from the participants' experiences expressed in the open session and observed by the researchers. The results were classified firstly following the four main topics that guided the discussion: Take-home ideas, values learnt, previous HIS experience, and initiatives that students would implement to improve their local settings and also the recently visited ones during the Summer Program. The three researchers participating in the analysis decided to classify the answers to the first three topics into categories according to the number of times repeated, while the results of the fourth topic were gathered by

country, since the analysis of the data showed that the content of the students' answers was associated with their place of origin.

### **3. Results**

About the first topic, researchers explored the main idea that students referred as having learnt. After the analysis of this topic, eight categories came up corresponding to the most-repeated ideas (Table 1).


**Table 1.** Categories regarding the students' main ideas on healthcare improvement science (HIS).

Continuing with topic 1, in Table 2 eight categories have been gathered according to whom is responsible for them: "Internal" indicates that it is the student/future professional who is responsible for the action and "External" refers to when the responsibility lies with another person/organization.

**Table 2.** Category classification per responsible agent.


In the second topic the most significant value learnt for each student was highlighted. After the analysis of the answers, classification was performed with regard to the five most repeated values for the students. In order, the most repeated value was Teamwork, followed by Respect, Passion, and Humanization of Care/Compassion, with Communication being the least repeated.

Thirdly, the question "What would you improve in this context and in your context" was asked. This section is about the exchange of improvement, which reflects the different improvements and/or changes that students think can be made both in Spain, where the course took place, and in their country of origin. For the response analysis, the thematic units extracted from the first question have been reused, defined, and finally a selection of the most repeated answers has been presented in Table 3.


**Table 3.** Topics and student quotations.


Finally, in the Table 4 the fourth topic discussed, "Have you ever had any type of improvement science subject or previous experience?", was analyzed per country and the responses were grouped after reaching a consensus among the participants themselves.

**Table 4.** HIS experiences per country.


### **4. Discussion**

This study aimed to evaluate the European nursing students' experiences and perceptions after an educational intervention on healthcare improvement science (HIS). This qualitative study and others have demonstrated how relevant healthcare improvement science is at all professional and educational stages for the nursing profession [1,8,10]. Developing and evaluating this educational intervention from the perspective of the ISTEW project modules will contribute to the ISTEW project main aim by taking a step towards standardizing HIS culture across Europe [7,9]. During the implementation of the modules, the researchers' team agreed to evaluate the intervention every year and integrate students' feedback and needs through participatory action research methodology according to the experience presented in this manuscript. The inclusion of the open session discussion in 2019 permitted a deeper exploration of students' feedback. The study team understood how important it is to have a full understanding of the student's perspective to build bridges between theory and practice, enabling them to succeed in this transition process.

This research contributes to an understanding of how healthcare improvement science education provides nursing students with the confidence to make changes in their future work settings, delivering safe, effective, person-centered, efficient, equitable, and timely care [9]. To assure and follow up on the lessons learned as well as implementation in the work settings by students, further prospective research is needed [17,18]. Future courses with the new HISEF version combined with qualitative PAR are being planned with a virtual format due to the SARS-CoV-2 (COVID-19) pandemic.

The methodology used is effective in capturing student transformation, experiences, and perceptions during the course. The new section during the 2019 course and presented in the tables was perfectly combined with the HISEF to deeply understand students' perspectives and experiences. In relation to the main categories and topics identified, a tendency can be observed. In accordance with the results obtained in the literature reviewed conducted by Lillo et al. [8], keywords like "nursing empowerment", "nursing research" or "healthcare systems" are important with regard to student involvement with HIS education. However, as also mentioned in the previous study, the disparities among European countries create difficulties in healthcare improvement science standardization. This context can be seen as a weakness, but the authors used it as a strength to increase knowledge exchange among students due interactions during the course. From this research and previous publications on the field a conclusion can be made: Due to HIS disparities, educational interventions should include an international perspective. It has been observed that in Europe, HIS is understood and practiced in different ways according to the country. If a more comprehensive and broader perspective on HIS is to be achieved, educators, students, and finally healthcare systems should benefit from international educational exchanges and networking [8]. There is evidence suggesting that supporting staff at the early stages is the key step to driving systems into sustainable changes to promote patient-centeredness [19]. On this basis, the improvement of science education early in nursing careers relies on a common understanding of best practices and improvement methods that have the potential to redirect healthcare settings towards values such as safety or compassion, with a natural impact on patients' quality of care [20]. Improvement science has the potential to develop, but all related interventions must be evaluated [21]. HIS benefits need to be evidenced and all efforts in its development will be crucial for the future of healthcare systems [22,23].

### *Limitations*

The content analysis method selected had a potential risk regarding the researchers' implication when analyzing the data and drawing conclusions. To prevent this, three researchers participated in the analysis process through an analysis triangulation. On the other hand, quantitative data obtained from the HISEF should be prospectively compared with the qualitative information collected, improving both evaluation methods in order to capture the students' experiences as accurately as possible. Moreover, students from other countries and from other health professions should be included towards to provide more evidence. However, despite the limitations, this paper is a starting point that provides useful information about nursing students' interactions within a global HIS perspective.

In relation to the qualitative technique used, the type of open discussion group ran the risk of leaving out feedback from those participants who were less self-confident in expressing their opinion in public. In order to avoid this, all participants were asked one-by-one in a safe and open atmosphere, encouraging them to express their opinions and facilitating the discussion among all members. Finally, further evaluation rounds would be needed in future educational interventions in order to see if the last HISEF version after the 2019 course better captured quantitative data and whether the results were coherent with the qualitative data collected through the open discussions. Further course editions are planned as soon as face-to-face education and travel between countries without restrictions are possible.

### **5. Conclusions**

The new summer course evaluation process was conducted successfully, and the students' experiences and perceptions were well captured, as detailed previously. Students improved their critical thinking and knowledge in HIS and professional values and learned about the ways things are done in other cultural contexts. The educators also had the chance to improve the didactics, contents, and organization of the course. The PAR method is useful for students to reflect about course contents and ideas for improvement. An increase in students' motivation, inspiration, and willingness for a transformation based on improvement emerged. Nevertheless, with the current available data long-term consequences in healthcare systems cannot be demonstrated at this early stage. A longer follow-up phase for students is needed.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/1660-460 1/18/3/1298/s1, Document S1: The full transcribed notes.

**Author Contributions:** Conceptualization, M.C.S.-D. and M.L.-C.; methodology, M.L.-C.; software, P.V.; validation, M.C.S.-D., M.L.-C., P.V., and A.K.; formal analysis, M.C.S.-D., M.L.-C., P.V., and A.K.; investigation, M.C.S.-D., M.L.-C., P.V., and A.K.; resources, M.L.-C.; data curation, M.C.S.-D., M.L.-C., P.V., and A.K.; writing—original draft preparation, A.K.; writing—review and editing, M.C.S.-D., M.L.-C., P.V., and A.K.; visualization, M.C.S.-D.; supervision, M.L.-C.; project administration, M.C.S.-D.; funding acquisition, M.L.-C. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research was financed by the European Union-funded ERASMUS Lifelong Learning Project, ISTEW: Improvement Science Training for European Healthcare Workers (Project No. 539194- LLP-1-2013-1-UK-ERASMUS-EQR).

**Institutional Review Board Statement:** This study was approved by the IRB/IACUC of the University of Jesenice as project partner (10/01/002/2014-SHT). All participants received information about the research, its purpose, and characteristics (verbal, online, and in paper format). Participants´ rights were explained. Their participation was voluntary and they could stop at any time they wished. It was clarified that the collected notes were anonymous and for private use in this study, and were only to be used for research purposes.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data is presented in the paper.

**Acknowledgments:** The authors would like to acknowledge the contributions of all other ISTEW project colleagues for the development of the modules used in the educational intervention evaluated and all professors from the universities participating in this course.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Article* **Job Attractiveness and Job Satisfaction of Dental Hygienists: From Japanese Dental Hygienists' Survey 2019**

**Yuki Ohara 1,2,\*, Yoshiaki Nomura 3, Yuko Yamamoto 4, Ayako Okada 5, Noriyasu Hosoya 4, Nobuhiro Hanada 3, Hirohiko Hirano 2,6 and Noriko Takei <sup>1</sup>**


**Abstract:** Job attractiveness and job satisfaction are important factors in the continuity of employment among healthcare professionals. The aim of this study was to assess job satisfaction and job attractiveness among dental hygienists in Japan. The Japan Dental Hygienists Association conducted a survey of the employment status of Japanese dental hygienists in 2019. Questionnaires were distributed to all 16,722 members, and 8932 were returned (Collection rate: 53.4%). Data from 7869 currently working dental hygienists were analysed in this study. We analysed seven items of job attractiveness, 14 items of job satisfaction, and 13 items of request for improving the working environment. Item response theory and structural equation modelling (SEM) were utilized for the analysis. For attractiveness of dental hygienists' work, respondents placed greater emphasis on the fact that dental hygienists needed national qualifications rather than on income stability. SEM showed that job satisfaction consisted of two factors, 'Value for work' and 'Working environment', as did job attractiveness, with 'Contribution' and 'Assured income'. Value for work affects the contribution to people, and, employment environment affects assured income. Improving job satisfaction and work environments could help to improve the employment rate of dental hygienists, which could positively influence patient care.

**Keywords:** dental hygienist; job attractiveness; job satisfaction; work environment

### **1. Introduction**

The Japanese Dental Hygienists Law states that the mission of dental hygienists is the prevention of oral disease under the instruction of dentists by following treatments, including the mechanical removal of deposits found on the healthy root surface and under healthy free gingiva, drug application on the tooth and oral cavity, assisting in dental treatment, and oral health instructions [1]. Dental hygienists in Japan play an important role as healthcare professionals and have been asked to perform a wide variety of clinical practice skills in the Japanese super-aging society. There is a demand for visiting home dental care, oral care for hospitalised patients, and oral health management for older people requiring long-term care. Previous studies have revealed that oral health management of dental hygienists for older adults or hospitalised patients is effective in improving not only oral health, but also general health conditions [2–4], thus, emphasizing the social role of dental hygienists as professionals in oral health management. However, the employment rate of dental hygienists in Japan is very low compared to other countries [5,6]. According to a national survey from 2014, the number of registered dental hygienists in Japan was

**Citation:** Ohara, Y.; Nomura, Y.; Yamamoto, Y.; Okada, A.; Hosoya, N.; Hanada, N.; Hirano, H.; Takei, N. Job Attractiveness and Job Satisfaction of Dental Hygienists: From Japanese Dental Hygienists' Survey 2019. *IJERPH* **2021**, *18*, 755. https://doi.org/10.3390/ ijerph18020755

Received: 26 December 2020 Accepted: 15 January 2021 Published: 17 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

approximately 250,000, but the number of employed dental hygienists was 116,299 [7]. An insufficient number of dental hygienists may lead to serious problems that affect the supply of dental health care services. Prevention of leaving jobs and support for re-employment are important in maintaining a stable employment rate. Therefore, the development of a positive working environment for dental hygienists is important [7].

Job satisfaction is an important prerequisite for a good work environment [8]. Previous studies have reported that job satisfaction is a key factor in continuing employment, especially for healthcare professionals, including dental professionals [9–11]. Johns et al. reported that perceived job boredom and lack of benefits helped determine whether a dental hygienist would leave clinical practice. However, salary was implicated as a reason for continuing work [12]. Given these findings, perceived job attractiveness and satisfaction, including employment stability and specialty as a dental hygienist, may lead to motivation and positive attitudes toward work, which in turn may promote individual career formation.

To ensure stability of the dental hygienist workforce, it is necessary to determine which issues affect dental hygienists and analyse their effects on job attractiveness and satisfaction. However, little is known about how dental hygienists perceive their job attractiveness and satisfaction in Japan. The aim of this study was to clarify the issues of Japanese dental hygienists regarding their job satisfaction, job demands, and work environment.

### **2. Materials and Methods**

### *2.1. Study Design and Participants*

The Japan Dental Hygienists Association has been conducting surveys on the employment status of dental hygienists in Japan every five years since 1981 [5]. Anonymous questionnaires were distributed to all members of the Japan Dental Hygienists Association on 16 October 2019 by post, and the questionnaires returned by 30 November 2019 were used for the analysis. A total of 16,722 questionnaires were distributed by post, and 8932 were returned (collection rate was 53.4%). Among them, 1063 were from dental hygienists leaving their jobs, which were removed from the analysis, since the data whose did not worked as dental hygienists at the time of the survey, might not reflect the actual situation. This study was approved by the Ethics Committees of the Tsurumi University School of Dental Medicine (approval No. 1837), which was conducted in accordance with the Declaration of Helsinki. Informed written consent was obtained from all participants.

### *2.2. Questionnaire*

The questionnaire used in this study consisted of 101 items related to demographic factors, employment status, work content, value of work, etc. We analysed 34 items regarding job attractiveness and satisfaction in addition to the factors dental hygienists feel would improve the work environment. The questionnaires originally created by authors. Job attractiveness was evaluated by seven dichotomous questions about, for example, being a professional, national qualification, and income stability. The questionnaire regarding job satisfaction consisted of 14 items rated on a five-point ordinal scale. The questionnaire regarding the factors dental hygienists feel would improve the work environment consisted of 13 dichotomous questions.

### *2.3. Statistical Analysis*

Cross-tabulation was performed on age group and the items of job attractiveness, and the factors dental hygienists feel would improve the work environment. Correspondence analysis was performed with this cross-tabulation. To visualize the relationships, the results were illustrated graphically as biplots [13]. A three-parameter logistic model with item response theory (IRT) analysis was applied to calculate item discrimination, item difficulties, and item guesses for job attractiveness and satisfaction [1,13,14]. Item response and information curves are graphically illustrated. The analyses were carried out using R software version 3.50 (Institute for Statistics and Mathematics, Wien, Australia) with the LTR and irtoys packages using the following formula:

$$P\_i(\theta) = \frac{(1 - c\_i)}{1 + e^{-Da\_i(\theta - b\_i)}} \tag{1}$$

where *ai*: discrimination, b*i*: difficulty and *ci*: guessing.

Factor analysis with varimax rotation was performed to determine the latent variables for structural equation modelling (SEM). The structural relationship between job attractiveness and job satisfaction was calculated using AMOS software (24.0, IBM, Tokyo, Japan).

### **3. Results**

### *3.1. Participant Characteristics*

The age of the participants was 46.4 ± 11.9 years (median: 48 years, range: 20–81 years). Thirty-five participants (0.4%) were men. The year of experience as a licensed dental hygienist was 20.2 ± 11.4 years (median: 20, range: 0–61). Figure 1 shows the results of descriptive statistics for the items of job attractiveness (A), job satisfaction (B), and the factors dental hygienists feel would improve the work environment (C). In relation to the reason dental hygienist work was attractive, the highest percentage cited 'National license' (95.8%), followed by 'Highly specialised work' (93.2%) and 'Contributions to people and society' (91.3%). For job satisfaction, the highest proportion cited 'Worthwhile job' (84.3%), followed by 'Liking dental hygienists' work' (83.2%), and 'Feeling the value of hygienist's license' (79.0%). Regarding the factors dental hygienists feel would improve the work environment, the most frequently responses were 'Improved salary' (72.5%), followed by 'Enhanced evaluation of specialisation and qualification' (61.3%). Biplots of age group for each question are presented in Figure S1.

### *3.2. IRT Analysis for Job Attractiveness and the Factors Dental Hygienists Feel Would Improve the Work Environment*

Using factor analysis we categorised the 14 items regarding job satisfaction into two factors: 'Value for work' and 'Working environment'. Similarly, the seven items of job attractiveness were categorised into two factors, 'Contribution' and 'Assured income'. The 13 items regarding the factors dental hygienists feel would improve the work environment were categorised into three factors (Table S1). The attractiveness of dental hygienists' work and the aforementioned factors were analysed using a 3 three-parameter logistic model based on IRT.

Figure 2 shows item response curves and item information curves for the attractiveness of dental hygienists' work (A) and the factors dental hygienists feel would improve the work environment (B). The constructed models are shown in Table S2. For attractiveness of dental hygienists' work, item response curves shifted backward. The steepness of the curve at its inflexion point provides a measure of the discriminatory power of the item. Discrimination refers to how well an item can distinguish between respondents with low ability levels and those with high ability levels. In this case, respondents with high ability indicates responded 'Yes' often for the items, whereas respondents with low ability levels a low are relatively flat have low discrimination.

**Figure 1.** Simple tabulation of participants' response to each questionnaire. Bar graphs shows the participant's response to each questionnaire regarding job attractiveness (**A**), job satisfaction(**B**), and the factors dental hygienists feel would improve the work environment (**C**).

**Figure 2.** Item response curve and item information curve for the items regarding job attractiveness of dental hygienists' work and the factors dental hygienists feel would improve the work environment. (**A**) Job attractiveness (**B**) The factors dental hygienists feel would improve the work environment.

The horizontal axis shows the participant's ability and the item response curve axis shows the positive response to each item. Ability, shown on the horizontal axis, indicates the standardized weighted sum of the positive response of the items. That is, the closer the forward area, the more negative the question, and the closer the backward area, the more likely the answer is positive. The item response curve shows how precisely each item measures latent traits at various levels. A greater area under this curve indicates that 'Yes' was answered for all items at a higher rate, and these items may shape attractiveness to work for dental hygienists. Among them, items of 'National qualification' and 'Easy to change work place and gain employment' had a probability of higher than 0.5 at the origin point, which indicates that more than half of dental hygienists answered 'Yes' for these

items. The item response curves for 'Stable income' and 'Easy to change work place and gain employment' were steep, which indicates that the responses to these items have a clear cut-off point. The three items of 'Protects people and their health', 'Direct interaction and assistance for people', and 'Contribution to people and society' were located backward direction, which indicates that most of the dental hygienists answered 'Yes 'for these items. Regarding the item information curve of job attractiveness, the item 'National qualification' had little information and was in a backward direction, which indicates that most dental hygienists responded 'Yes' to these items. 'Stable income' had the highest item information, followed by 'Easy to change work place and gain employment'. These curves are located near the Y axis, which indicates that about half of the dental hygienists responded 'Yes' to these items. Where dental hygienists answered 'Yes' to these items, they responded 'Yes' to all other items.

Figure 2B shows the item response and information curves for the working environment. Many participants indicated a need to improve salary conditions, such as having regular pay raises and enhancing evaluation of specialisation and qualification. Neither IRT, nor item information curves regarding the working environment were as systematic as attractiveness to work, and there was no characteristic trend. Item response curves and item information curves for job attractiveness and demands for professional improvement analysed per factor extracted by factor analysis are shown in Figure S2. All items on the item response curve for job attractiveness were shifted in a backward direction; thus, many dental hygienists considered all items to be important, and in the item information curve, peaks for 'Easy to change work place and gain employment' indicated a tendency for dental hygienists to preferentially place importance relative to other items. In relation to the factors dental hygienists feel would improve the work environment, the tendency of the item response curve showed that the proportion of dental hygienists who answered 'Improved salary conditions' was relatively high, and items such as 'Better long-term care support' were less emphasised. Information from item information curves indicated that the items of guaranteed employment stability had greater information.

### *3.3. SEM for Job Satisfaction and Attractiveness*

SEM was conducted to visualise the influence of job satisfaction on the attractiveness of dental hygienists' jobs (Figure 3). All paths were statistically significant. 'Value for work' significantly affected 'Contribution', and 'Working environment' affected 'Assured income' to some extent.

**Figure 3.** Path diagram of job satisfaction and attractiveness of dental hygienists' work; RMSEA: Root Mean Square Error of Approximation.

### **4. Discussion**

In this large-scale study of dental hygienists in Japan, we investigated the association between job attractiveness and satisfaction, and the current status of the factors dental hygienists feel would improve the work environment. To the best of our knowledge, this is the first report describing the detailed characteristics of occupational awareness among Japanese dental hygienists, which cannot be clarified by the results of simple descriptive statistics. This study has been conducted by the Japan Dental Hygienists Association every five years. Many of the items were dichotomous responses. This survey confirmed the results of the previous survey. Dichotomous responses lack depth of information compared to those rated on a Likert-type scale. However, when applied to item response analysis, results obtained using dichotomous variable are easy to interpret [15]; this study utilized the merits of such variables. Item response theory analysis is a powerful analytical method, especially for dichotomous variables. It is widely used in educational research and tests, such as the widely-known TOEFL. It is also applicable in medical research. Valuable information, rather than a simple descriptive analysis of frequency, can be presented using IRT. The slope and location of item information curve can provide valuable information on the response pattern in a questionnaire. We have been frequently applying IRT for in our research studies. When interpreting the descriptive analysis of job attractiveness, more than 90% of dental hygienists gave positive responses for all items except 'Stable income' and 'Easy to change work place and gain employment'. Moreover, the item response curve revealed that the curves of all items were shifted backward, that is, many respondents responded that the work of dental hygienists was attractive. This result suggests that most dental hygienists find value in their jobs. Therefore, the strength that the dental hygienists perceive attractiveness of these tasks is an important factor for their work continuity. Most dental hygienists recognised attractiveness in the stability of their status as a worker, that is, having a national qualification made it easy to change where they work.

Notably, direct involvement with people and contributions to life and society tended to be perceived as attractive only if other factors were met. According to the item information curve, income stability and easy to change work place and gain employment had high item information. These two items were more attractive than the other items. The results of IRT and factor analysis indicated that many dental hygienists considered that easy to change work place and stable income were more important than national qualifications (Table S1[A]). In contrast, for 'Contribution' factors, all items were presented as sigmoidal curves. This suggests that dental hygienists find more value regarding aspects related to the contribution of their work as job attractiveness increases (Table S1B).

Factors directly linked to daily life, such as employment status and income stability, may be prerequisites for the attractiveness of work as a dental hygienist. With respect to t the factors dental hygienists feel would improve the work environment, item information curves of salary and appraisal of specialty and license were backwards. This indicates that many dental hygienists requested these two items rather than other working conditions. Conversely, item information curves for childcare support and shortened working hours were forward-facing, indicating that a limited number of dental hygienists requested for the improvement of these two conditions. When comparing the item response curve and item information curves of the factors dental hygienists feel would improve the work environment with attractiveness, curves were gentle sloped sigmoid curves and were in a limited area. This indicates that even though the salary and appraisal of specialty were common requests, the need to improve other conditions depended about each dental hygienist. In other words, the perception of the working environment may be influenced by the circumstances and view of each dental hygienist; thus, a subdivided validation of each of these factors is necessary.

A previous study reported that reducing the workload, enhancing welfare, and career developments were associated with job satisfaction among healthcare staff in China [16]. However, the results of this study showed that the demands about salary and employment

stability were more pronounced than the workload. This trend of salary emphasis was like findings from previous studies about dental care providers [17–19].

The results from the SEM showed that factors related to the working environment significantly influenced factors of assured income regarding job attractiveness. Previous studies have also reported that turnover of healthcare professionals is caused by dissatisfaction with their work, but it is inferred that the factors causing dissatisfaction may differ depending on job content and educational background [16,20]. In particular, improvement in salary may improve the job satisfaction of dental hygienists in Japan. Detailed verification is necessary for the improvement of working conditions of dental hygienists for the planning of specific measures to prevent turnover. Therefore, further study is necessary to investigate the association between leaving jobs and job satisfaction. The results of the SEM showed that the job satisfaction of dental hygienists presented their characteristics as professionals. Supporting people's health, such as contributions to people and society, had high loadings. Ayers et al. reported that one of the independent factors associated with career satisfaction among New Zealand dental therapists was whether they felt that they were a valued part of the dental community [19], so increasing the value of work may increase job satisfaction. The improvement of both the contribution to people and society and assured salary may be issues for ensuring dental hygienists' satisfaction and improving the quality of dental services in Japan.

There are some limitations to the present study. First, the participants may have a variety of backgrounds. For example, years of education before obtaining a dental hygienist's license, years of clinical experience, and place of employment may have led to differences in job attractiveness, satisfaction, and the factors dental hygienists feel would improve the work environment. Correspondence analysis also revealed the characteristics of the participants according to their generations, which warrants the need for in-depth examination in the future [21,22]. Second, the duties of dental hygienists are stipulated by the legislation and regulations of each country, and the specific content varies widely, so the results have limited generalizability outside of Japan. Job satisfaction is a key factor in the stable career formation of healthcare providers; therefore, studies comparing and examining differences on a global scale are desirable in the future.

### **5. Conclusions**

In conclusion, the results indicated that Japanese dental hygienists find that the stability of their occupation and employment is equally important to their contribution to people and society, and that these factors are highly relevant to job satisfaction. Improving job satisfaction and work environments could help prevent high turnover among dental hygienists. In particular, it is important to improve their working environment, so that it leads to improved salary conditions, and enhanced assessment of professionalism and qualifications.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/1660-4 601/18/2/755/s1, Table S1: Results of factor analysis of job satisfaction, job attractiveness, and the factors dental hygienists feel would improve the work environment, Table S2: Three parameter logistic model based on item response theory, Figure S1: Biplots of age group and job attractiveness(A), the factors dental hygienists feel would improve the work environment (B). Navy plots correspond to age group of the participants. Closely located plots are meaning highly coincident, Figure S2: Item response curve and item information curve of for the items regarding job attractiveness of dental hygienists' work and the factors dental hygienists feel would improve the work environment by each factor.

**Author Contributions:** Conceptualisation and data curation, Y.O., Y.N., and N.T.; Formal analysis, Y.O. and Y.N.; Validation, Y.O., Y.N., Y.Y., A.O., N.H. (Noriyasu Hosoya), N.H. (Nobuhiro Hanada), H.H., and N.T.; Visualisation, Y.O., and Y.N.; Writing-original draft, Y.O. and Y.N.; Writing-review & editing, Y.Y., A.O., N.H. (Noriyasu Hosoya), N.H. (Nobuhiro Hanada), and N.T. All authors have read and agreed to the published version of the manuscript.

**Funding:** This survey was carried out by the annual found of the Japan Dental Hygienists' Association.

**Institutional Review Board Statement:** This study was approved by the Ethics Committees of the Tsurumi University School of Dental Medicine (approval No. 1837).

**Informed Consent Statement:** This study was conducted in accordance with the Declaration of Helsinki. Informed written consent was obtained from all participants.

**Data Availability Statement:** The data of the present study were used under license for the current study and, therefore, are not publicly available.

**Acknowledgments:** The authors acknowledge Editage for English language editing.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Review* **Broken Promises to the People of Newark: A Historical Review of the Newark Uprising, the Newark Agreements, and Rutgers New Jersey Medical School's Commitments to Newark**

**Rosy C. Franklin 1, Ryan A. Behmer Hansen 1, Jean M. Pierce 2, Diomedes J. Tsitouras <sup>3</sup> and Catherine A. Mazzola 4,\***


**Citation:** Franklin, R.C.; Behmer Hansen, R.A.; Pierce, J.M.; Tsitouras, D.J.; Mazzola, C.A. Broken Promises to the People of Newark: A Historical Review of the Newark Uprising, the Newark Agreements, and Rutgers New Jersey Medical School's Commitments to Newark. *IJERPH* **2021**, *18*, 2117. https://doi.org/10.3390/ ijerph18042117

Academic Editor: Paul B. Tchounwou

Received: 20 December 2020 Accepted: 17 February 2021 Published: 22 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Abstract:** Many have referred to the coronavirus disease 2019 crisis and intertwined issues of structural racism as "twin pandemics". As healthcare workers in Newark, New Jersey, a city heavily affected by the twin pandemics, we recognize that health workforce changes must be grounded in our community's recent history. The objective of this essay is to briefly describe the relationship between organized medicine, state and local leaders, and the people of Newark. We begin with a discussion of Newark in the 1950s and 1960s: its people experienced poor socioeconomic conditions, terrible medical care, and the many sequelae of abhorrent racism. Plans to establish a New Jersey Medical School in Newark's Central Ward also threatened to displace many residents from their homes. We then describe the Newark Agreements of 1968, which formalized a social contract between the state, business leaders, and people of Newark. In part, the Medical School committed to indefinitely promoting public health in Newark. We share progress towards this goal. Finally, we document key healthcare administrative decisions facing our community today. Stakeholder opinions are shared. We conclude that the Newark Agreements set an important standard for communities across the country. Creative solutions to healthcare policy may be realized through extensive community collaboration.

**Keywords:** health workforce; workforce policy; health equity; racism; history; medicine; medical education

### **1. Introduction**

Many have referred to the coronavirus disease 2019 (COVID-19) crisis and intertwined issues of global structural racism as "twin pandemics" [1–3]. Structural racism is defined as overarching systems, large-scale social forces, ideologies, organizations, and processes which interact to contribute to racial injustices and reinforce disparities [4]. The contemporary COVID-19 pandemic has illustrated the convergence of structural racism and health [5]. The United States medical community has renewed its interest in investigating the many effects of broader socioeconomic conditions on public health [6–8]. As medical students and physicians in Newark, NJ, we know that public health interventions are unlikely to prove effective if not informed by the past [9,10]. For Newark, we believe that decisions regarding healthcare administration and education must be grounded in our community's recent history. In our recent history, a premier healthcare organization and training facility were established under conditions agreed upon with community members. Since their establishment, the school and hospital have remained integral pieces of the community and have taken significant steps to improve Newark's public health. The "twin pandemics" are pressing issues which align to exacerbate structural inequalities. In response, changes

to the healthcare system may be necessary, but the sociomedical history of Newark remains significant and informative, with important implications for healthcare decisions today.

During the 1950s and 1960s, the people of Newark experienced both poor socioeconomic conditions and the effects of abhorrent racism. Concurrently, plans to establish a New Jersey Medical School in Newark's Central Ward threatened to displace many Newark residents from their homes. The historic Newark Agreements of 1968 (sometimes referred to as the Newark Accords), detail a compromise borne out of lengthy negotiations between the state, many business leaders, and Newark's own community leaders. The legitimacy of Rutgers New Jersey Medical School (NJMS) and its primary teaching hospital, University Hospital (UH), is grounded in said Agreements. Moreover, development of Rutgers NJMS and UH in Newark's Central ward established the two entities as critical cornerstones of healthcare provision and a place of work for some of Newark's most impoverished minority groups. Today, ongoing administrative discussions regarding merging these entities with a private health system [11–14] threaten to erode decades of trust built between Newark's people and its medical school. Some fear that valuable resources, indispensable personnel, and employment opportunities may be funneled from a still-ailing Newark community to New Jersey's more affluent suburban communities [12–15]. The diversion of resources from programs meant to benefit Newark's minority populations (including, but not limited to, communities of color and Hispanic communities) is not without precedent: in the 1960s, Mayor Hugh Addonizio diverted funding from the community-led United Community Corporation (UCC) [16]; and in 2018, Rutgers diverted inpatient pediatrics specialists and resources from UH [17].

Therefore, the objective of this essay is to briefly describe the relationship between organized medicine, state and local leaders, and the people of Newark, NJ. We begin with a description of how socioeconomic factors, combined with civic unrest, culminated in the Newark Uprisings of 1967. Discussions to establish a New Jersey Medical School, and the reasons for its location, are shared. The subsequent Newark Agreements and outcomes of said Agreements to present day are discussed. Finally, we consider today's discourse about the future of Rutgers New Jersey Medical School and Newark's UH.

This interdisciplinary work aims to emphasize that local health policy decisions are, and have always been, complex. By specifically describing the relationship between Newark's medical school and its community, we hope that readers of all backgrounds will better appreciate the role that factors such as race, power, and politics play in determining the ultimate health status of a community. The presence and investment in medical training facilities in Newark have helped to curb sociomedical inequities. University Hospital is an essential safety net hospital in Newark. Readers may learn from Newark's local history and apply relevant lessons to improve public health in their own communities. Accordingly, this manuscript is broken down into thematic sections. Readers may therefore gain a specific appreciation for each individual topic while simultaneously appreciating the intersecting themes present in each section.

### **2. Newark 1950–1967**

To appreciate the magnitude of the Newark Agreements, it is imperative to first review the socioeconomic conditions in Newark, New Jersey in the 1950s and 1960s. In 1950, Newark's population was 439,000 [18], where the Black community comprised 17% of that population [19]. Over the next two decades, social segregation was prevalent. Due to population growth as well as concurrent "white flight", in which many white New Jersey residents left urban areas for suburban areas [20], the Black population in Newark reached 63% of the total by 1968 [19]. Unfortunately, the white flight of the 1950s and 1960s left Newark devoid of many of its manufacturing industries [19,20]; rates of unemployment were over 15% in the Black community [16]. Newark had high crime rates [16], and with one-third of the houses "substandard", Newark had "the highest percentage of substandard housing for any city of comparable size" [19]. Forty-five percent of adults over the age of 25 had less than an eighth-grade education [19]. Unsurprisingly, these socioeconomic circumstances contributed to the community's poor public health metrics: in the late 1960s, Newark had a high burden of substance use [16], sexually transmitted infections [16], the highest incidence of new tuberculosis cases in the country [19], as well as the highest maternal and infant mortality rates in the country [19]. While surrounding suburban towns had some of the best hospitals in the nation [19], Newark City Hospital was referred to as the "Slaughterhouse" or the "Butcherhouse" [19]. In 1969, Newark City Hospital had two nurses for every 39 patients; a rate dramatically lower than the reported state of New Jersey's requirement of 1 nurse per 6–8 patients [19].

In response, the President Johnson administration sponsored an antipoverty program called the Model Cities Program in 1966 [16,19]. Model Cities gave federal funding to local governments with objectives to revitalize urban centers [16]. Newark leaders recognized the applicability of this program to their city; Newark's 1967 Model Cities application stated that, "Today, poverty and the problems of racial transition are common to most older cities, especially in the Northeast. However, there are few cities anywhere in the nation where these and other problems extend so widely and cut so deeply as in Newark" [19]. Concurrently, sponsored by the federal Office of Economic Opportunity, Newark's UCC was developed to help Newark's poor. The UCC was a grassroots community action program aiming to "[enforce] housing codes and [train] minority citizens to qualify for high-paying union jobs in the construction industry" [16]. Given its unique element of community leadership, the UCC was a focal point for emerging Black power [16]. Election laws which prohibited individuals from voting unless they resided in Newark for at least six months, for example, had hindered many poor Black citizens from political engagement [19]. Funding from UCC helped Newark to sponsor a young Black man, Kenneth Gibson, to challenge the white Democratic Mayor Hugh Addonizio [16]. Gibson's initial mayoral bid was unsuccessful [16]. Addonizio lobbied the federal government to divert funding from the UCC (for which funding stagnated by 1967 [16]) towards those programs in which he had more governmental control, such as the Model Cities program [16,21]. Unfortunately, the Model Cities program was poorly coordinated and has been cited to have contributed to Newark's further socioeconomic breakdown [16]. Despite efforts from the federal government and local organizations to improve urban conditions, inequities in Newark persisted.

The social, economic, and medical injustices of the 1950s and 1960s culminated in members of Newark's African American community feeling "despair, rage, impotence, racial pride, and the sense that police had a double standard ... that condoned brutality toward Black Americans" [16]. Indeed, the civil rights movement of the 1960s was characterized by broader national unrest. Police violence sparked multiple rebellions across the country in the mid-1960s, including in: Watts, California; Detroit, Michigan; Tampa, Florida; Cincinnati, Ohio; Cleveland, Ohio; Chicago, Illinois; and Atlanta, Georgia [16,22]. As early as 1957, multiple African American newspapers and the Mayor's Commission on Intergroup Relations in Newark documented "widespread mistreatment of Black Newarkers by police" [22]. Newark citizens, led by activist Amiri Baraka and the Congress of Racial Equity, joined their counterparts in Philadelphia and New York City in demanding for a civilian review of policing [22]. These activists led public demonstrations demanding a civilian review board for the majority-white Newark police force [22]. Mayor Addonizio sided with the Police Director and Newark Police Benevolent Association in obstructing these reviews [22].

### **3. Newark Uprising**

On July 12, 1967, a Black Newark taxicab driver named John Smith was arrested for a traffic violation and was beaten by police [16,22,23]. The UCC and many community members rallied in support of John Smith, while the police arrived dressed in riot gear [16]. Violence broke out [16,23]. The National Guard was mobilized [16]. White-owned stores were looted [16,23]. From July 12–17, similar community uprisings occurred in Elizabeth, Englewood, Plainfield, and New Brunswick, New Jersey [16]. Twenty-four of the 26 total

deaths were of Black civilians, most of whom were killed by police firearms [22]. The "typical" rioter was found to be "an individual who had resided in Newark for greater than ten years" [19]. As one individual stated, "They had taken all the land over on 12th Ave. by 'eminent domain'. They said they were going to build townhouses but never did. We had no place to move, jobs, housing or schools" [16]. Even when the New Jersey state's Select Commission on Civil Disorders recommended a Newark review board in the aftermath, Mayor Addonizio objected to its formation [22]. Notably, the President Johnson National Advisory Commission on Civil Disorders report stated that "white racism is essentially responsible for the explosive mixture [resulting in widespread national upset] which has been accumulating in our cities since the end of World War II" [20].

In many historical sources, these events have been termed the Newark Riots [16,21,24,25]. A minority of sources instead describe these events as the Newark Uprising or Rebellion [22,26]. Given the historical context, we believe the terms Uprising or Rebellion are more appropriate descriptors of the events in Newark in 1967, in comparison to the more inflammatory term "Riot." Throughout the remainder of this manuscript, we hereby utilize the term "Uprising."

There are many factors that are believed to have contributed to the Newark Uprising. These include racism, widespread political disenfranchisement and voter suppression, poor housing and landlords, unemployment and job discrimination, poor health conditions, poor schooling, the conflicting goals of Newark's multiple anti-poverty programs and the ensuing funding cuts to the UCC, police brutality, and the passing over of selecting a Black community member for a Newark Board of Education position [16,19,20,23,26]. One final factor noted by Marin to contribute to the Newark Uprising was "the New Jersey state medical school's move to Newark's Central Ward" [16]. As Duhl and Steetle stated, the medical school issue "helped create the atmosphere in which only a spark was needed to kindle the riot fire" [19].

### **4. Medical School Plans**

Concurrent with the social segregation, racism, widespread social unrest, poverty, and worsening public health status were developments to organize medicine in northern New Jersey. Nationally, concerns about physician shortages dominated discussions around physician workforce policy during the 1950s, 1960s, and early 1970s [27]. Federal initiatives during this time included construction grants to medical schools to bolster production of new physicians [27]. Leaders in North New Jersey, likewise, recognized the need to increase healthcare workforce training locally. Several developments in local organized medicine were implemented over the next few decades. Reasons for the developments included the provision of ideal workforce training, the provision of care for the poor, improvements to the economy, and (as some residents felt) "community control".

In 1949, Jersey City Medical Center applied for a National Institute of Health (NIH) research training grant and, in response, the National Heart Advisory Council suggested a New Jersey medical school be established [21]. Medical schools formed after World War II were typically not built in impoverished urban areas [21]. The proposal to form a New Jersey medical school was supported by the state of New Jersey, the city of Jersey City, and the American Medical Association (AMA) [21], but it was challenged by Seton Hall College and the local Catholic Church, who proposed that a "Seton Hall Medical School" be erected in Jersey City instead [21]. Seton Hall Medical School was ultimately established in Jersey City in 1956, but it quickly developed financial difficulties, resulting in the state reclaiming its control in 1965 and renaming it "New Jersey College of Medicine and Dentistry (NJCMD)" [11,24]. Due to the Jersey City Mayor's frequent interference with the medical school's affairs, NJCMD planned to move from the city by the mid-1960s [21].

Discussions quickly began regarding NJCMD moving to either Madison, an affluent suburban town in northern New Jersey, or to Newark [19]. Although faculty favored relocation to Madison, there were conflicts with their local community hospital [16]. In comparison, as of 1962, Newark Mayor Addonizio had offered Newark City Hospital "to

any medical school interested in taking it over" [19]. The Newark site was the preferred destination by Essex County's 13-member delegation, as well as reportedly by many "medical, civic, educational, religious, business organizations, and municipalities" [19]. Addonizio saw the medical school as an opportunity to revitalize Newark [16], increase employment within the city [19], gain increased funding through the Model Cities Program [16], and create "one of the finest medical facilities in the country" with research laboratories and a new "University Hospital" [16]. He believed the patients in Newark would be ideal for medical students in training [16]. In turn, Addonizio promised NJCMD a total of 167 acres in Newark's Central Ward, including those occupied by Newark City Hospital [21]. The area, which he declared "blighted", would be claimed by eminent domain [16].

In response to these housing threats, two community organizations formed: the Newark Area Planning Association (NAPA) led by Yale law student Junius Williams, and the Committee Against Negro and Puerto Rican Removal (The Committee) led by Newark public school teacher Harry Wheeler [19] and chairman Louise Epperson [16]. Harry Wheeler stated, "the real reason for courting the medical school was that Addonizio wanted to disperse the Negro's political power" [19]. Both NAPA and The Committee used legal and administrative tactics to fight the medical school proposal [19]. They acknowledged that medical school discussions lacked a plan for the relocation of residents displaced by NJCMD's construction and did not include plans for the inclusion of city construction trade unions in building of the school [19]. In addition, there was a fear that "a new University Hospital . . . would be likely to exclude poor city residents" [21].

In June of 1967, immediately precipitating the Newark Uprising, NJCMD agreed to move to Newark [19]. Both the NAACP Legal Defense Fund and NAPA protested to the federal Department of Housing and Urban Development (HUD) that HUD's relocation procedures were violated by the medical school plan [16]. HUD Undersecretary Robert Wood and Health, Education, and Welfare Undersecretary Wilbur J. Cohen called for community-wide negotiations [16]. Many believed the medical school was "being used as a pawn" in a broader struggle for community control [19]. One individual stated, "no school in the history of medical education has been created under such circumstances" [16].

The ensuing medical school relocation discussions have prompted many poignant reflections:


### **5. Newark Agreements/Accords**

On 1 March 1968, after extensive local meetings and negotiations, the revolutionary Newark Agreements were signed [28], with amendments finalized on April 30 of the same year [16]. These Agreements are a historic social contract between the Newark community, the medical school, and governments at the local, state, and federal level [28]. Parties had agreed to commence construction of the academic medical center in the Central Ward of Newark [11,28]. The following points, as summarized in the excellent article by Marin, were included in the Agreements [16]:

	- a. "develop a comprehensive health plan for Newark's low-income community";
	- b. "develop a comprehensive community mental health plan for Newark's lowincome community";
	- c. "operate community health programs";
	- d. "formulate and coordinate training programs in health services and professions";
	- e. "assist the school in actively recruiting minority students, faculty, and professional staff";
	- f. "work with the school to develop 'career ladders' for non-professionals in the health field";
	- g. "periodically review the adequacy of community health services being provided by the school and make suggestions for change".

### **6. After the Agreements**

The Newark Agreements' principles are binding and, therefore, success in fulfilling the aforementioned commitments has varied [21]. Initially, there were many challenges. In the immediate aftermath of both the Uprising and the Agreements, vigilante groups of white individuals formed in Newark under the guise of protecting their "families and homes" [19]. Many more white and middle-class Black individuals moved away from Newark permanently [16]. In 1970, borne out of accusations of poor-quality medical care at Newark City Hospital and the poor treatment of employees, Newark community members protested the hospital [16]; this culminated in resignation of the NJMS president. Then, in the winter of 1971, the community attempted to block construction of a teaching hospital over concerns that it would function as a white referral hospital; ultimately, then-Mayor Kenneth Gibson helped in deciding to construct a single hospital (UH) to meet the needs of all community members [16]. Although perhaps most significantly, the Newark Community Health Council failed in its mission due to supposed in-fighting among members [16]. The Council was replaced in 1971 by the Board of Concerned Citizens, created and governed by the NJMS Board of Trustees [16]. Authors have noted that "the mission of developing a comprehensive health program for the community was substantially lost" and, instead, the new Board served only as an ambassador between school and community [16].

In 1970, Governor William Cahill enacted legislation merging NJCMD into a broader "College of Medicine and Dentistry of New Jersey" (CMDNJ), and the medical school adopted the title of "New Jersey Medical School" (NJMS), which it still holds to this day [29]. By May 10, 1976, the Newark campus was completed, including the medical school, dental school, Community Mental Health Center, and primary teaching hospital and level-one trauma center for the entire state of New Jersey [28]. A 1977 conference held to assess the school's progress in upholding the Newark Agreements found that: NJMS had invested substantially in Newark City Hospital [16]; the majority of the 2600 new jobs created were held by Newark community members [16]; citizen relocation agreements had been upheld [16]; and NJMS had begun to establish community health services, including a Family Health Center, preventative medicine and substance abuse programs, an ambulance service, and a CompreHealth health maintenance organization/ healthcare delivery system dedicated to Newark citizens [16]. In addition, NJMS had the largest enrollment of minority students of any medical school in the country, excluding two historically Black medical schools [16]. This fact remains true to date [11,13].

In 1981, the College of Medicine and Dentistry of New Jersey was reestablished as the "University of Medicine and Dentistry of New Jersey" (UMDNJ), of which NJMS was a part, making it "the largest freestanding public university of health sciences in the United States" [11,28]. Additionally, in the 1980s, the AIDS epidemic would take a significant toll on both Newark and its healthcare system [30]. By 1989, New Jersey ranked fourth in the nation in the number of reported AIDS cases [30]. It also had the highest percentage of women with HIV infection in the entire nation [30]. Newark was a major focus for infection. Today, New Jersey remains as "the epicenter of the HIV epidemic" [31]. Even early on, UMDNJ-NJMS physicians were at the forefront of the epidemic. Based on clinical experiences at UMDNJ-NJMS, a group led by Dr. James Oleske published a landmark study on immune deficiency syndrome in children, which for the first time drew attention to the fact that AIDS could affect children as well [32].

In 1994, the American Association of Medical Colleges (AAMC) awarded an Outstanding Community Service Award to the medical school at UMDNJ [16]. UH's mission as a valued community resource has been emphasized throughout its establishment, including in a 2010 New Jersey Higher Education Task Force report [11]. Per this report, UH and the medical school campus have played a fundamental role in Newark's community, economic, and cultural revitalization [11]. UH cares for the most uninsured patients in the state of New Jersey, and is the only public acute-care hospital in the state [14]. The report specifically notes that they believe continued additions to the Newark medical campus "[build] on and [enhance] the historic Newark Agreements" [11]. Several NJMS-sponsored community outreach organizations do important work in Newark, as briefly reviewed later in this article. NJMS is the oldest school of medicine in the state of New Jersey, and today it receives the most NIH funding for basic and clinical sciences out of all schools in the state [13].

However, rumblings of community discontent have persisted: a notable piece in late 1987 in the Newark Star-Ledger, penned by Joan Whitlow, reasserted that the school had engaged in discriminatory hiring practices, as well as failed to further increase minority student enrollment at the medical school [16]. In 2013, oversight of the medical school was restructured such that UH became an independent NJ state entity, while NJMS remained within Rutgers Biomedical and Health Sciences (RBHS) [33].

In the last few years, there have been several notable changes at UH. In early 2018, UH submitted a proposal to the New Jersey Department of Health (DOH) to reduce inpatient beds for children from twenty-three to four due to low patient volume [14,34]. Specialist physicians in pediatric trauma and resident pediatric physicians would be transferred to a different hospital, Newark Beth Israel [34]. Given that UH is the only Level 1 trauma center in New Jersey, many physicians and nurses stated that reducing pediatric care in this hospital would be detrimental for families in Newark [34]. In fact, some feared that the move would jeopardize UH's entire status as a Level 1 trauma center [14,34]. As Dr. James Oleske, professor of pediatrics, stated at the time, "this is a death blow for our medical school's commitment for that segment of the population ... To abandon the Central Ward and take pediatrics away from University Hospital is a terrible mistake" [34]. Dean of Rutgers New Jersey Medical School and Chair of University Hospital's Board of Trustees, Robert Johnson, stated that the hospital "wasn't built to have pediatrics in it" and that he has attempted to move pediatrics out of UH since the late 1990s [34]. As of July 2018, it was reported that UH withdrew this DOH proposal [35]. New Jersey Governor Philip Murphy ordered the DOH to appoint a monitor to review this situation with UH. The ensuing 2018 report stated that, "[UH] started to decrease its pediatric bed complement without the documented approval of the State ... in seeming contradiction to the Restructuring Act and the 1968 [Newark] Agreement" [17]. Indeed, this decision was made without even

alerting Newark's mayor [14]. To the best of our knowledge, the inpatient pediatrics unit at UH remains significantly downsized.

In late 2018, the UH CEO John N. Kastanis resigned amid both calls from the mayor and scrutiny from the DOH regarding an *Acinetobacter baumannii* bacterial outbreak at UH, ending a short tenure at a hospital already juggling significant changes [36,37]. The outbreak in question may have claimed the lives of three infants in the neonatal intensive care unit [36]. Just half a year later, in 2019, Rutgers University President Robert L. Barchi announced that he too would step down after the upcoming school year [38]. This resignation ended a seven-year tenure in which Rutgers both joined the Big Ten Conference and completed the largest higher education merger in American history [38].

### **7. Newark Today**

It is critical to acknowledge that economic stability, neighborhood and physical environment, education, food, community and social context, and healthcare system are all known social determinants of health [39]. In other words, the broader lives and lived experiences of those in Newark impacts the ultimate health status of those individuals. As of 2017, Newark has a population of 285,154 individuals [40], representing growth of 4.2% since 2000 [40]. Median age is approximately 34 years [39,40], and 51% are female [39]. The racial demographics have changed since the 1960s: today, approximately 89% of the population are racial/ ethnic minorities, with 48.6% Black and 35.6% Hispanic/ Latino [39]. In addition, 30.6% of the city's population is foreign-born [39], 54.2% speak English only, and 30.9% speak Spanish [39].

It is also critical to acknowledge that the modern socioeconomic state of Newark was largely determined decades ago; systems of structural racism have helped to translate poor conditions for minority groups during the 1950s and 1960s into poor conditions for minority groups now. The estimated poverty rate in Newark is generally 29% and, among children, increases to 39.5% [39]. Newark had an estimated median household income in 2017 of USD 35,167 (compared to the New Jersey average of USD 80,088) [40]. Black and Hispanic or Latino individuals are more likely than their white Newark counterparts to live in poverty [40]. Only 16% have a Bachelor's degree or higher (compared to 37.6% of New Jersey residents) [39]. Accordingly, the unemployment rate, as of March 2019, was 6.7% [39]. Nearly 28% of Newark households have utilized Supplemental Nutrition Assistance Program (SNAP) benefits in the past 12 months [39]. The most common industries in Newark as of 2017 are healthcare (11.2%), followed by construction (9.6%) and accommodation and food services (7.6%) [40]. Notably, 18.4% of women in Newark work in healthcare [40].

Although the demographics of Newark's population have shifted, persistent socioeconomic disadvantages continue to influence a host of poor public health outcomes. Today, an estimated 39.0% of Newark adults have hypertension (vs. 28.6% for NJ overall), 15.7% have diabetes (8.7% NJ), 8.4% have chronic obstructive pulmonary disease (5.1% NJ), 7.6% have coronary artery disease (5.8% NJ), 4.6% have had strokes (2.2% NJ), and 25.0% have lost all of their teeth (13.3% NJ) [39]. The percentage of Newark women giving birth in 2015 without any prenatal care was 3.1%, compared to 1.4% for New Jersey overall [39]. The overall infant mortality rate is 11.6% (vs. 4.4% for NJ overall), with a rate of 15.1% for Black infants (8.7% NJ) [39]. Although only about 3.1% of New Jersey's population resides in Newark, 17.4% of New Jersey's primary and secondary syphilis cases are in Newark [39]. These issues are partially because 28.9% of Newark individuals aged 18–64 lack health insurance (compared to 15.7% in New Jersey, overall) [39]. Although passage of the Patient Protection and Affordable Care Act in 2010 increased rates of covered individuals in Newark from 71.9% to 81.9% in 2015, there are still approximately 50,000 individuals lacking insurance [39].

It is critical to understand the concept of social determinants of health, particularly during the global COVID-19 pandemic. Authors have written extensively about the role of structural racism in exacerbating existing health disparities in the time of COVID-19 [5]. We wish to emphasize that the people of Newark have similarly been impacted [41,42]. Since the beginning of the pandemic, activists warned that people from Black and brown communities in New Jersey would disproportionately be burdened by COVID-related morbidity and mortality [43] and, unfortunately, they were correct [41,42], emphasized in the sobering figure created by the New Jersey Policy Perspective in October 2020 [42]. This new health disparity is a product of Newark having disproportionately fewer medical and economic resources than surrounding wealthy communities in Newark [44]. In October, USA Today did an entire expose on the role of structural racism and health disparities in ravaging Black and Brown communities in Essex County (Newark's county) during the COVID pandemic [45]. We encourage all readers to review this piece. It explains how during the first wave of the COVID pandemic, "Essex County was among the top 10 in the country for its death rate". In addition, Newark's Mayor Ras Baraka emphasized that many Newark citizens do not have the luxury to isolate in a basement or attic, apart from their family members, during quarantine [44]. As a result, many Newarkers have been contracting COVID within their own homes [44]. Even with the rollout of COVID vaccines, offered to various community members at Newark's own New Jersey Medical School campus to date, Mayor Baraka and others have expressed fear that the people of Newark may lack trust in the medical system to such an extent that they are unwilling to receive the vaccine [44].

Violence is also a known public health issue [18]. A 2014 Department of Justice investigation of the Newark Police Department found widespread discriminatory policing and excessive use of force [22]. In 2015, Newark Mayor Ras Baraka signed an executive order to establish the Civilian Complaint Review Board (CCRB) to evaluate the Newark Police Department [22]. Notably, civil rights activists in Newark have persistently demanded for a CCRB since the 1970s [22]; the issue was frequently revisited, politicized, and then opposed by key political leaders until 2015 [22]. Although the crime index in Newark has decreased from 2013 to 2018 [40], issues of police brutality still permeate civic life in Newark as well as in cities across America. Police brutality, which disproportionately affects people of color, may manifest as physical, sexual, and emotional abuse; modulating stress levels and contributing to the development of many downstream acute and chronic health issues [22]. These experiences of stress may be carried forward throughout generations [22]; a sobering reality for families in Newark. To this day, activists and allies continue to advocate for health equity, anti-racism, and safety in their communities. On just one 2020 summer day in Newark, for example, over 12,000 individuals participated in a peaceful protest of police brutality [23]. There is, however, more work to be done.

### **8. Medical School Today**

Certainly, the primary purpose of the medical school is to train tomorrow's physicians. However, it is apparent that the school's unique founding, out of extensive negotiations with its community, has had an enduring impact on the school's values. Below, we present some of the ways in which New Jersey Medical School's student body have demonstrated a deep understanding of the school's mission to address the public health needs of its community. The relative success of these efforts is beyond the scope of this manuscript; we encourage readers to review the specific published manuscripts briefly cited below.

One of the major social determinants of health is the healthcare system [39]. Key system factors include insurance coverage, physician availability and cultural humility, available language services, and quality of care [39]. Rutgers New Jersey Medical School currently has many efforts to address these factors as well as other social determinants of health; it is nationally recognized for its community collaboration [11,13]. It is our privilege to highlight a mere select few civic engagement efforts spearheaded by NJMS' medical students and published in academic journals.

Briefly, the African-American Brain Health Initiative is a university-community partnership which "combines community engagement, education and training, and brain health research" [46]. It aims to promote brain health and participation in brain-related

research initiatives among elderly African Americans in Newark [46]. The Ironbound Initiative is a student-led group at Rutgers NJMS, in conjunction with public and private community organization partnerships, which works to build trust between the healthcare system and the growing Latino community [47]. They have partnered with Mantena Global Care, a Brazilian community organization in Newark, to disseminate COVID-19-related information [47].

Benefits of New Jersey Medical School community endeavors extend beyond the provision of healthcare and connection to resources for the local community; they serve as opportunities for medical students to learn from the people they hope to serve. MiniMed is an outreach program designed to "empower the powerless to communicate more effectively with clinicians" via providing opportunities for medical students to interact with people from disadvantaged social groups in a non-threatening context [48]. For example, medical students have prepared and delivered lectures to inmates at Kintock Group facilities [48]. Through partnering with the Kintock Group and Newark Renaissance House, which is a nonprofit residential therapeutic community to assist chemically dependent women and children, participating medical students may become more familiarized with the circumstances, social programs, and healthcare needs for these patients [48]. Other ongoing school efforts are described online [49,50].

### **9. Medical School Tomorrow**

Presently, Rutgers Biomedical and Health Sciences includes two separate medical schools, as well as colleges of nursing, pharmacy, dentistry, and other health sciences [12,33]. A proposal to merge the medical schools, which have campuses in Newark (NJMS) and New Brunswick (Robert Wood Johnson Medical School), has not yet been decided publicly [12]. However, Chancellor Brian Strom, hired to oversee RBHS in 2013, has proposed to combine the medical schools in the future, believing "a single accredited institution stretched over two urban campuses—would be better for students and attract more research dollars" [12]. Currently, Strom serves as Chancellor of both medical schools, is on the board of UH, and is also on the board of the New Jersey Barnabas Health System [51]. Rutgers states the arrangement with RWJ Barnabas Health "is designed to create a higher quality, more sustainable health care system throughout northern and central New Jersey" [12]. Additionally, former Governor Chris Christie, who has acted as a lobbyist for several NJ hospitals during the pandemic, has been hired by RWJ as a consultant [52].

Despite Chancellor Strom's vision, several notable groups have expressed concern about the future in Newark. Indeed, UH is the primary teaching hospital for NJMS. These concerns have come in the form of letters by Senator Ron Rice, on behalf of members of the NJ Legislative Black Caucus, and from various health care unions (Health Professionals and Allied Employees; Communications Workers of America Local 1031; American Association of University Professors Biomedical Health Science of New Jersey) and sent to the Rutgers University President and to New Jersey Governor Phil Murphy [12,14]. They have cautioned that "Rutgers' partnership with a massive private hospital system and efforts to reorganize its two medical schools will drain staff and other resources from the urban hospital, which serves many vulnerable patients in Newark and functions as the state's only public acute-care facility" [12]. Indeed, some fear that UH will lose its status as an academic teaching center or close entirely [12]. It has been asserted that Rutgers plans are being developed "without sufficient public and stakeholder input" [12,15]. Over 1500 faculty from both medical schools have publicly opposed any such medical school merger at this time [15,51]. Faculty have contended that even the Council appointed to review the question of a medical school merger [53] does not sufficiently represent elected faculty leaders from either medical school [15].

Newark Mayor Ras Baraka strongly opposes the merger of the two healthcare systems, characterizing it as "an unregulated and premeditated takeover that will leave Newark residents without critical resources" [13,51]. In an open letter published on NJ.com, he stated this is "one of the ways that systemic racism rears its ugly head" [13]. He fears that

Newark's residents will lose access to quality healthcare at UH and, therefore, current health disparities will be exacerbated. Baraka notes that these discussions violate the Newark Agreements [13]. Proper needs assessments, review for compliance with state laws and regulations, and public hearings—per the Mayor and others—have not been performed [13,14,51]. As he states, "somehow, when it affects those that are disadvantaged, there can be found a way to bypass process and procedure altogether" [13]. Of note, CWA president John Rose has pointed out that "many of [RBHS's] other facilities are in suburban areas that tend to generate more lucrative reimbursements, compared with urban hospitals that care for high numbers of patients on Medicaid" [12].

Publicly, a spokesperson for RWJBarnabas Health explained that "RWJBarnabas Health respects and is sensitive to the unique histories of the medical schools, University Hospital and the City of Newark, and always seeks to demonstrate that respect. We are excited about the opportunities for enhancing the health of all New Jersey residents with our plans, and nothing within RWJBH's relationship with Rutgers University negates either the terms or the spirit of the Newark Agreements" [12]. Rutgers states that UH and "public health in Newark remain priorities for the university as it evolves" [12]. Neal Buccino, associate director of media relations for Rutgers University states that "Rutgers is and remains fully committed to University Hospital and Newark, and no future organizational changes, should they occur, will change that" [12]. They have shared that they "understand the mayor's passion" [51].

While reflecting on these current discussions, we are reminded of a quote by New Jersey Governor Richard J. Hughes upon creation of the Newark Agreements. At that time, he stated that, "the lengthy negotiations were designed to ensure that major advances in housing and relocation, community health services, training and employment opportunities, and community participation will in fact occur when the [school] comes to Newark ... [representing] a pattern of constructive social action that brings together, as full equals, public official and private citizen, Black and white—a pattern that nourishes and dignifies everyone associated with it and that portends only good for Newark" [16].

It appears critical that New Jersey Medical School's future is handled with similar care. The ongoing presence of Rutgers NJMS at 185 South Orange Avenue, Newark, will continue to benefit some of the least privileged members of our society. Continued investment in UH is necessary for it to remain a place of employment, teaching, and healing for Newark's citizens; a 2010 report even stated that, "Rutgers needs to continue to expand not only its academic programming in Newark, but must commit to enhancing an ongoing residential and community presence in the city" [11]. These facts are emphasized by the unprecedented challenges posed by the current COVID-19 pandemic [6,54,55]. Achieving health equity for New Jersey's most vulnerable patients must be a top priority. Striving toward success in this regard requires careful and strategic organization of New Jersey's health workforce. The historic agreements established in the Newark Accords should not be undermined. Accordingly, when considering any business decisions to relocate any faculty, staff, and students from the Newark campus, we believe engaging in a similar approach as the 1968 Newark Agreements is in order. Communication is paramount. Our past teaches us that more creative, and ultimately more mutually beneficial solutions, will be generated by doing so.

### **10. Conclusions**

We believe that it is imperative to know our community's recent history. Elsewhere we have written about the ethical obligation of medical schools, generally, to engage with their communities to improve public health [49]. This article emphasizes that, among medical schools, Rutgers New Jersey Medical School's commitment to Newark is meaningfully unique. Born out of widespread socioeconomic injustices, Rutgers NJMS have committed, indefinitely, to actively promote public health in Newark, codified explicitly in the historic Newark Agreements. We know of no other social contract like this one.

Today's many instances of police brutality against Black and brown people, combined with the global COVID-19 pandemic that disproportionately harms and kills racial and ethnic minorities [7,8,54,55], are reminiscent of the devastation experienced by Newark citizens back in the 1960s. Additionally, current considerations to effectively re-distribute resources from RBHS' Newark medical campus to more affluent hospitals eerily echo Mayor Addonizio's redistribution of funding from the UCC to programs in which he could exert more control in the 1960s.

The Agreements set an important standard for communities across the country: hospital health policy decision-making may be a collaborative endeavor. Accordingly, we believe it is imperative that members of Rutgers NJMS, Rutgers RBHS, and affiliate faculty, staff, residents, and students review Newark's recent history. These members must realign themselves with the commitments made by our own predecessors in the Newark Accords. When considering novel business ventures to potentially alter hospital structures, reorganize the health workforce, or to merge medical schools, all must remember that NJMS as we know it exists only because of extensive negotiations with the people of Newark. The ensuing Agreements lack an expiration date.

Of course, other works have more substantially explored the recent history of our community [19], as well as documented the Newark Accords [16] and the development of New Jersey Medical School [21]. Indeed, Robert Curvin's book *Inside Newark* provides a comprehensive, authoritative analysis of recent sociopolitical developments in our city [26]. The goal of this interdisciplinary manuscript, therefore, is to connect the historical inequities in Newark to the current health care policy discussions in Newark. We found that it was impossible to adequately discuss historical health policy in Newark without simultaneously documenting efforts to achieve racial justice in Newark. Ultimately, we know that Newark's story emphasizes the complex interplay between race, politics, and medicine in shaping a community's past, present, and future public health. We believe the themes of this discussion are universal: Public health matters, local history matters, and creative health policy solutions may be implemented with improved communication. An international audience may find this piece thought-provoking. Hopefully, readers of all backgrounds will apply lessons described in this brief review to pursue health equity within their own communities.

**Author Contributions:** Idea, R.A.B.H., R.C.F., C.A.M., D.J.T. and J.M.P.; methodology, R.A.B.H., R.C.F. and C.A.M.; data curation, R.A.B.H., R.C.F. and C.A.M.; formal analysis, R.A.B.H., R.C.F. and C.A.M.; writing—original draft preparation, R.A.B.H., R.C.F., C.A.M., D.J.T. and J.M.P.; writing—review and editing, R.A.B.H., R.C.F., C.A.M., D.J.T. and J.M.P.; supervision, C.A.M.; funding acquisition, n/a. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** We would like to thank the many community and healthcare leaders in Newark, NJ, for their inspiration and commitment to improving our public health.

**Conflicts of Interest:** J.M.P. is the Public Policy Coordinator at Health Professionals and Allied Employees; D.J.T. is Executive Director at the American Association of University Professors— Biomedical and Health Sciences of New Jersey; C.A.M., R.C.F., D.J.T., and J.M.P. are members of Newark Community Coalition. These organizations have publicly opposed efforts to restructure Rutgers New Jersey Medical School and/or Newark's University Hospital without at least first engaging in a robust process of community discussion. However, none of these organizations have influenced the research nor conclusions made in this article.

### **References**


### *Review* **Planning the Future Oral Health Workforce: A Rapid Review of Supply, Demand and Need Models, Data Sources and Skill Mix Considerations**

**Madhan Balasubramanian 1,2,3,\*, Aliya Hasan 3, Suruchi Ganbavale 3,4, Anfal Alolayah <sup>3</sup> and Jennifer Gallagher <sup>3</sup>**


**Abstract:** Over the last decade, there has been a renewed interest in oral health workforce planning. The purpose of this review is to examine oral health workforce planning models on supply, demand and needs, mainly in respect to their data sources, modelling technique and use of skill mix. A limited search was carried out on PubMed and Web of Science for published scientific articles on oral health workforce planning models between 2010 to 2020. No restrictions were placed on the type of modelling philosophy, and all studies including supply, demand or needs based models were included. Rapid review methods guided the review process. Twenty-three studies from 15 countries were included in the review. A majority were from high-income countries (*n* = 17). Dentists were the sole oral health workforce group modelled in 13 studies; only five studies included skill mix (allied dental personnel) considerations. The most common application of modelling was a workforce to population ratio or a needs-based demand weighted variant. Nearly all studies presented weaknesses in modelling process due to the limitations in data sources and/or non-availability of the necessary data to inform oral health workforce planning. Skill mix considerations in planning models were also limited to horizontal integration within oral health professionals. Planning for the future oral health workforce is heavily reliant on quality data being available for supply, demand and needs models. Integrated methodologies that expand skill mix considerations and account for uncertainty are essential for future planning exercises.

**Keywords:** health workforce; operational models; planning; skill mix; integration

### **1. Introduction**

The health workforce is the backbone of health systems, fundamental towards achieving universal health coverage (UHC) and meeting sustainable development goals (SDGs) [1–3]. Planning for the future health workforce is a complex process, requiring trade-offs across multiple health professional objectives in education, training and regulation, and numerous uncertainties due to transition health environments (demographic, epidemiologic and technology) [4]. In general health workforce planning aims to achieve a proper balance between supply and demand of health professionals [5]. The philosophy behind planning is to ensure the right number of health personnel, with the right training and skill sets are available at the right place and at the right time to meet population needs, but at an acceptable cost and quality [6]. The process is not just technical, but a political one [6]. Planning decisions on the number, type and distribution of health personnel depend and

**Citation:** Balasubramanian, M.; Hasan, A.; Ganbavale, S.; Alolayah, A.; Gallagher, J. Planning the Future Oral Health Workforce: A Rapid Review of Supply, Demand and Need Models, Data Sources and Skill Mix Considerations. *IJERPH* **2021**, *18*, 2891. https:// doi.org/10.3390/ijerph18062891

Academic Editors: Paul B. Tchounwou and Takaaki Tomofuji

Received: 19 January 2021 Accepted: 9 March 2021 Published: 12 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

are influenced by a range of social, economic and professional values enshrined within underlying health systems.

Planning for the future oral health workforce presents its unique challenges. First, the profession of dentistry, in many countries, has remained historically 'distinct' from the medical, nursing and broader health professions [7]. Silos are visible in the education and practice of dental professionals, that also extend to policy and planning decisions [7–9]. Second, dentists are at the center of the dental profession, entrusted with the responsibility of providing leadership, and serving as the first point of contact for the majority of oral health conditions [7]. The allied dental workforce—dental hygienists, dental therapists, dual qualified hygienist/therapists, dental technicians, denturists, prosthetists and dental assistants—support the dentist in the provision of care. However, the acceptance of allied dental professionals vary country to country and potentially reflect on the use of skill mix in planning decisions [10]. Third, dental specialists are the gatekeepers of the profession, providing exceptional services to 'special' patients, and serving as a focal point for quality benchmarks, innovation and adoption of new procedures, clinical research and education of the dental team [11]. Atleast 10 distinct specialist dental professionals exist: orthodontists, oral and maxillofacial surgeons, prosthodontists, periodontists, endodontists, paediatric dentists, oral pathologists, oral medicine, special needs and dental public health specialists. Not all dental specialities gain equal importance in the planning exercise. A further challenge in oral health workforce planning is consideration for both horizontal (i.e., within profession skill mix) and vertical (i.e., skill mix outside the dental profession) integration in planning models.

Traditionally, four broad approaches to health workforce planning have been identified in the literature: needs-based, utilisation or demand-based, health workforce to population ratio, and target setting approach [4,12]. Each of these approaches includes atleast one or more of the basic building blocks in modelling: supply, demand and need [13]. Supply models estimate the number of health personnel available based on the current stocks, flows/migration, and newly trained personnel. Demand or needs model estimate health personnel required to meet the underlying population demand or needs respectively. Needs are identified through epidemiological surveys, accounting for diseases prevalence and health status. Demand is identified through health service utilisation. Supply and demand/needs models are usually presented together, so the combined model can determine the gap in health personnel availability. Planning models are also classified as being deterministic or stochastic [12]. Deterministic models assume the outcome is certain, and always deliver the same results for the same input values. On the other hand, stochastic models allow for the introduction of random changes and provide means for building an element of uncertainty in the overall planning models.

Over the last decade, there has been a renewed interest in oral health workforce planning [14–16]. The purpose of this review is to examine oral health workforce planning models on supply, demand and needs, mainly in respect to their data sources, modelling technique and use of skill mix. We also identify strengths and weaknesses in these workforce models and provide insights on how oral health workforce planning can evolve in the future to meet changing population needs and demands, improving health outcomes and health systems performance.

### **2. Methods**

The study was based on a rapid review approach adapted from Khangura et al. [17] and Thomas et al. [18]. Rapid reviews are a type of systematic reviews, where components of a regular systematic review are simplified or made more efficient to produce information in a shorter span of time, but with minimal impact to quality [19]. In recent years, rapid reviews have emerged as an efficient solution to synthesizing evidence to support health policymaking and health systems strengthening by providing high-quality evidence in a timely and cost-effective manner [20]. Our rapid review involved the following steps: (i) defining a review/research question (ii) developing a search strategy (iii) establishing

selection/eligibility criteria (iv) screening and study selection (v) data extraction and (vi) synthesis of findings. We have adhered to Khangura et al.'s descriptive synthesis of findings and emphasis on translation of findings to policy and practice [18,19]. Targeted searching of key databases, and data abstraction by mapping study characteristics were adopted from Thomas et al. [17,19]. While no generic trend or adherence to any particular variant of rapid reviews have been observed in recent reviews [19,21], our methodological underpinning to key schools of rapid review thought streamlines our approach and philosophy.

### *2.1. Research Question*

The following question was formulated for this part of the review: What are the main operational models, data sources and techniques used in oral health workforce planning?

### *2.2. Search Strategy*

A comprehensive search strategy was designed in consultation with an expert librarian at Kings College London to capture the relevant literature on the topic of interest. We included four broad categories in our search criteria: healthcare and workforce planning, dental service provision, dental staffing, modelling techniques and skill mix. Specific MeSH terms and keywords, along with Boolean operators were used to build the search. This list was refined by conducting a group discussion among all authors to arrive at a consensus. Electronic searches were carried out in 2 different databases: PubMed and Web of Science. The search strategy was designed for PubMed interface, and later revised for Web of Science. A limited literature search was undertaken for relevant titles, abstracts and keywords (please see Supplementary Tables S1 and S4). Standard techniques such as using truncation methods and searching for relevant references from the bibliography provided in searched papers and were also used. Manual forward-backward search or citation tracking of the identified articles were performed using Scopus and Google Scholar. The search process and identification of articles was carried in the second half of 2020, between September and December.

### *2.3. Eligibility Criteria*

Published original research articles on oral health workforce planning were included in the review. Studies need to have followed a workforce modelling approach to estimate the current or future requirements of oral health personnel (dentists, dental specialists, therapists, hygienists, or other allied dental personnel). No restrictions were placed on the type of modelling philosophy, and all studies including supply, demand or needs-based estimates were included. Studies could range from simple dentist population ratios, to more complex skill mix and scenario-based models. Articles published between 2010 and 2020 in English language were included.

Commentaries, reviews, policy briefs, government reports, working papers, opinions, perspectives, conference abstracts, letter to editors, dissertations/thesis, or evidence summaries were excluded in this review. Oral health workforce modelling should have been the main aspect of the paper—studies that only identified oral health workforce requirements without any supporting methods or modelling approaches were excluded. Studies should have also focused on oral health personnel as the basic unit for modelling—studies focusing on dental practices or facilities were excluded.

### *2.4. Study Selection*

First, one of the reviewers (S.G.) identified all articles via database searching, duplicates were excluded and imported the final list into a web/mobile based systematic review management application called Rayyan (Qatar Computing Research Institute, Doha, Qatar) [22]. The tool is mainly designed to expedite the initial screening of abstracts, titles and keywords using a process of semi-automation while incorporating a high level of usability [22]. Duplicates were removed. Four reviewers (M.B., A.H., A.A., S.G.) carried out the selection of articles. Articles that did not fit the eligibility criteria were excluded. If

limited information was available in the initial scanning process, the full text was obtained to determine eligibility. Later, the full text of all selected articles was read, and further limited to only relevant articles based on the selection criteria. Lack of agreement or conflict arising in the selection of articles were resolved through group discussions and consultation with the senior author (J.G.).

### *2.5. Data Extraction, Synthesis and Reporting*

Extraction of data from selected papers was performed by using pre-defined criteria. We extracted a range of study characteristics including: author/year, country of research, aim of study, workforce/population modelled, model type, supply/demand/needs models, data sources, findings, strengths/limitations, policy implications and conclusion. All authors were involved in discussing the emerging data to decide on relevance and decide any modifications in the data extraction framework for the study. Data extraction was conducted using an MS Excel template, which was later developed into a MS Access database for improved usability. We followed a descriptive approach in synthesis and reporting of data, based on Khangura et al.'s rapid review methodology [18]. The focus of this paper is limited to detailed characteristics of the supply, demand and needs model, how these models were developed and the sources of data for these models.

### **3. Results**

A total of *n* = 3047 potential articles were identified through database and citation searching. Following the removal of duplicates, *n* = 2748 articles were available for title/abstract/keyword screening. A total of *n* = 2727 articles were excluded (*n* = 64 after group discussion and conflict resolution), providing *n* = 23 articles for data extraction and qualitative synthesis. Figure 1 provides the PRISMA flowchart of the study selection. A list of selected studies for the rapid review will full citation of articles is provided in Supplementary Table S3.

**Figure 1.** PRISMA Flowchart for the Rapid Review.

### *3.1. Main Study Characteristics*

The main characteristics of the 23 selected studies are provided in Table 1. These publications were from 15 different countries across the world: Australia [23], Canada [24], Chile [25], China [26,27], Japan [28], Kuwait [29], India [30], Ireland [31], Malaysia [32,33], Oman [34], Sri Lanka [35], Taiwan [36], Trinidad & Tobago [37], the United Kingdom [38–41], and the United States of America [42–45]. Seven studies were based in the WHO American Region, followed by the European (*n* = 5) and Western Pacific Regions (*n* = 4). Most of the studies were also based on high-income group World Bank countries (*n* = 17). It is important to note that no studies were identified from the WHO African Region or lowincome group World Bank countries Dentists were the dominant oral health workforce group modelled across 13 studies [24–31,34–36,43,44]. Five studies considered both dental and allied dental workforce (including therapists, hygienists, clinical technicians, denturists) in the workforce models [32,33,38–40]. Four studies specifically modelled the dental specialist workforce, including all dental specialities [41] or covering any of the limited specialist groups: oral and maxillofacial surgeons [23], orthodontists [37] or pediatric dentists [45]. One study has modelled all three oral health workforce groups: dentists, pediatric dentists (specialists) and dental hygienists (allied dental professionals) [42]. The population being modelled in the studies ranged from the full population of the country/region (*n* = 10) [23,25,28–30,34–36,40,43] or limited to include a specific group such as children (*n* = 4) [26,37,42,45], adults (*n* = 3) [31–33], or older people (*n* = 1) [38]. Four studies focused on population-based at a specific catchment area such as province/state (Liaoning Province, China [27]; Kentucky, USA [44]; Georgia, USA [42]) or a service/administrative zone (South Central Strategic Health Authority, England/UK [39]; Canadian Armed Forces service areas, Canada [24])

A number of workforce modelling types were observed in the selected studies, with the most common application being the workforce to population ratio (*n* = 10) [25,27–30,34,36,37,41,44] followed by a needs-based/demand-weighted (*n* = 5) [23,35,38,39,45] variant. One article compared both the workforce to population ratio and needs based demand weighted models in the same study [24]. Four studies used a needs-based model [26,31–33]; and three a demand or utilization based model alone [40,42,43].

### *3.2. Detailed Study Characteristics*

Table 2 presents detailed study characteristics of the supply, demand and needs models along with various data sources and techniques used in developing these models.

### 3.2.1. Supply Models and Data Sources

A total of 18 studies in the review have presented supply models. Existing stock of the dental workforce has been determined in all these studies, with the most common estimation being through the use of dentist registrations data, obtained via a national dental council or a regulatory authority (*n* = 7) [29–31,35,38,41,42,44]. Two studies from the USA have determined estimates using state dental regulatory authorities, namely from Georgia [42] and Kentucky [44]. Brailsford & De Silva [35], prepared a separate national register for the study accommodating registrations, record matching and panel interview to identify existing stock and currency of practice. Gallagher et al. [38] used a range of sources (registrations, dental practice survey and NHS government data) in determining the existing stock of oral health workforce in England, UK. In addition, four other studies have used mainly dentist surveys in accounting for existing workforce numbers [23,27,28,43]. Studies in Australia [23] and Japan [28] have utilized national dental workforce surveys in determining more detailed estimates on the stock of dentists. A few studies have also used government data from sector specific areas such as health services [36,38,39,42] or armed forces [24].




**Table 1.**

*Cont.*



205


**Table 1.**

*Cont.*


**Table 2.** Detailed

characteristics

 of supply and

demand/needs

 model of selected studies.



**Table 2.**

*Cont.*

the

study/approach

210





214


### *IJERPH* **2021**, *18*, 2891

**Table 2.** *Cont.*




218






Surdu et al. [45] documented an elaborate use of national dental association registrations data for determining supply estimates of pediatric dentists, in addition to survey and workforce publications from government sources. Nine studies have included flow estimates within their supply models, through the inclusion of migration, retirement, absence, return to work and deaths [23,28,31,34–36,38,39,45]. Ten studies have included newly trained dentists in the supply model [23,25,28,30,31,34–36,38,39,45], mostly through information available from dental school completions. Brailsford & De Silva [35] also incorporated a student survey to understand student motivations and career expectations. A few studies have also accommodated government regulations and potential for newly created dentists/hygienist places in their supply estimates [34,38,39].

Studies have represented overall workforce participation either through dental personnel numbers alone (*n* = 5) [25,29,30,34,44] or accounting for clinical or part time hours worked and determining full time equivalent dentists (*n* = 7) [24,31,35,38,39,42,45]. Ju et al. [23] and Ishimaru et al. [28] have used work status questions from surveys in determining workforce participation.

### 3.2.2. Demand Models, Population Only Estimates and Data Sources

Demand models, represented as a needs-based demand weighted or utilization/ demand model were presented in nine studies [23,35,38–40,42,43,45]. At the basic level estimates were presented as only population numbers in eight studies [25,27–30,34,36,44]. Population estimates were sourced from national or state-based census sources, government departments, or a combination of both. Seven studies [23,24,35,38–40,45] estimated the expressed demand through available data on oral health status, and converted the demand to workforce requirements as minutes, dentists or FTE dentists. Brailsford De Silva [35] used FDI/WHO method in estimating services needed per person—based on people who actively express the need for care from a population survey. Three studies in the UK (Gallagher [38,39]; Wanyoyi, [40]) have used NHS treatment data to arrive at a very detailed estimates of demand and workforce requirement. A simple estimation of demand was reported in Eklund and Balit [43]—the proportion of dental visits people make in a year (determined form a previous publication) in estimating workforce requirements.

### 3.2.3. Needs Models and Data Sources

The review identified four studies [26,31–33] that have predominantly used a needs model in determining workforce requirements. All four studies used a population survey to determine oral health status and treatment needs. Three studies were limited in survey design or sample size or research question: Sun et al. [26] surveyed only 12-year-olds in China, and Ab-Murat et al. [32,33] surveyed 30–54-year-old university employees at a single site (public university) in Malaysia. Ab-Murat et al. [32,33] also focused on specific aspects covering periodontal and prosthodontic treatment needs, which were measured through two approaches: a normative approach and socio dental approach. Face-to-face questionnaires were also used to determine oral health impacts and behaviours. Both Ab-Murat et al. [32,33] and Sun et al. [26] used panel interviews to determine treatment timings, helping in the estimation of workforce requirement. Sun et al. [26] further expanded the needs aspect (determined for 12-year olds) to whole population in China by utilizing care provision ratios, adopted from a previous study. In contrast, Ahern et al. (2019) [31] used a more comprehensive oral health survey dataset that covered all adults (15+ years old) in Ireland. The population survey included questions on oral health status, behaviours, impacts and visiting patterns to determine service timings and workforce requirements in FTE dentists.

### 3.2.4. Skill Mix Considerations

The use of skill mix in modelling that take into account the contribution or influence of different workforce groups towards supply, demand and/or need models has been limited. Only seven studies accounted for skill mix variations [32,33,38–40,42,45]. The common

application of skill mix was the use of allied dental teams (dental therapists, hygienists, denturists) along with dentists [32,33,38–40,42]. Surdu et al. [45] have applied specialist pediatric dentists along with general dentists in skill mix models for planning pediatric dental workforce. Studies that used skill mix accounted for changes in the provision of services by the extended dental team and how their participation effectively altered the future workforce requirements for oral health care. None of the studies examined the provision of oral health care outside the main oral health workforce groups i.e., accounting for possible care provision by medical, nursing, pharmacy or broader allied health workforce teams. While a few studies have discussed the concept of skill mix within dental teams, they haven't included it within the modelling approaches.

Almost all the models being presented were deterministic; only one study included a stochastic element in their modelling approach [38]. A number of studies identified limitations in relation to data sources, either data being unavailable or on the quality of planning data. Other limitations highlighted were being single site studies, small sample size (see Supplementary Table S3).

### **4. Discussion**

The review examined oral health workforce planning models within the published scientific literature over the last 10 years. Many studies were from high-income countries; no studies were identified from low-income countries and the WHO African region. Calculating workforce to population ratios were the most common modelling approach, followed by needs-based demand weighted approaches. Needs-based approaches had limitations in the population being studied and/or the nature of oral health need assessments being undertaken. Lack of quality data for the modelling exercise is omnipresent in all sources of supply, demand and needs. Very few studies have made use of skill mix considerations in their models. Studies have not accounted for uncertainty of outcomes, or randomness in their modelling exercises, and were mostly deterministic in nature.

Workforce to population ratios, though commonly used in oral health workforce planning studies, represent a crude ratio and bring several shortcomings to the planning process. First, this ratio is based on assumptions of homogeneity across the numerator (i.e., all dental personnel are active and equally productive and will remain so) and that the denominator (i.e., all populations) will have similar oral health needs and will remain constant) [6,12]. This ratio does less justice to address differences in dentist practice activity or productivity (across age, sex, levels of experience, area of practice) or varying levels of oral disease prevalence, dental care utilisation or demographic, socio-economic differences of across population groups. Second, maldistribution of health personnel across different geographic areas, practice types (public or private) or facilities (hospitals, clinics) cannot be adequately represented using a single workforce to population ratio [5]. While it is possible to offer some comparisons using workforce to population ratios at global, region, country, state/area, facility levels, its inability to account for the intrinsic differences in dentist and disease characteristics would still prevail. Third, the ratio does not help us in understanding progress made in achieving wider health system objectives and performance benchmarks in regard to accessibility, equity, quality and efficiency, [6] particularly as the most basic aspect of access 'coverage' within countries can differ, particularly between urban and rural areas (ref)Nevertheless, the workforce to population ratio approach is less demanding in terms of data and it brings simplicity in terms of providing a snapshot estimate to health planners [4,5,46]. Our review identified studies from Kuwait [29], Trinidad and Tobago [37], Chile [25], Oman [34], Taiwan [36], India [30], China [26], UK (dental specialists) [41], and Kentucky (USA) [44] using a workforce to population ratio approach. It should be noted that all these studies also identified limitations in data sources or non-availability of quality data and they have resorted to using workforce to population ratio as a means of commencing the oral health workforce planning process [47].

Demand based planning approaches primarily make use of health service utilisation data. Our review has identified studies that use both dentist surveys [43] and administrative data such as electronic health records (EHR) [40] for extracting oral health service utilisation data. Traditionally, survey-based methods have been popular in understanding practice activity of oral health personnel, and the nature and type of services they offer to patients. For example, in Australia, dentist practice activity surveys have been the cornerstone of oral health workforce policy and planning since early 1980- s [48–51]. In recent years, however, the adoption and use of computerised systems and use of EHRs in dental practices and hospitals are becoming more common in many counties [52]. EHRs provide a viable, cost efficient and timely alternative to understand dental service utilisation data, against surveys that are more time consuming and resource intensive [53,54]. However, the use of EHRs is still in its infancy in terms of data quality and consistency in systems across public and private sectors [55–57]. As a large proportion of dentists practice in the private sector [7], it becomes important to find avenues to improve consistency as well as building data repositories for research and planning purposes. The International Association for Dental Research (the peak global dental research body), and its Network for Practice Based Research [58] has raised the importance of partnerships across private and public dental sector and Universities to improve quality, consistency and use of oral health service data collected in dental clinics or hospitals for research purposes.

Needs based approaches are more reflective of the underlying oral health needs of population. Such models take into account oral health conditions such as caries levels, periodontal status or missing teeth [26,31–33]. Data for needs in selected studies in the review are from population oral health surveys. However, not all studies were comprehensive, and were limited in the type of population group being surveyed or type of needs being assessed, or oral health status questions being studied.

The use of skill mix is a vital component in health workforce models, as it helps to accommodate task sharing and team work both across members of the dental team and wider medical, nursing and allied health teams. Prior planning theories and methods have to failed to incorporate skill-mix in planning designs [59]. The use of skill mix is an important factor when undergoing oral health workforce planning as it helps to determine the future framework of the oral health team in terms of number, size and consequently patient base [38]. In order to predict the future of the oral health workforce, it is important to appreciate the changes within society in terms of comparing and contrasting oral health need and demands, whilst balancing this against the supply of the dental workforce [60]. Vertical integration of the oral health workforce with other health professionals is also vital moving in the future, as in a post COVID era its logical to argue for greater collaboration with all members of the medical, dental and social teams so as to meet the growing needs and demands of the population [7]. Future research in oral health workforce planning needs to accommodate both horizontal and vertical integration within their planning exercises. A major issue for concern is planning the future dental speciality workforce. As gatekeepers of the dental profession, dental specialists are vital towards setting quality benchmarks, identifying divers for innovation and change. Planning exercises will need to extend to involve dental specialists along with general dentists and other members of the dental team in order to best serve the needs of the population.

The study identified lack of consistency and quality of workforce data arising from a wide variety of data sources. Supply data sources have particularly problematic due to the number of sources required to identify the stock, flows and newly trained. Our prior research has identified a range of inconsistencies across countries in these supply data sources and necessity for advocacy and solutions in improving registration and migration data on dental professionals [8].

### *Limitations*

This rapid review included only published scientific research articles between 2010 to 2020. We limited our focus only on the past decade, as several advocacy and major progress on health workforce planning by global, regional and national organisations were prevalent during this period. Health workforce planning could also be conducted as an 'in-house' exercise by planning organisations which is more prevalent in grey literature. Our decision to include only published peer reviewed articles was conscious, mainly due to the rapid review method adopted [17,18], but also able to understand how well oral health workforce planning is represented in our scientific literature. One possible limitation is the fact that the senior author of this review is also active in workforce modelling and has authored several of the publications; however, we took account of this by having a wider research team work on the data extraction and analysis, recognising the importance also of having an expert in the field involved. We also identify a recent review published by a different group of colleagues on a similar topic [61]. Whilst our study differs in terms of review question, methods, and framework used in synthesis, the findings together make a major contribution to this important but relatively unexplored field of oral health workforce planning. The purpose of our review was to distinctly focus on data sources, techniques and skill mix considerations. Our method of synthesis was comprehensive to the above three parameters.

### **5. Conclusions**

Planning for the future oral health workforce is heavily reliant on quality data being available for supply, demand and needs models. Studies have presented with a lack of uniformity and accepted standards in oral health workforce modelling approaches and reporting. Integrated methodologies that expand the skill mix considerations and introduce randomness and system dynamics to account for uncertainty are essential for future planning exercises.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/1660-460 1/18/6/2891/s1, Supplementary Table S1: Search Strategy for the Rapid Review; Supplementary Table S2: Limitations mentioned in selected studies. Supplementary Table S3: List of selected studies for Rapid Review with full citation. Supplementary Table S4: PubMed Search with MeSH terms.

**Author Contributions:** All authors were equally involved in the design, development and conduct of the rapid review. M.B. wrote the first original draft of the manuscript, and revised the manuscript based on feedback from all authors. S.G. and A.H. conducted the search and identified the first list of articles. M.B., A.H., S.G., & A.A. were involved in scanning, identification and data extraction. J.G., provided oversight and supervision. All authors have read and agreed to the published version of the manuscript.

**Funding:** M.B. is supported by an NHMRC Sidney Sax Research Fellowship (GNT: 1121576). S.G. is supported by the Portsmouth Global PhD Scholarship (2019–2022). The contents are solely the responsibility of the administering institution and the authors and do not reflect the views of funding bodies.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** All data presented in the study are available in the tables and figures included in this paper.

**Conflicts of Interest:** J.G. is a co-author on a few of the published papers in this review.

### **References**


### *Comment*

**Blood Lead Concentrations in Newark Children. Comment on Franklin, R.C.; Behmer Hansen, R.A.; Pierce, J.M.; Tsitouras, D.J.; Mazzola, C.A. Broken Promises to the People of Newark: A Historical Review of the Newark Uprising, the Newark Agreements, and Rutgers New Jersey Medical School's Commitments to Newark.** *Int. J. Environ. Res. Public Health* **2021,** *18***, 2117.**

**James M. Oleske 1,\* and John D. Bogden <sup>2</sup>**

**Citation:** Oleske, J.M.; Bogden, J.D. Blood Lead Concentrations in Newark Children. Comment on Franklin, R.C.; Behmer Hansen, R.A.; Pierce, J.M.; Tsitouras, D.J.; Mazzola, C.A. Broken Promises to the People of Newark: A Historical Review of the Newark Uprising, the Newark Agreements, and Rutgers New Jersey Medical School's Commitments to Newark. *Int. J. Environ. Res. Public Health* 2021, *18*, 2117. *IJERPH* **2021**, *18*, 2887.

https://doi.org/10.3390/ijerph18062887

Academic Editors: Madhan Balasubramanian and Stephanie Short

Received: 3 March 2021 Accepted: 10 March 2021 Published: 12 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

<sup>1</sup> Department of Pediatrics, Rutgers New Jersey Medical School, Newark, NJ 07102, USA

<sup>2</sup> Department of Microbiology, Biochemistry, and Molecular Genetics, Rutgers New Jersey Medical School, Newark, NJ 07102, USA; john.bogden@rutgers.edu

**\*** Correspondence: oleskejm@njms.rutgers.edu

The recently published article of RC Franklin et al. in the *International Journal of Environmental Research and Public Health* [1] provides a detailed historical review, beginning with the decades of the 1950s and 1960s, of health disparities of Black and Hispanic Newark, New Jersey residents that included substance abuse and sexually transmitted infections, as well as the highest incidence of tuberculosis and maternal and infant mortality in the United States. The manuscript also notes that current Newark residents continue to experience a high incidence of chronic diseases and deficiencies that include food insecurity, hypertension and strokes, diabetes, coronary artery disease, and loss of all their teeth. The article, however, does not describe another serious and glaring health disparity, the long history of high past and current blood–lead concentrations and lead poisoning in Newark children.

One author of this article (JDB) started work as a postdoctoral researcher at New Jersey Medical School in 1971, and was responsible for developing the initial "Lead Laboratory". This laboratory analyzed blood samples of Newark children 5 days a week, with 50–100 + samples received daily; many were "stat" samples. The other author (JMO) was a recent New Jersey Medical School graduate beginning a residency in Pediatrics at the Martland Medical Center; his inpatients included children hospitalized with lead poisoning. The high blood–lead levels (BLLs) of children revealed by our laboratory analyses at that time were primarily the result of ingestion of small paint chips from peeling indoor paint and/or inhalation of indoor lead-containing paint dust. A key factor was a lack of adequate maintenance and remediation/renovation of painted surfaces in Newark housing rented by low-income families. Inhalation of airborne lead from automobile exhaust was also a major source of exposure for both children and adults. Lead is a cumulative toxin and continued daily exposure can eventually result in elevated BLLs with significant negative multi-organ system health consequences, including hematological and neurological abnormalities. A significant percent of Newark children during this era had elevated blood–lead concentrations high enough to require emergency hospitalization for multi-organ system abnormalities that even became life-threatening in scope. Lead is a neurotoxin and many of these children were diagnosed with mild cognitive dysfunction and even more severe permanent brain damage.

Although other large USA cities in the Northeast (Philadelphia, Baltimore, New York City) also had poorly maintained housing and many children with lead poisoning,

Newark had an especially severe problem [2], with 41.2% of 25,260 mostly Newark children tested between 1970 and 1976 having BLLs ≥30 mcg/dL. Of these 15.6% had high BLLs ≥ 40 mcg/dL and 1.8% had dangerously high BLLs ≥60 mcg/dL that typically required hospitalization for observation for toxicity and intravenous pharmacologic therapy. The substantial number of Newark children found to have high BLLs during the early 1970s may be explained by the older age of much Newark housing, inadequate maintenance efforts to prevent and remove peeling and flaking paint, and our extensive testing of children's BLLs—enabled by substantial funding. In contrast, more than 95% of young New Jersey children currently have BLLs less than 2.0 mcg/dL, but we could not find a single Newark child in the 1970s with a BLL below 5.0 mcg/dL.

The lead crisis of the 1960s and 1970s would have been worse without the dedicated care provided by Newark pediatricians at St Michael's Medical Center, Newark Beth Israel Medical Center, and our faculty at Martland Hospital. A clinical challenge in the 1970s for pediatricians was the collection of an adequate volume (1.5 mL or more) of venous blood for lead analysis from screaming children in the presence of their anxious parents —most often the mothers, but sometimes the more threatening fathers. Handing out lollipops helped, but it was the skill and rapidity of the blood drawer as well as the good work of the holder of the squirming/moving/crying child that saved the day.

New Jersey law requires pediatricians to order testing of BLLs of all children at both 12 and 24 months of age, and also prior to age 6 for all children not tested when younger. Children with known or suspected lead exposure should also be tested. Compliance with these regulations in New Jersey has been very good, with 86% of these children tested in 2018. This testing reveals that, although the mean BLLs of Newark and other New Jersey children are much lower now [3], there are still New Jersey children with elevated BLLs greater than the current guideline of 5.0 mcg/dL.

As an element, lead cannot decompose, and thus has an infinite environmental halflife. It was present in paint used in almost all older homes built in the United States before the 1970s. Although much of this housing in Newark and elsewhere in the United States has been professionally "de-leaded", in other housing the lead is still there in lower layers of dried paint on painted surfaces such as walls and windows, where it is especially prone to flaking and chipping. The city of Newark is using funding from a US Department of Housing and Urban Development (HUD) grant to continue abatement/removal of lead paint in housing and is also finishing its effort to the replace about 18,000 "lead service" plumbing lines to provide lead-free water to many of Newark's houses.

In older housing that has not undergone lead abatement, layers of dried lead paint will still be on painted surfaces dozens and even hundreds of years from now. Thus, lead exposure will continue to be a threat to young children and the need to test them for elevated BLLs should be recognized and continued.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*International Journal of Environmental Research and Public Health* Editorial Office E-mail: ijerph@mdpi.com www.mdpi.com/journal/ijerph

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18