How to Engage Health Care Workers in the Evaluation of Hospitals: Development and Validation of BSC-HCW1—A Cross-Sectional Study
Abstract
:1. Introduction
1.1. History of Balanced Scorecard (BSC)
1.2. The Impact of BSC
1.3. BSC Perspectives and Dimensions
1.4. Health Care Sector and HCWs in Palestine
1.5. The Conceptual Framework
1.5.1. Managerial Perspective
1.5.2. Financial Compensation and Rewards Perspective
1.5.3. Knowledge and Growth Perspective
1.5.4. Technology Perspective
1.5.5. Internal Process Perspective
1.5.6. External Perspective
1.5.7. Customer Perspective
1.5.8. Sociodemographic Factors
2. Methods
2.1. Research Design
2.2. Item and Scale Generation
2.3. Linguistic Validation and Translation
2.4. Pretest and Internal Consistency
2.5. Sampling Procedure and Power Calculation
2.6. Ethical Consideration
2.7. Data Collection and Participants
2.8. Statistical Analysis
2.8.1. EFA
2.8.2. CFA
2.8.3. Correlations
2.8.4. Convergent and Discriminant/Divergent Validity
3. Results
3.1. Item Generation and Scoring
3.2. The Instrument’s Structure and Items
3.2.1. The Managerial Perspective
3.2.2. The Financial Perspective
3.2.3. The Internal Perspective
3.2.4. The Knowledge and Growth Perspective
3.2.5. The Technology Perspective
3.2.6. The Environment/External Perspective
3.2.7. The Customer Perspective
3.3. The Pretest and the Internal Consistency
3.4. Linguistic Validation and Translation
3.5. Sample Size and Characteristics
3.6. Statistical Analysis
3.6.1. Testing the Normal Distribution
3.6.2. Structural Validity in EFA
3.6.3. Structural Validity in CFA
3.6.4. Internal Consistency
3.6.5. Convergent and Discriminant Validity
4. Discussion
4.1. Discussion of the Main Results
4.2. Comparison with BSC Studies
4.3. Strengths and Limitations
4.4. Practical Implications
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Construct | Code | No. | Question |
---|---|---|---|
FIN | ESF2 | Q1 | I receive financial incentives based on my performance |
ESF1 | Q2 | I feel that my salary suits my responsibilities and competencies | |
TECH | EST3 | Q3 | I believe that hospital information system interface is user friendly |
EST5 | Q4 | I believe that hospital information system and technology at this hospital makes generating reports easier, faster, and more accurate | |
EST1 | Q5 | This hospital has a technology/information system | |
EST6 | Q6 | I believe that hospital information system and technology at this hospital makes my work efficient and productive | |
WTLB | ESTI4 | Q7 | The quantity of work assigned to me is reasonable with the time given |
ESTI1 | Q8 | I have sufficient time to rest and eat during my working day | |
ESTI3 | Q9 | I am able to make a work–life balance and a good time management | |
ESTI2 | Q10 | I am able to spend a sufficient time with each patient | |
QUALDEV | ESD3 | Q11 | The hospital provides me education on medication updates that is related to my specialty |
ESD1 | Q12 | The hospital provides me education updates regarding the diseases in my specialty | |
ESQ2 | Q13 | The hospital medications and disposables are of high quality | |
ESQ1 | Q14 | The hospital equipment helps me in offering high quality medical services for patients | |
ESQ3 | Q15 | Quality is top priority at this hospital | |
ENG | ESMO5 | Q16 | My manager engages me in the planning and taking decision process |
ESMO6 | Q17 | I am given enough authority and power to make decisions in my position | |
ESMO4 | Q18 | My manager understands and adequately support me when I face an urgent hard situation | |
MGMTEVAL | ESE2 | Q19 | My direct superiors explain and discuss the strengths and weaknesses in my assessment with me |
ESM2 | Q20 | I believe that my superiors are taking the right decisions in work which supports the hospital strategy | |
ESM1 | Q21 | I believe that my superiors have the required competencies for their positions | |
ESE3 | Q22 | I believe that my assessment is fair and reflects my actual performance compared to your colleagues | |
MTR | ESC5 | Q23 | I trust what my direct manager tells or promises me with |
PTR | ESR3 | Q24 | I belief that patients respect healthcare workers at this hospital and trust them |
LOY | ESL3 | Q25 | I believe and feel that I want to keep working in this hospital for several years |
ESL5 | Q26 | I recommend this hospital to other colleagues or praise the hospital | |
ESL4 | Q27 | I believe and feel that my overall satisfaction is high | |
ESR4 | Q28 | I am proud to work with this hospital |
References
- Kaplan, R.; Norton, D. The Balanced Scorecard—Measures That Drive Performance. Harv. Bus. Rev. 1992, 70, 71–79. [Google Scholar] [PubMed]
- Amer, F.; Hammoud, S.; Khatatbeh, H.; Lohner, S.; Boncz, I.; Endrei, D. A Systematic Review: The Dimensions to Evaluate Health Care Performance and an Implication during the Pandemic. BMC Health Serv. Res. 2022, 22, 621. [Google Scholar] [CrossRef] [PubMed]
- Amer, F.; Hammoud, S.; Khatatbeh, H.; Lohner, S.; Boncz, I.; Endrei, D. The Deployment of Balanced Scorecard in Health Care Organizations: Is It Beneficial? A Systematic Review. BMC Health Serv. Res. 2022, 22, 65. [Google Scholar] [CrossRef]
- Speckbacher, G.; Bischof, J.; Pfeiffer, T. A Descriptive Analysis on the Implementation of Balanced Scorecards in German-Speaking Countries. Manag. Account. Res. 2003, 14, 361–388. [Google Scholar] [CrossRef]
- Bohm, V.; Lacaille, D.; Spencer, N.; Barber, C.E. Scoping Review of Balanced Scorecards for Use in Healthcare Settings: Development and Implementation. BMJ Open Qual. 2021, 10, e001293. [Google Scholar] [CrossRef]
- Kalender, Z.T.; Vayvay, Ö. The Fifth Pillar of the Balanced Scorecard: Sustainability. Procedia-Soc. Behav. Sci. 2016, 235, 76–83. [Google Scholar] [CrossRef]
- Duke-Children’s-Hospital Duke Children’ s Hospital Case Abstract. Available online: https://thepalladiumgroup.com/download?file=AED_1551102784_116attachment_dubai_boot_camp_-_march_2019_opt.pdf%23Dubai_Boot_Camp_-_March_2019_opt.pdf (accessed on 15 May 2021).
- Gagliardi, A.R.; Lemieux-Charles, L.; Brown, A.D.; Sullivan, T.; Goel, V. Barriers to Patient Involvement in Health Service Planning and Evaluation: An Exploratory Study. Patient Educ. Couns. 2008, 70, 234–241. [Google Scholar] [CrossRef] [PubMed]
- Anderson, N.N.; Baker, G.R.; Moody, L.; Scane, K.; Urquhart, R.; Wodchis, W.P.; Gagliardi, A.R. Approaches to Optimize Patient and Family Engagement in Hospital Planning and Improvement: Qualitative Interviews. Health Expect. 2021, 24, 967–977. [Google Scholar] [CrossRef]
- Korlén, S.; Richter, A.; Amer-Wåhlin, I.; Lindgren, P.; von Thiele Schwarz, U. The Development and Validation of a Scale to Explore Staff Experience of Governance of Economic Efficiency and Quality (GOV-EQ) of Health Care. BMC Health Serv. Res. 2018, 18, 963. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Curtis, T.; Abratt, R.; Rhoades, D.; Dion, P. Customer Loyalty, Repurchase and Satisfaction: A Meta-Analytical Review. J. Consum. Satisf. Dissatisf. Complain. Behav. 2011, 24, 1–26. [Google Scholar]
- Giacaman, R.; Abdul-Rahim, H.F.; Wick, L. Health Sector Reform in the Occupied Palestinian Territories (OPT): Targeting the Forest or the Trees? Health Policy Plan. 2003, 18, 59–67. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Barghouthi, M.; Lennock, J. Health in Palestine: Potential and Challenges; Palestine Economic Policy Research Institute (MAS): Jerusalem, Israel, 1997. [Google Scholar]
- World Health Organisation (WHO). Health Conditions in the Occupied Palestinian Territory, Including East Jerusalem, and in the Occupied Syrian Golan Report; World Health Organisation: Geneva, Switzerland, 2020. [Google Scholar]
- Ali, N.A. Violence against Health Professionals in Palestinian Hospitals: Prevalence, Sources, Responses and Prevention. Bethlehem Univ. J. 2012, 31, 26–43. [Google Scholar]
- Saeedi, M.; Al-Othman, N.; Rabayaa, M. Breaching the Bridge: An Investigation into Doctor-Patient Miscommunication as a Significant Factor in the Violence against Healthcare Workers in Palestine. Biomed Res. Int. 2021, 2021, 1–8. [Google Scholar] [CrossRef] [PubMed]
- Hammoudeh, W.; Kienzler, H.; Meagher, K.; Giacaman, R. Social and Political Determinants of Health in the Occupied Palestine Territory (OPT) during the COVID-19 Pandemic: Who Is Responsible? BMJ Glob. Health 2020, 5, e003683. [Google Scholar] [CrossRef]
- Maraqa, B.; Nazzal, Z.; Zink, T. Palestinian Health Care Workers’ Stress and Stressors During COVID-19 Pandemic: A Cross-Sectional Study. J. Prim. Care Community Health 2020, 11, 2150132720955026. [Google Scholar] [CrossRef] [PubMed]
- Alser, O.; Alghoul, H.; Alkhateeb, Z.; Hamdan, A.; Albarqouni, L.; Saini, K. Healthcare Workers Preparedness for COVID-19 Pandemic in the Occupied Palestinian Territory: A Cross-Sectional Survey. BMC Health Serv. Res. 2021, 21, 766. [Google Scholar] [CrossRef] [PubMed]
- Mabuchi, S.; Alonge, O.; Tsugawa, Y.; Bennett, S.; Bennett, S. Measuring Management Practices in Primary Health Care Facilities—Development and Validation of Management Practices Scorecard in Nigeria. Glob. Health Action 2020, 13, 1763078. [Google Scholar] [CrossRef]
- Xuan Tran, B.; Van Minh, H.; Hinh, N.D. Factors Associated with Job Satisfaction among Commune Health Workers: Implications for Human Resource Policies. Glob. Health Action 2013, 6, 18619. [Google Scholar] [CrossRef] [PubMed]
- Momanyi, G.O.; Adoyo, M.A.; Mwangi, E.M.; Mokua, D.O. Value of Training on Motivation among Health Workers in Narok County, Kenya. Pan Afr. Med. J. 2016, 23, 261. [Google Scholar] [CrossRef]
- Amer, F.; Hammoud, S.; Onchonga, D.; Alkaiyat, A.; Nour, A.; Endrei, D.; Boncz, I. Assessing Patient Experience and Attitude: BSC-PATIENT Development, Translation, and Psychometric Evaluation—A Cross-Sectional Study. Int. J. Environ. Res. Public Health 2022, 19, 7149. [Google Scholar] [CrossRef] [PubMed]
- Aklilu, M.; Warku, W.; Tadele, W.; Mulugeta, Y.; Usman, H.; Alemu, A.; Abdela, S.; Hailemariam, A.; Birhanu, E.; La Torre, G. Assessment of Job Satisfaction Level and Its Associated Factors among Health Workers in Addis Ababa Health Centers: A Cross-Sectional Study. Adv. Public Health 2020, 2020, 1–6. [Google Scholar] [CrossRef]
- Deressa, W.; Worku, A.; Abebe, W.; Gizaw, M.; Amogne, W. Availability and Use of Personal Protective Equipment and Satisfaction of Healthcare Professionals during COVID-19 Pandemic in Addis Ababa, Ethiopia. Arch. Public Health 2021, 79, 1–35. [Google Scholar] [CrossRef] [PubMed]
- Purvis, B.; Mao, Y.; Robinson, D. Three Pillars of Sustainability: In Search of Conceptual Origins. Sustain. Sci. 2019, 14, 681–695. [Google Scholar] [CrossRef] [Green Version]
- Khatatbeh, H.; Al-Dwaikat, T.; Oláh, A.; Onchonga, D.; Hammoud, S.; Amer, F.; Prémusz, V.; Pakai, A. The Relationships between Paediatric Nurses’ Social Support, Job Satisfaction and Patient Adverse Events. Nurs. Open 2021, 8, 3575–3582. [Google Scholar] [CrossRef]
- Pandey, M. Impact of Work-Life Balance on Job Satisfaction of Women Doctors. Probl. Perspect. Manag. 2016, 14, 319–324. [Google Scholar] [CrossRef] [Green Version]
- Poghosyan, L.; Aiken, L.H.; Sloane, D.M. Factor Structure of the Maslach Burnout Inventory. Int. J. Nurs. Stud. 2009, 46, 894–902. [Google Scholar] [CrossRef] [Green Version]
- Kalliath, T.; Morris, R. Job Satisfaction among Nurses: A Predictor of Burnout Levels. J. Nurs. Adm. 2002, 32, 648–654. [Google Scholar] [CrossRef]
- Deriba, B.K.; Sinke, S.O.; Ereso, B.M.; Badacho, A.S. Health Professionals’ Job Satisfaction and Associated Factors at Public Health Centers in West Ethiopia. Hum. Resour. Health 2017, 15, 1–7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Batura, N.; Skordis-Worrall, J.; Thapa, R.; Basnyat, R.; Morrison, J. Is the Job Satisfaction Survey a Good Tool to Measure Job Satisfaction amongst Health Workers in Nepal? Results of a Validation Analysis. BMC Health Serv. Res. 2016, 16, 308. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- American Psychological Association APA Dictionary of Psychology. Available online: https://dictionary.apa.org/ (accessed on 19 February 2022).
- Keiningham, T.L.; Cooil, B.; Aksoy, L.; Andreassen, T.W.; Weiner, J. The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Customer Retention, Recommendation, and Share-of-Wallet. Manag. Serv. Qual. Int. J. 2007, 17, 361–384. [Google Scholar] [CrossRef] [Green Version]
- Bonenberger, M.; Aikins, M.; Akweongo, P.; Wyss, K. The Effects of Health Worker Motivation and Job Satisfaction on Turnover Intention in Ghana: A Cross-Sectional Study. Hum. Resour. Health 2014, 12, 43. [Google Scholar] [CrossRef] [Green Version]
- Ayalew, E.; Workineh, Y. Nurses’ Intention to Leave Their Job and Associated Factors in Bahir Dar, Amhara Region, Ethiopia, 2017. BMC Nurs. 2020, 19, 1–7. [Google Scholar] [CrossRef]
- Blaauw, D.; Ditlopo, P.; Maseko, F.; Chirwa, M.; Mwisongo, A.; Bidwell, P.; Thomas, S.; Normand, C. Comparing the Job Satisfaction and Intention to Leave of Different Categories of Health Workers in Tanzania, Malawi, and South Africa. Glob. Health Action 2013, 6, 127–137. [Google Scholar] [CrossRef] [PubMed]
- Masum, A.K.M.; Azad, M.A.K.; Hoque, K.E.; Beh, L.-S.; Wanke, P.; Arslan, Ö. Job Satisfaction and Intention to Quit: An Empirical Analysis of Nurses in Turkey. PeerJ 2016, 4, e1896. [Google Scholar] [CrossRef] [Green Version]
- Taneja, U. Brand Image to Loyalty through Perceived Service Quality and Patient Satisfaction: A Conceptual Framework. Health Serv. Manag. Res. 2021, 34, 250–257. [Google Scholar] [CrossRef]
- Nilsson, K.; Hertting, A.; Petterson, I.L.; Theorell, T. Pride and Confidence at Work: Potential Predictors of Occupational Health in a Hospital Setting. BMC Public Health 2005, 5, 92. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Willits, F.K.; Theodori, G.L.; Luloff, A.E. Another Look at Likert Scales. J. Rural Soc. Sci. 2016, 31, 126. [Google Scholar]
- Kaarna, M. The Importance of Job Satisfaction in Hospital Quality Processes; Nordic School of Public Health: Gothenburg, Sweden, 2007. [Google Scholar]
- Hayduk, L.A.; Littvay, L. Should Researchers Use Single Indicators, Best Indicators, or Multiple Indicators in Structural Equation Models? BMC Med. Res. Methodol. 2012, 12, 159. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Petrescu, M. Marketing Research Using Single-Item Indicators in Structural Equation Models. J. Mark. Anal. 2013, 1, 99–117. [Google Scholar] [CrossRef] [Green Version]
- Krueger, P.; Brazil, K.; Lohfeld, L.; Edward, H.G.; Lewis, D.; Tjam, E. Organization Specific Predictors of Job Satisfaction: Findings from a Canadian Multi-Site Quality of Work Life Cross-Sectional Survey. BMC Health Serv. Res. 2002, 2, 6. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Domagała, A.; Bała, M.M.; Storman, D.; Peña-Sánchez, J.N.; Świerz, M.J.; Kaczmarczyk, M.; Storman, M. Factors Associated with Satisfaction of Hospital Physicians: A Systematic Review on European Data. Int. J. Environ. Res. Public Health 2018, 15, 2546. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tenaw, Z.; Siyoum, M.; Tsegaye, B.; Werba, T.B.; Bitew, Z.W. Health Professionals Job Satisfaction and Associated Factors in Ethiopia: A Systematic Review and Meta-Analysis. Health Serv. Res. Manag. Epidemiol. 2021, 8, 1–12. [Google Scholar] [CrossRef]
- Jahrami, H.; Alshuwaikh, Z.; Panchasharam, G.; Saif, Z. Job Satisfaction Survey of Healthcare Workers in the Psychiatric Hospital. Bahrain Med. Bull. 2011, 33, 199. [Google Scholar]
- van Pelt, S.; Massar, K.; van der Eem, L.; Shields-Zeeman, L.; de Wit, J.B.F.; Ruiter, R.A.C. “If You Don’t Have Enough Equipment, You’re Not Going to Provide Quality Services”: Healthcare Workers’ Perceptions on Improving the Quality of Antenatal Care in Rural Tanzania. Int. J. Afr. Nurs. Sci. 2020, 13, 100232. [Google Scholar] [CrossRef]
- Hulin, C.L.; Smith, P.C. A Linear Model of Job Satisfaction. J. Appl. Psychol. 1965, 49, 209–216. [Google Scholar] [CrossRef]
- Kim, M.K.; Arsenault, C.; Atuyambe, L.M.; Kruk, M.E. Predictors of Job Satisfaction and Intention to Stay in the Job among Health-Care Providers in Uganda and Zambia. Int. J. Qual. Health Care 2021, 33, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Shahnazi, H.; Daniali, S.S.; Sharifirad, G. Job Satisfaction Survey among Health Centers Staff. J. Educ. Health Promot. 2014, 3, 35. [Google Scholar] [CrossRef] [PubMed]
- Kaplan, R.S. Conceptual Foundations of the Balanced Scorecard. Handbooks Manag. Account. Res. 2009, 3, 1253–1269. [Google Scholar] [CrossRef]
- Boateng, G.O.; Neilands, T.B.; Frongillo, E.A.; Melgar-Quiñonez, H.R.; Young, S.L. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front. Public Health 2018, 6, 149. [Google Scholar] [CrossRef] [PubMed]
- Simbar, M.; Rahmanian, F.; Nazarpour, S.; Ramezankhani, A.; Eskandari, N.; Zayeri, F. Design and Psychometric Properties of a Questionnaire to Assess Gender Sensitivity of Perinatal Care Services: A Sequential Exploratory Study. BMC Public Health 2020, 20, 1–13. [Google Scholar] [CrossRef]
- Dalkey, N.; Helmer, O. An Experimental Apllication of Deplhi Method to Use of Experts. Manag. Sci. 1963, 9, 458–467. [Google Scholar] [CrossRef]
- Zamanzadeh, V.; Ghahramanian, A.; Rassouli, M.; Abbaszadeh, A.; Alavi-Majd, H.; Nikanfar, A.-R. Design and Implementation Content Validity Study: Development of an Instrument for Measuring Patient-Centered Communication. J. Caring Sci. 2015, 4, 165–178. [Google Scholar] [CrossRef]
- Tyser, A.R.; Abtahi, A.M.; McFadden, M.; Presson, A.P. Evidence of Non-Response Bias in the Press-Ganey Patient Satisfaction Survey. BMC Health Serv. Res. 2016, 16, 350. [Google Scholar] [CrossRef] [Green Version]
- Cho, E.; Kim, S. Cronbach’s Coefficient Alpha: Well Known but Poorly Understood. Organ. Res. Methods 2015, 18, 207–230. [Google Scholar] [CrossRef]
- Thompson, S.K. Sampling, 3rd ed.; A John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2012. [Google Scholar]
- Palestine Economic Policy Research Institute. MAS Palestinian Health Sector Assessment: An Analytical Study at the Macro Level; Palest; Palestine Economic Policy Research Institute: Ramallah, Palestine, 2020; pp. 1–98. [Google Scholar]
- Comrey, A.L.; Lee, H.B. A First Course in Factor Analysis; Psychology Press: London, UK, 2013; ISBN 9781315827506. [Google Scholar]
- Williams, B.; Onsman, A.; Brown, T. Exploratory Factor Analysis: A Five-Step Guide for Novices. Australas. J. Paramed. 2010, 8. [Google Scholar] [CrossRef] [Green Version]
- Bentler, P.M.; Chou, C.-P. Practical Issues in Structural Modeling. Sociol. Methods Res. 1987, 16, 78–117. [Google Scholar] [CrossRef]
- Knafl, G.J.; Grey, M. Factor Analysis Model Evaluation through Likelihood Cross-Validation. Stat. Methods Med. Res. 2007, 16, 77–102. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- PMOH. Health Annual Report, Palestine 2020. May 2021. Available online: https://Moh.Ps (accessed on 10 April 2022).
- Sedgwick, P. Non-Response Bias versus Response Bias. BMJ 2014, 348, g2573. [Google Scholar] [CrossRef]
- David Susman Attitudes and Behavior in Psychology. Available online: https://www.verywellmind.com/ (accessed on 19 February 2022).
- Henry Mintzberg: Mintzberg on Management. Inside Our Strange World of Organizations 1989, New York and London: Free Press/Collier Macmillan. 418 Pages. Organ. Stud. 1990, 11, 599. [CrossRef]
- Larsen, R.; Warne, R.T. Estimating Confidence Intervals for Eigenvalues in Exploratory Factor Analysis. Behav. Res. Methods 2010, 42, 871–876. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Cattell, R.B. The Scree Test for The Number Of Factors. Multivariate Behav. Res. 1966, 1, 245–276. [Google Scholar] [CrossRef] [PubMed]
- Hooper, D.; Coughlan, J.; Mullen, M.R. Structural Equation Modelling: Guidelines for Determining Model Fit. Electron. J. Bus. Res. Methods 2008, 6, 53–60. [Google Scholar] [CrossRef]
- Shi, D.; Lee, T.; Maydeu-Olivares, A. Understanding the Model Size Effect on SEM Fit Indices. Educ. Psychol. Meas. 2019, 79, 310–334. [Google Scholar] [CrossRef]
- Cohen, J. Set Correlation and Contingency Tables. Appl. Psychol. Meas. 1988, 12, 425–434. [Google Scholar] [CrossRef]
- Tavakol, M.; Dennick, R. Making Sense of Cronbach’s Alpha. Int. J. Med. Educ. 2011, 2, 53–55. [Google Scholar] [CrossRef]
- Peterson, R.A.; Kim, Y. On the Relationship between Coefficient Alpha and Composite Reliability. J. Appl. Psychol. 2013, 98, 194–198. [Google Scholar] [CrossRef]
- Ab Hamid, M.R.; Sami, W.; Mohmad Sidek, M.H. Discriminant Validity Assessment: Use of Fornell & Larcker Criterion versus HTMT Criterion. J. Phys. Conf. Ser. 2017, 890, 012163. [Google Scholar] [CrossRef]
- Hair, J.; Hult, G.T.M.; Ringle, C.S.M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM); SAGE Publications, Incorporated: Los Angeles, CA, USA, 2014; ISBN 78-1-4522-1744-4. [Google Scholar]
- Fornell, C.; Larcker, D.F. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
- Hair, J.F., Jr.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis, 7th ed.; Pearson Prentice Hall: Upper Saddle River, NJ, USA, 2009. [Google Scholar]
- Plichta, S.B.; Kelvin, E.A. Munro’s Statistical Methods for Health Care Research, 6th ed.; Lippincott Williams & Wilkins: Philadelphia, PA, USA, 2011; ISBN 9781451144802. [Google Scholar]
- Bookter, A.I. Convergent and Divergent Validity of the Learning Transfer Questionnaire; Agricultural & Mechanical College, Louisiana State University: Baton Rouge, LA, USA, 1999. [Google Scholar]
- Kaiser, H.F.; Rice, J. Little Jiffy, Mark Iv. Educ. Psychol. Meas. 1974, 34, 111–117. [Google Scholar] [CrossRef]
- Hu, L.; Bentler, P.M. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
- Baumgartner, H.; Homburg, C. Applications of Structural Equation Modeling in Marketing and Consumer Research: A Review. Int. J. Res. Mark. 1996, 13, 139–161. [Google Scholar] [CrossRef] [Green Version]
- Iqbal, M.R.; Chaudhuri, A. COVID-19: Results of a National Survey of United Kingdom Healthcare Professionals’ Perceptions of Current Management Strategy—A Cross-Sectional Questionnaire Study. Int. J. Surg. 2020, 79, 156–161. [Google Scholar] [CrossRef]
- McGill, N.; Weller-Newton, J.; Lees, C. A New Survey Tool for Evaluating Pandemic Preparedness in Health Services. BMC Health Serv. Res. 2022, 22, 708. [Google Scholar] [CrossRef] [PubMed]
- VanGeest, J.B.; Johnson, T.P.; Welch, V.L. Methodologies for Improving Response Rates in Surveys of Physicians. Eval. Health Prof. 2007, 30, 303–321. [Google Scholar] [CrossRef] [PubMed]
- Flanigan, T.; McFarlane, E.; Cook, S. Conducting Survey Research among Physicians and Other Medical Professionals: A Review of Current Literature. Sect. Surv. Res. Methods 2008, 1, 4136–4147. [Google Scholar]
- Fincham, J.E. Response Rates and Responsiveness for Surveys, Standards, and the Journal. Am. J. Pharm. Educ. 2008, 72, 43. [Google Scholar] [CrossRef] [Green Version]
- Smith, H.; Kim, I. Balanced Scorecard at Summa Health System. J. Corp. Account. Financ. 2005, 16, 65–72. [Google Scholar] [CrossRef]
- Yang, M.-C.; Tung, Y.-C. Using Path Analysis to Examine Causal Relationships Among Balanced Scorecard Performance Indicators for General Hospitals. Health Care Manag. Rev. 2006, 31, 280–288. [Google Scholar] [CrossRef]
- Lupi, S.; Verzola, A.; Carandina, G.; Salani, M.; Antonioli, P.; Gregorio, P. Multidimensional Evaluation of Performance with Experimental Application of Balanced Scorecard: A Two Year Experience. Cost Eff. Resour. Alloc. 2011, 9, 7. [Google Scholar] [CrossRef] [Green Version]
- Ajami, S.; Ebadsichani, A.; Tofighi, S.; Tavakoli, N. Medical Records Department and Balanced Scorecard Approach. J. Educ. Health Promot. 2013, 2, 7. [Google Scholar] [CrossRef] [Green Version]
- Gao, H.; Chen, H.; Feng, J.; Qin, X.; Wang, X.; Liang, S.; Zhao, J.; Feng, Q. Balanced Scorecard-Based Performance Evaluation of Chinese County Hospitals in Underdeveloped Areas. J. Int. Med. Res. 2018, 46, 1947–1962. [Google Scholar] [CrossRef] [Green Version]
- Catuogno, S.; Arena, C.; Saggese, S.; Sarto, F. Balanced Performance Measurement in Research Hospitals: The Participative Case Study of a Haematology Department. BMC Health Serv. Res. 2017, 17, 1–11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rabbani, F.; Pradhan, N.A.; Zaidi, S.; Azam, S.I.; Yousuf, F. Service Quality in Contracted Facilities. Int. J. Health Care Qual. Assur. 2015, 28, 520–531. [Google Scholar] [CrossRef] [PubMed]
- Rowe, J.S.; Natiq, K.; Alonge, O.; Gupta, S.; Agarwal, A.; Peters, D.H. Evaluating the Use of Locally-Based Health Facility Assessments in Afghanistan: A Pilot Study of a Novel Research Method. Confl. Health 2014, 8, 24. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Widyasari, N.L.G.; Adi, N.R. Balanced Scorecard Implementation in the Government Hospital. Russ. J. Agric. Socio-Econ. Sci. 2019, 93, 285–291. [Google Scholar] [CrossRef]
- Teklehaimanot, H.D.; Teklehaimanot, A.; Tedella, A.A.; Abdella, M. Use of Balanced Scorecard Methodology for Performance Measurement of the Health Extension Program in Ethiopia. Am. J. Trop. Med. Hyg. 2016, 94, 1157–1169. [Google Scholar] [CrossRef] [Green Version]
- Edward, A.; Kumar, B.; Kakar, F.; Salehi, A.S.; Burnham, G.; Peters, D.H. Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years’ Evaluation in Afghanistan. PLoS Med. 2011, 8, e1001066. [Google Scholar] [CrossRef] [Green Version]
- Hansen, P.M.; Peters, D.H.; Niayesh, H.; Singh, L.P.; Dwivedi, V.; Burnham, G. Measuring and Managing Progress in the Establishment of Basic Health Services: The Afghanistan Health Sector Balanced Scorecard. Int. J. Health Plan. Manag. 2008, 23, 107–117. [Google Scholar] [CrossRef]
- Peters, D.H.; Noor, A.A.; Singh, L.P.; Kakar, F.K.; Hansen, P.M.; Burnhama, G. A Balanced Scorecard for Health Services in Afghanistan. Bull. World Health Organ. 2007, 85, 146–151. [Google Scholar] [CrossRef]
- Josey, C.; Kim, I. Implementation of the Balanced Scorecard at Barberton Citizens Hospital. J. Corp. Account. Financ. 2008, 19, 57–63. [Google Scholar] [CrossRef]
Characteristics | n | % | Characteristics | n | % |
---|---|---|---|---|---|
Age | Years of experience | ||||
20–29 years | 198 | 43.6 | 0–2 years | 149 | 32.8 |
30–39 years | 163 | 35.9 | 3–5 years | 107 | 23.6 |
40–49 years | 59 | 13 | 6–9 years | 79 | 17.4 |
50–59 years | 26 | 5.7 | 10–13 years | 45 | 9.9 |
60 years or above | 8 | 1.8 | More than 14 years | 74 | 16.3 |
Gender | Income | ||||
Female | 232 | 51.1 | 2000–3000 NIS | 108 | 23.8 |
Male | 222 | 48.9 | 3001–4000 NIS | 129 | 28.4 |
Department | 4001–5000 NIS | 101 | 22.2 | ||
Mixed | 18 | 4.0 | 5001–6000 NIS | 50 | 11.0 |
Pediatric | 73 | 16.1 | Higher than 6000 NIS | 66 | 14.5 |
Internal medicine | 81 | 17.8 | Profession | ||
Surgery | 98 | 21.6 | Doctor | 156 | 34.4 |
Emergency | 91 | 20.0 | Nurse | 298 | 65.6 |
Gynecology | 93 | 20.5 |
Number of HCWs (Total = 454) | % | Number of Hospitals (Total = 14) | % | |
---|---|---|---|---|
Administrative style | ||||
NGO | 170 | 37 | 5 | 35.71 |
Public | 145 | 32 | 5 | 35.71 |
Private | 111 | 24 | 3 | 21.43 |
UNRWA | 28 | 6 | 1 | 7.14 |
City | ||||
Hebron | 87 | 19.16 | 3 | 21.43 |
Jerusalem | 40 | 8.81 | 1 | 7.14 |
Nablus | 166 | 36.56 | 5 | 35.71 |
Qalqilya | 28 | 6.17 | 1 | 7.14 |
Ramallah | 92 | 20.26 | 3 | 21.43 |
Tulkarm | 41 | 9.03 | 1 | 7.14 |
Area | ||||
North | 235 | 51.76 | 7 | 50.00 |
Middle | 132 | 29.07 | 4 | 28.57 |
South | 87 | 19.16 | 3 | 21.43 |
Accreditation status | ||||
Yes | 97 | 21.37 | 3 | 21.43 |
No | 357 | 78.63 | 11 | 78.57 |
Size | ||||
Small (No. of beds < 80) | 133 | 29.30 | 5 | 35.71 |
Medium (No. of beds 80–160) | 188 | 41.41 | 5 | 35.71 |
Large (No. of beds > 160) | 133 | 29.30 | 4 | 28.57 |
Component | Item | Item Code | Factor | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | |||
TECH | Hospital information systems and technology make access to patients’ records easier, faster, and more accurate. | EST4 | 0.923 | ||||||||||||||
The hospital information system and technology make generating reports easier, faster, and more accurate. | EST5 | 0.863 | |||||||||||||||
The hospital information system and technology make my work efficient and productive. | EST6 | 0.767 | |||||||||||||||
I believe that the hospital information system interface is user-friendly. | EST3 | 0.736 | |||||||||||||||
HCWDEV | The hospital provides me with education on medication updates related to my specialty. | ESD3 | 0.972 | ||||||||||||||
The hospital provides me with access to the latest medical books and journals. | ESD4 | 0.811 | |||||||||||||||
The hospital provides me with educational updates regarding the diseases in my specialty. | ESD1 | 0.721 | |||||||||||||||
This hospital provides me with an access to the newest books, databases, and scientific papers. | ESD5 | 0.705 | |||||||||||||||
MGMTEVAL | I believe that my superiors have the required competencies for their positions. | ESM1 | 0.951 | ||||||||||||||
My superiors are making the right decisions in work which support the hospital strategy. | ESM2 | 0.804 | |||||||||||||||
The management in this hospital asks for staff feedback, perceptions, and care for their satisfaction. | ESM3 | 0.515 | |||||||||||||||
WTLB | The quantity of work assigned to me is reasonable with the time given. | ESTI4 | 0.708 | ||||||||||||||
I have sufficient time to rest and eat during my working day. | ESTI1 | 0.668 | |||||||||||||||
I can make a work–life balance and good time management. | ESTI3 | 0.660 | |||||||||||||||
I can spend sufficient time with each patient. | ESTI2 | 0.596 | |||||||||||||||
LOY | My overall satisfaction is high. | ESL4 | 0.627 | ||||||||||||||
I want to keep working in this hospital for several years. | ESL3 | 0.599 | |||||||||||||||
MSQUAL | The hospital medications and disposables are of high quality. | ESQ2 | 0.939 | ||||||||||||||
The hospital equipment helps me in offering high-quality medical services for patients. | ESQ1 | 0.685 | |||||||||||||||
Quality is a top priority at this hospital. | ESQ3 | 0.587 | |||||||||||||||
FIN | I receive financial incentives based on my performance. | ESF2 | 0.836 | ||||||||||||||
I feel that my salary suits my responsibilities and competencies. | ESF1 | 0.529 | |||||||||||||||
ENG | My manager engages me in the planning and decision-making process. | ESMO5 | 0.670 | ||||||||||||||
My manager understands and adequately supports me when I face an urgent, complex situation. | ESMO4 | 0.604 | |||||||||||||||
I am given enough authority and power to make decisions in my position. | ESMO6 | 0.536 | |||||||||||||||
REPUT | I am proud to work with this hospital. | ESR4 | 0.653 | ||||||||||||||
I believe that patients respect health care workers at this hospital and trust them. | ESR2 | 0.637 | |||||||||||||||
I believe that this hospital has a better reputation than other hospitals in Palestinian. | ESR3 | 0.533 | |||||||||||||||
MGMT COMM | Communication with management is frequent, and they keep me updated with sufficient information to do my job. | ESC4 | 0.839 | ||||||||||||||
I trust what my direct manager tells me or promises me. | ESC5 | 0.651 | |||||||||||||||
ACC | It is easy to access the hospital when a case is urgent. | ESA2 | 0.937 | ||||||||||||||
Hospital location is close to where I live. | ESA1 | 0.672 | |||||||||||||||
ITRODP | New employees are well introduced to the job description, and the specifications are clear in the job contract. | ESEM1 | 0.615 | ||||||||||||||
SAF | Safety standards are implemented and assured (masks, gloves, sanitizers, etc.). | ESS2 | 0.663 | ||||||||||||||
NBR | When errors are reported a blame free policy is taken by managers. | ESS1 | 0.503 | ||||||||||||||
Percentage of variance (%) Total variance = 66.72% | 21.08 | 7.25 | 4.80 | 4.46 | 3.87 | 3.55 | 2.86 | 2.71 | 2.62 | 2.49 | 2.39 | 2.34 | 2.16 | 2.07 | 2.07 | ||
Eigenvalues | 10.75 | 3.70 | 2.45 | 2.27 | 1.97 | 1.81 | 1.46 | 1.38 | 1.33 | 1.27 | 1.22 | 1.19 | 1.10 | 1.06 | 1.05 |
EFA [77,80,83] | CFA [84] | ||
---|---|---|---|
Criteria for Good Fit | Measurements | Criteria for Good Fit | Measurements |
- KMO: 0.6: low adequacy 0.7: medium adequacy 0.8: high adequacy 0.9: very high adequacy - Bartlett’s test p-value < 0.05 - Inclusion/exclusion criteria for the components: 1 - Eigenvalues ≥ 1 2 - Visual assessment of Cattell’s scree plot. -Inclusion/exclusion criteria for the items: 3 - The factor loading ≥ 0.50. 4 - Factor loadings on the assigned construct ≥ all cross-loading of other constructs. | - KMO = 0.832 (Chi square = 5442.68, degrees of freedom = 1275) - Bartlett’s test p-value < 0.001 - 15 components which have Eigenvalues above 1 - Cumulative variance = 66.72% - Cattell’s scree plot: keep 10 components | - χ2/df < 5 and closer to zero - The p-value > 0.05 - GFI - CFI - TLI GFI, CFI, and TLI close to 0.95 - RMSEA < 0.06 - SRMR ≤ 0.08 | χ2/df = 1.33 p-value < 0.001 GFI = 0.875 CFI = 0.958 TLI = 0.948 RMSEA = 0.041 SRMR = 0.0557 - 9 constructs |
Factor | CR | IIC | CITC | AVE | MGMTEVAL | ENG | FIN | QUALDEV | TECH | WTLB | LOY | MTR |
---|---|---|---|---|---|---|---|---|---|---|---|---|
MGMTEVAL | 0.769 | 0.373–0.701 | 0.550–0.653 | 0.455 | 0.675 | |||||||
ENG | 0.727 | 0.398–0.467 | 0.503–0.554 | 0.472 | 0.503 ** | 0.687 | ||||||
FIN | 0.694 | 0.493 | 0.493 | 0.533 | 0.288** | 0.216** | 0.730 | |||||
QUALDEV | 0.829 | 0.334–0.581 | 0.534–0.600 | 0.494 | 0.492** | 0.364** | 0.392** | 0.702 | ||||
TECH | 0.878 | 0.483–0.703 | 0.620–0.729 | 0.645 | 0.278** | 0.253** | 0.055 | 0.296** | 0.803 | |||
WTLB | 0.760 | 0.345–0.484 | 0.483–0.610 | 0.448 | 0.308** | 0.207** | 0.429** | 0.446** | 0.055 | 0.670 | ||
LOY | 0.761 | 0.364–0.561 | 0.466–0.645 | 0.449 | 0.407** | 0.310** | 0.341** | 0.476** | 0.209** | 0.455** | 0.670 | |
MTR | - | - | - | - | 0.378** | 0.397** | 0.176** | 0.274** | 0.117* | 0.171** | 0.312** | - |
PTR | - | - | - | - | 0.358** | 0.208** | 0.319** | 0.460** | 0.176** | 0.378** | 0.393** | 0.190** |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Amer, F.; Hammoud, S.; Khatatbeh, H.; Alfatafta, H.; Alkaiyat, A.; Nour, A.I.; Endrei, D.; Boncz, I. How to Engage Health Care Workers in the Evaluation of Hospitals: Development and Validation of BSC-HCW1—A Cross-Sectional Study. Int. J. Environ. Res. Public Health 2022, 19, 9096. https://doi.org/10.3390/ijerph19159096
Amer F, Hammoud S, Khatatbeh H, Alfatafta H, Alkaiyat A, Nour AI, Endrei D, Boncz I. How to Engage Health Care Workers in the Evaluation of Hospitals: Development and Validation of BSC-HCW1—A Cross-Sectional Study. International Journal of Environmental Research and Public Health. 2022; 19(15):9096. https://doi.org/10.3390/ijerph19159096
Chicago/Turabian StyleAmer, Faten, Sahar Hammoud, Haitham Khatatbeh, Huda Alfatafta, Abdulsalam Alkaiyat, Abdulnaser Ibrahim Nour, Dóra Endrei, and Imre Boncz. 2022. "How to Engage Health Care Workers in the Evaluation of Hospitals: Development and Validation of BSC-HCW1—A Cross-Sectional Study" International Journal of Environmental Research and Public Health 19, no. 15: 9096. https://doi.org/10.3390/ijerph19159096