Application of Machine Learning Methods in Nursing Home Research
Abstract
:1. Background
2. Methods
2.1. Study Design
2.2. Ethical Considerations
2.3. Measurements/Instruments
2.3.1. Prediction Variables
2.3.2. Outcomes Variable
2.4. Data Collection/Procedure
2.5. Data Analysis
2.5.1. Statistical Data Analysis
2.5.2. Machine Learning
2.6. Variable Selection
2.7. Data Preprocessing
2.8. Predicting Modeling
2.9. Evaluation of Prediction Models
3. Results
3.1. Characteristics of the Organizations (n = 60)
3.2. Predicting Modeling
3.3. Predictive Performance
4. Discussion
5. Conclusions
Recommendations for Future Research
Author Contributions
Funding
Conflicts of Interest
Availability of Data and Materials
Abbreviations
CNA | Certified nursing assistant |
DON | Director of nursing |
HPRD | Hours per resident day |
ML | Machine learning |
NH | Nursing Home |
RF | Random forest |
RN | Registered nurse |
SVM | Support vector machine |
References
- Becker, C.; Rapp, K. Fall Prevention in Nursing Homes. Clin. Geriatr. Med. 2010, 26, 693–704. [Google Scholar] [CrossRef] [PubMed]
- Gray-Miceli, D.; De Cordova, P.B.; Crane, G.L.; Quigley, P.; Ratcliffe, S.J. Nursing Home Registered Nurses’ and Licensed Practical Nurses’ Knowledge of Causes of Falls. J. Nurs. Care Qual. 2015, 31, 1–160. [Google Scholar] [CrossRef] [Green Version]
- Gray-Miceli, D.; Ratcliffe, S.J.; Thomasson, A. Elderly fallers at greatest risk for head injury. Gerontologist 2013, 53, 489. [Google Scholar] [CrossRef] [Green Version]
- Vandenberg, A.E.; Van Beijnum, B.-J.; Overdevest, V.G.; Capezuti, E.; Johnson, T.M. US and Dutch nurse experiences with fall prevention technology within nursing home environment and workflow: A qualitative study. Geriatr. Nurs. 2016, 38, 276–282. [Google Scholar] [CrossRef] [PubMed]
- Chaccour, K.; Darazi, R.; El Hassani, A.H.; Andres, E. From Fall Detection to Fall Prevention: A Generic Classification of Fall-Related Systems. IEEE Sens. J. 2016, 17, 812–822. [Google Scholar] [CrossRef]
- Seijo-Martinez, M.; Cancela, J.M.; Ayán, C.; Varela, S.; Vila, H. Influence of cognitive impairment on fall risk among elderly nursing home residents. Int. Psychogeriatr. 2016, 28, 1975–1987. [Google Scholar] [CrossRef]
- Van Doorn, C.; Gruber-Baldini, A.L.; Zimmerman, S.; Hebel, J.R.; Port, C.L.; Baumgarten, M.; Quinn, C.C.; Taler, G.; May, C.; Magaziner, J.; et al. Dementia as a Risk Factor for Falls and Fall Injuries Among Nursing Home Residents. J. Am. Geriatr. Soc. 2003, 51, 1213–1218. [Google Scholar] [CrossRef]
- Whitney, J.; Close, J.C.; Jackson, S.H.; Lord, S.R. Understanding Risk of Falls in People With Cognitive Impairment Living in Residential Care. J. Am. Med Dir. Assoc. 2012, 13, 535–540. [Google Scholar] [CrossRef]
- Hasegawa, J.; Kuzuya, M.; Iguchi, A. Urinary incontinence and behavioral symptoms are independent risk factors for recurrent and injurious falls, respectively, among residents in long-term care facilities. Arch. Gerontol. Geriatr. 2010, 50, 77–81. [Google Scholar] [CrossRef]
- Shin, J.H. Nursing Staff Characteristics on Resident Outcomes in Nursing Homes. J. Nurs. Res. 2019, 27, e7–e9. [Google Scholar] [CrossRef]
- Kron, M.; Loy, S.; Sturm, E.; Nikolaus, T.; Becker, C. Risk indicators for falls in institutionalized frail elderly. Am. J. Epidemiol. 2003, 158, 645–653. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zarowitz, B.J.; O’Shea, T.; Tangalos, E.; Berner, T.; Ouslander, J.G.; Allen, C. Clinical Burden and Nonpharmacologic Management of Nursing Facility Residents with Overactive Bladder and/or Urinary Incontinence. Consult. Pharm. 2015, 30, 533–542. [Google Scholar] [CrossRef] [PubMed]
- Bor, A.; Matuz, M.; Csatordai, M.; Szalai, G.; Bálint, A.; Benkő, R.; Soós, G.; Doró, P. Medication use and risk of falls among nursing home residents: A retrospective cohort study. Int. J. Clin. Pharm. 2017, 39, 408–415. [Google Scholar] [CrossRef] [PubMed]
- Paliwal, Y.; Slattum, P.W.; Ratliff, S.M. Chronic Health Conditions as a Risk Factor for Falls among the Community-Dwelling US Older Adults: A Zero-Inflated Regression Modeling Approach. BioMed Res. Int. 2017, 2017, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Wang, G.; Lam, K.-M.; Deng, Z.; Choi, K.-S. Prediction of mortality after radical cystectomy for bladder cancer by machine learning techniques. Comput. Biol. Med. 2015, 63, 124–132. [Google Scholar] [CrossRef] [PubMed]
- Quinlan, J.R. C4.5: Programs for Machine Learning; Morgan Kaufmann: San Francisco, CA, USA, 1993. [Google Scholar]
- McCarthy, J.F.; Marx, K.A.; Hoffman, P.E.; Gee, A.G.; O’Neil, P.; Ujwal, M.L.; Hotchkiss, J. Applications of Machine Learning and High-Dimensional Visualization in Cancer Detection, Diagnosis, and Management. Ann. N. Y. Acad. Sci. 2004, 1020, 239–262. [Google Scholar] [CrossRef]
- Stylianou, N.; Akbarov, A.; Kontopantelis, E.; Buchan, I.; Dunn, K.W. Mortality risk prediction in burn injury: Comparison of logistic regression with machine learning approaches. Burns 2015, 41, 925–934. [Google Scholar] [CrossRef]
- Lee, H.Y.; Blegen, M.A.; Harrington, C. The effects of RN staffing hours on nursing home quality: A two-stage model. Int. J. Nurs. Stud. 2014, 51, 409–417. [Google Scholar] [CrossRef]
- Veyron, J.-H.; Friocourt, P.; Jeanjean, O.; Luquel, L.; Bonifas, N.; Denis, F.; Belmin, J. Home care aides’ observations and machine learning algorithms for the prediction of visits to emergency departments by older community-dwelling individuals receiving home care assistance: A proof of concept study. PLoS ONE 2019, 14, e0220002. [Google Scholar] [CrossRef]
- Fu, M.R.; Wang, Y.; Li, C.; Qiu, Z.; Axelrod, D.; Guth, A.A.; Scagliola, J.; Conley, Y.; Aouizerat, B.E.; Qiu, J.M.; et al. Machine learning for detection of lymphedema among breast cancer survivors. mHealth 2018, 4, 17. [Google Scholar] [CrossRef]
- Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed.; Springer: New York, NY, USA, 2009. [Google Scholar] [CrossRef]
- Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
- Furey, T.S.; Cristianini, N.; Duffy, N.; Bednarski, D.W.; Schummer, M.; Haussler, D. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics 2000, 16, 906–914. [Google Scholar] [CrossRef] [PubMed]
- Center for Medicare and Medicaid. Staffing Data Submission PBJ. 2018. Available online: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/Staffing-Data-Submission-PBJ.html (accessed on 20 April 2019).
- Boyle, D.A. Countering compassion fatigue: A requisite nursing agenda. Online J. Issues Nurs. 2011, 16, 2. [Google Scholar] [CrossRef] [PubMed]
- Ho, T.K. Random Decision Forests. In Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, QC, Canada, 14–16 August 1995; pp. 278–282. [Google Scholar] [CrossRef]
- Hosmer, D.W., Jr.; Lemeshow, S.; Sturdivant, R.X. Applied Logistic Regression; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar] [CrossRef]
- Suykens, J.; Vandewalle, J. Least Squares Support Vector Machine Classifiers. Neural Process. Lett. 1999, 9, 293–300. [Google Scholar] [CrossRef]
- Lee, S.-K.; Son, Y.-J.; Kim, J.; Kim, H.-G.; Lee, J.-I.; Kang, B.-Y.; Cho, H.; Lee, S. Prediction Model for Health-Related Quality of Life of Elderly with Chronic Diseases using Machine Learning Techniques. Health Inform. Res. 2014, 20, 125–134. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nindrea, R.D.; Aryandono, T.; Lazuardi, L.; Dwiprahasto, I. Diagnostic Accuracy of Different Machine Learning Algorithms for Breast Cancer Risk Calculation: A Meta-Analysis. Asian Pac. J. Cancer Prev. 2018, 19, 1747–1752. [Google Scholar]
- Steyerberg, E.W.; Van Der Ploeg, T.; Van Calster, B. Risk prediction with machine learning and regression methods. Biom. J. 2014, 56, 601–606. [Google Scholar] [CrossRef]
- Eriksen, L.R.; Turley, J.P.; Denton, D.; Manning, S. Data mining: A strategy for knowledge development and structure in nursing practice. Stud. Health Technol. Inform. 1997, 46, 383–388. [Google Scholar] [CrossRef]
- Milos, V.; Bondesson, Å.; Magnusson, M.; Jakobsson, U.; Westerlund, T.; Midlöv, P. Fall risk-increasing drugs and falls: A cross-sectional study among elderly patients in primary care. BMC Geriatr. 2014, 14, 40. [Google Scholar] [CrossRef]
- Kelkar, A.; Datta, R.; Elkins, J. What Factors Predict Falls in Older Adults Living in Nursing Homes: A Pilot Study. J. Funct. Morphol. Kinesiol. 2018, 4, 1000. [Google Scholar] [CrossRef]
- Hanlon, J.T.; Zhao, X.; Naples, J.G.; Aspinall, S.L.; Perera, S.; Nace, D.A.; Castle, N.G.; Greenspan, S.L.; Thorpe, C.T. Central Nervous System Medication Burden and Serious Falls in Older Nursing Home Residents. J. Am. Geriatr. Soc. 2017, 65, 1183–1189. [Google Scholar] [CrossRef] [PubMed]
- Berry, S.D.; Placide, S.G.; Mostofsky, E.; Zhang, Y.; Lipsitz, L.A.; Mittleman, M.A.; Kiel, U.P. Antipsychotic and Benzodiazepine Drug Changes Affect Acute Falls Risk Differently in the Nursing Home. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 2015, 71, 273–278. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Delbaere, K.; Close, J.C.; Menz, H.B.; Cumming, R.G.; Cameron, I.D.; Sambrook, P.N.; March, L.M.; Lord, S.R. Development and validation of fall risk screening tools for use in residential aged care facilities. Med. J. Aust. 2008, 189, 193–196. [Google Scholar] [CrossRef] [PubMed]
- Cameron, I.D.; Kurrle, S.; Crotty, M. Hip fracture rehabilitation for people living in nursing homes: Randomised trial. Ann. Phys. Rehabil. Med. 2018, 61, e339. [Google Scholar] [CrossRef]
- Morris, V.; Wagg, A. Lower urinary tract symptoms, incontinence and falls in elderly people: Time for an intervention study. Int. J. Clin. Pract. 2007, 61, 320–323. [Google Scholar] [CrossRef]
- Leland, N.; Gozalo, P.; Teno, J.; Mor, V. Falls in newly admitted nursing home residents: A national study. J. Am. Geriatr. Soc. 2012, 60, 939–945. [Google Scholar] [CrossRef] [Green Version]
- Spector, W.D.; Shaffer, T.; Potter, D.E.B.; Limcangco, M.R.; Correa-De-Araujo, R. Risk Factors Associated with the Occurrence of Fractures in U.S. Nursing Homes: Resident and Facility Characteristics and Prescription Medications. J. Am. Geriatr. Soc. 2007, 55, 327–333. [Google Scholar] [CrossRef]
- Thomas, K.S.; Hyer, K.; Castle, N.G.; Branch, L.G.; Andel, R.; Weech-Maldonado, R. Patient Safety Culture and the Association with Safe Resident Care in Nursing Homes. Gerontologist 2012, 52, 802–811. [Google Scholar] [CrossRef] [Green Version]
- Shin, J.H. Why Do We Require Registered Nurses in Nursing Homes? Using Longitudinal Hierarchical Linear Modeling. J. Nurs. Sch. 2018, 50, 705–713. [Google Scholar] [CrossRef] [Green Version]
- Shin, J.H.; Hyun, T.K. Nurse Staffing and Quality of Care of Nursing Home Residents in Korea. J. Nurs. Sch. 2015, 47, 555–564. [Google Scholar] [CrossRef]
- Taylor, J.A.; Parmelee, P.; Brown, H.; Strothersiii, H., III; Capezuti, E.; Ouslander, J.G. A Model Quality Improvement Program for the Management of Falls in Nursing Homes. J. Am. Med. Dir. Assoc. 2007, 8, S26–S36. [Google Scholar] [CrossRef] [PubMed]
- Belle, V.V.; Pelckmans, K.; Suykens, J.A.; Huffel, S.V. Learning transformation models for ranking and survival analysis. J. Mach. Learn. Res. 2011, 12, 819–862. [Google Scholar]
Model | Hours Per Resident Day of Secretary General | Proportion of Residents with Psychiatric Medication | Proportion of Residents with Urinary Incontinence | Hours Per Resident Day of Care Worker | Current Number of Resident in a Nursing Home | CNA_HPRD | Proportion of Residents with Aggressive Behavior | Proportion of Residents with Cognitive Decline | Turnover of Care Worker | Maximum Capacity of Residents | Total |
---|---|---|---|---|---|---|---|---|---|---|---|
Logistic regression | O | O | O | O | 4 | ||||||
Random forest | O | O | O | O | O | O | 6 | ||||
SVM linear | O | O | O | O | O | 5 | |||||
SVM polynomial | O | O | O | O | 4 | ||||||
SVM radial | O | O | O | O | 4 | ||||||
SVM sigmoid | O | O | O | O | O | 5 | |||||
- | 5 | 4 | 4 | 3 | 3 | 3 | 2 | 2 | 1 | 1 | - |
No. | Variable | Importance Score |
---|---|---|
1 | Hours per resident day of the secretary general | 1.8334 |
2 | Proportion of residents with psychiatric medications | 1.5208 |
3 | Proportion of residents with urinary incontinence | 1.2823 |
4 | Hours per resident day of care workers | 1.0291 |
5 | Current number of residents in a nursing home | 0.8903 |
6 | CNA_HPRD | 0.8418 |
7 | Proportion of residents with aggressive behavior | 0.7985 |
8 | Proportion of residents with cognitive decline | 0.7785 |
9 | Turnover of care worker | 0.7754 |
10 | Maximum capacity of residents | 0.6628 |
Variable | Frequency | % | M | SD | Min | Max |
---|---|---|---|---|---|---|
Capacity | 75.13 | 55.83 | 7 | 296 | ||
Number of current residents | 66.73 | 48.51 | 7 | 295 | ||
Location of organizations | ||||||
Metropolitan (over million) | 29 | 48.3 | ||||
Urban location (over round half million) | 13 | 21.7 | ||||
Local small city (5–50 thousand) | 13 | 21.7 | ||||
Rural area (less than 50 thousand) | 5 | 8.3 | ||||
Ownership | ||||||
Profit | 10 | 16.7 | ||||
Not for profit | 49 | 81.7 | ||||
Religious affiliation | ||||||
No religion | 39 | 65.0 | ||||
Christianity | 11 | 18.5 | ||||
Catholic | 3 | 5.5 | ||||
Buddhism | 3 | 5.5 | ||||
Others | 3 | 5.5 | ||||
Affiliated hospitals | ||||||
No | 5 | 8.3 | ||||
Yes | 54 | 90.0 | ||||
Average age of residents | 83.60 | 2.404 | 80 | 89 | ||
Determined Long-term care insurance grade by the Korean National Health Insurance Corporation (%) | ||||||
1st a | 12.00 | 8.82 | 0 | 41 | ||
2nd b | 22.40 | 11.46 | 0 | 60 | ||
3rd c | 35.95 | 14.39 | 0 | 90 | ||
4th d | 22.41 | 16.95 | 0 | 90 | ||
5th e | 1.18 | 3.00 | 0 | 15 | ||
Unrated | 5.98 | 19.48 | 0 | 100 | ||
Gender (%) | ||||||
Male | 20.77 | 11.35 | 0 | 64 | ||
Female | 78.95 | 11.30 | 36 | 100 | ||
Long-term care facility grade (%) | ||||||
Grade A (Superior) | 23 | 38.3 | ||||
Grade B (Above average) | 8 | 13.3 | ||||
Grade C (Average) | 6 | 10.0 | ||||
Grade D (Below average) | 7 | 11.7 | ||||
Grade E (Poor) | 0 | 0.0 | ||||
Ungraded | 16 | 26.7 | ||||
Number of registered nurses | 1.48 | 2.89 | 0 | 17 | ||
Number of certified nurse aides | 1.95 | 1.44 | 0 | 6 | ||
Number of care worker | 26.58 | 20.52 | 1 | 132 | ||
Hours per resident day | ||||||
Registered nurses | 0.19 | 0.24 | 0.00 | 1.03 | ||
Certified nurse aides | 0.36 | 0.26 | 0.00 | 1.42 | ||
Care worker | 3.82 | 1.63 | 1.32 | 9.76 | ||
Director | 0.26 | 0.23 | 0.03 | 1.42 | ||
Secretary general | 0.12 | 0.12 | 0.00 | 0.51 | ||
Social worker | 0.39 | 0.27 | 0.00 | 1.42 | ||
Dietician | 0.09 | 0.09 | 0.00 | 0.36 | ||
Administrative staff | 0.14 | 0.22 | 0.00 | 1.36 | ||
Turnover (%) | ||||||
Registered nurses | 9.49 | 21.61 | 0.00 | 100.00 | ||
Certified nurse aides | 12.54 | 22.49 | 0.00 | 66.66 | ||
Care worker | 19.06 | 18.50 | 0.00 | 86.17 | ||
Director | 4.17 | 19.07 | 0.00 | 20.15 | ||
Secretary general | 12.23 | 33.12 | 0.00 | 83.17 | ||
Social worker | 17.99 | 29.49 | 0.00 | 81.22 | ||
Dietician | 29.83 | 42.65 | 0.00 | 91.15 | ||
Cook | 38.46 | 36.31 | 0.00 | 95.27 | ||
Administrative staff | 21.39 | 35.17 | 0.00 | 88.19 |
Model | Accuracy | Sensitivity | Specificity | PPV | NPV |
---|---|---|---|---|---|
Logistic regression | 0.867 | 1.000 | 0.467 | 0.849 | 1.000 |
Random forest | 0.883 | 0.978 | 0.600 | 0.880 | 0.900 |
SVM linear | 0.867 | 0.978 | 0.533 | 0.863 | 0.889 |
SVM polynomial | 0.867 | 1.000 | 0.467 | 0.849 | 1.000 |
SVM radial | 0.850 | 0.978 | 0.467 | 0.846 | 0.875 |
SVM sigmoid | 0.850 | 0.933 | 0.600 | 0.875 | 0.750 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, S.-K.; Ahn, J.; Shin, J.H.; Lee, J.Y. Application of Machine Learning Methods in Nursing Home Research. Int. J. Environ. Res. Public Health 2020, 17, 6234. https://doi.org/10.3390/ijerph17176234
Lee S-K, Ahn J, Shin JH, Lee JY. Application of Machine Learning Methods in Nursing Home Research. International Journal of Environmental Research and Public Health. 2020; 17(17):6234. https://doi.org/10.3390/ijerph17176234
Chicago/Turabian StyleLee, Soo-Kyoung, Jinhyun Ahn, Juh Hyun Shin, and Ji Yeon Lee. 2020. "Application of Machine Learning Methods in Nursing Home Research" International Journal of Environmental Research and Public Health 17, no. 17: 6234. https://doi.org/10.3390/ijerph17176234