Evaluation of Complexity Measurement Tools for Correlations with Health-Related Outcomes, Health Care Costs and Impacts on Healthcare Providers: A Scoping Review
by
, , ,
Hiromitsu Kaneko
1,*,
Akiko Hanamoto
2,
Sachiko Yamamoto-Kataoka
3,
Yuki Kataoka
4,5,6,7
,
Takuya Aoki
6,8,
Kokoro Shirai
9 and
Hiroyasu Iso
9,10 1
Faculty of Medicine, Osaka University, Suita, Osaka 565-0871, Japan
2
Independent Researcher, Kita-ku, Kyoto 603-8233, Japan
3
Department of Health Informatics, Kyoto University Graduate School of Medicine/School of Public Health, Yoshida Konoe-cho, Sakyo-ku, Kyoto 606-8501, Japan
4
Department of Internal Medicine, Kyoto Min-Iren Asukai Hospital, Tanaka Asukai-cho 89, Kyoto 606-8226, Japan
5
Scientific Research Works Peer Support Group (SRWS-PSG), Osaka 541-0043, Japan
6
Section of Clinical Epidemiology, Department of Community Medicine, Kyoto University Graduate School of Medicine, Shogoin Kawara-cho 54, Kyoto 606-8507, Japan
7
Department of Healthcare Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Yoshida Konoe-cho, Kyoto 606-8501, Japan
8
Division of Clinical Epidemiology, Research Center for Medical Sciences, The Jikei University School of Medicine, 3-25-8 Nishishimbashi, Minato-ku, Tokyo 105-8461, Japan
9
Department of Social Medicine, Osaka University Graduate School of Medicine, Osaka 565-0871, Japan
10
Institute for Global Health Policy Research, Bureau of International Health Cooperation, National Center for Global Health and Medicine, Tokyo 162-8655, Japan
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(23), 16113; https://doi.org/10.3390/ijerph192316113
Submission received: 19 September 2022
/
Revised: 24 November 2022
/
Accepted: 24 November 2022
/
Published: 1 December 2022
Abstract
:Various tools to measure patient complexity have been developed. Primary care physicians often deal with patient complexity. However, their usefulness in primary care settings is unclear. This study explored complexity measurement tools in general adult and patient populations to investigate the correlations between patient complexity and outcomes, including health-related patient outcomes, healthcare costs, and impacts on healthcare providers. We used a five-stage scoping review framework, searching MEDLINE and CINAHL, including reference lists of identified studies. A total of 21 patient complexity management tools were found. Twenty-five studies examined the correlation between patient complexity and health-related patient outcomes, two examined healthcare costs, and one assessed impacts on healthcare providers. No studies have considered sharing information or action plans with multidisciplinary teams while measuring outcomes for complex patients. Of the tools, eleven used face-to-face interviews, seven extracted data from medical records, and three used self-assessments. The evidence of correlations between patient complexity and outcomes was insufficient for clinical implementation. Self-assessment tools might be convenient for conducting further studies. A multidisciplinary approach is essential to develop effective intervention protocols. Further research is required to determine these correlations in primary care settings.
1. Introduction
The concept of health includes mental and social well-being as well as physical well-being [1]. The conventional biomedical model requires that disease be treated as an entity independent of social behaviors; however, it is currently widely recognized that disease is a multifaceted concept that is not fully captured by the number or type of medical conditions or by previous healthcare costs [2,3]. To overcome the limitations of the conventional biomedical model, the concept of “patient complexity” has been proposed.
To date, there is no established definition of patient complexity. Instead, several explanations with different scopes have been suggested [4,5,6,7,8,9,10]. According to Nicolaus [11], patient complexity is a dynamic state in which biomedical, mental health, socioeconomic factors, and patient preferences are affected by each other. To deal with the patient complexity, conceptual models and complexity measurement tools are recognized. Some studies have developed conceptual models that describe complexity as the relationships among domains, such as diagnostic procedures, treatment strategies, interventions, multiple healthcare providers, or patients’ behaviors [4,5,9,10]. Conceptual models can suggest holistic approaches to address broader issues but sometimes are impractical for clinical implementation.
In contrast, other studies have developed tools to make complexity more easily quantifiable [6,7,8]. For instance, INTERMED evaluates biological, psychological, social, and healthcare domains through interviews in which scores are assigned to 20 items on a scale of 0 (no symptoms) to 3 (severe symptoms). Tools can only be applied to specific situations for which they are designed, such as for adult inpatients, but are more applicable to clinical cases. Various tools for measuring patient complexity have been developed for specific purposes and settings.
In this scoping review, we focused on both observational and interventional studies that examined outcomes related to complex patients stratified by complexity measurement tools. Contrary to our review article, previous studies reviewed these complexity measurement tools focusing on particular diseases [12]. While there exists a versatile range of tools, previous reviews did not clearly elucidate what kinds of measures could be used with the general adult populations or with patients but not in the context of specific diseases or settings, such as the ICU (intensive care unit) and home healthcare. Moreover, previous reviews did not investigate how complexity measurement tools are related to outcomes of general adult populations or patients, and the performance of healthcare providers/systems. Some studies described the characteristics of adult populations that are frequent users of healthcare providers/systems [13]; however, they did not examine the correlation with patient complexities.
Therefore, this scoping review clarified the general status of tools to measure patient complexity for the general adult population or patients, the association between patient complexity measured by these tools and patient outcomes, and the associated impact on healthcare providers/systems, especially from the perspective of primary care physicians. While specialists are concerned essentially with identifying whether a patient has the disease in which the specialist is an expert, primary care physicians as generalists must recognize a broad range of problems [14]; therefore, the care of highly complex patients is one of the specialties of primary care physicians, and such complexity is an essential concept in primary care. Accordingly, we focus particularly on the perspectives of primary care physicians.
2. Materials and Methods
As we described in the research protocol logged at the OSF (https://osf.io), we conducted this review following the PRISMA extension for scoping reviews (PRISMA-ScR) statement [15]. We also used the scoping review framework of the Joanna Briggs Institute (JBI) [16], following the five-stage approach listed below.
2.1. Stage 1: Identifying the Research Question
We were interested in the patient complexity of the general population or patients. The research questions were as follows:
RQ1. What tools to assess patient complexity are available for the general populations or patients?
RQ2. To what extent do health related patient outcomes differ according to the magnitude of the measured indicators?
RQ3. To what extent does the impact on healthcare providers/systems (e.g., healthcare costs, administrative costs, medical staff burnout) differ depending on the magnitude of the measured indicators?
2.2. Stage 2: Identifying Relevant Studies
We used the population, concept, and context (PCC) framework [16] to define the inclusion criteria as follows:
2.2.1. Participants
We included studies that considered adult general populations or patients (18 years and older). Studies were excluded if they were limited to specific conditions rather than general adult populations or patients, such as psychiatric patients, pediatric patients, and pregnant women. We excluded studies that focused only on patients with specific diseases, such as diabetes, heart failure, and rheumatoid arthritis. In addition, studies that included patients in particular settings (e.g., postoperative, ICU, home healthcare) were excluded. However, inpatients were included if this was not in a disease-specific context.
2.2.2. Concept
We reviewed extant literature that evaluated patient outcomes (e.g., health outcomes, disease control, patient quality of life) and the impact on healthcare providers/systems (e.g., healthcare costs, administrative costs, healthcare professional burnout) through complexity measurement tools. In order to cover related studies broader, we included not only observational studies but also interventional studies that could examine correlations between patient complexity stratified by tools and outcomes at baseline.
We defined patient complexity as “a typology of 4 overarching categories contributing to patient complexity: medical complexity, mental health disorders, socioeconomic factors, and individual patient behaviors or traits” [6]. Therefore, we excluded some tools which mainly focused on biomedical aspects such as comorbidity/multimorbidity measurement tools, end-of-life/palliative specific measures. In addition, we intended to examine the correlations between patient complexity as “exposure” and measured outcomes. We excluded tools which focused on predicting frequent visit/hospitalization, and panel-weighting management tools machine learning prediction tools/algorithms, and software tools, because they focused on predicting outcomes rather than measuring them or examining correlation with patient complexities. For instance, we did not include the frailty score [17], which is widely used and embedded in the electronic patient record systems in the United Kingdom.
We included criterion-related validity studies that validated the performance of the measurement tools against some other criteria, such as discriminative and predictive validities. However, studies of concurrent validity that analyzed correlations between other complexity measurement tools were excluded. All reliability studies other than those that addressed criterion-related validity, such as agreement in measurement results among examiners, were excluded because these reliability studies did not contain outcomes such as patient health related outcomes, healthcare costs, or impacts on healthcare providers.
2.2.3. Context
In this review article, as we stated in the previous section, we defined patient complexity as “a typology of 4 overarching categories contributing to patient complexity: medical complexity, mental health disorders, socioeconomic factors, and individual patient behaviors or traits” [6]. Followed by this definition, we analyzed the title, abstract, and index terms used to characterize the articles found in the preliminary search. We paid attention to select search terms so that they included “patient complexity” and related expressions, and tools or indicators which measured complexity. We also included studies of “patient complexity” as defined by the authors in any setting (including primary and secondary care settings) from any country. We searched the data for all available lengths of each database up until July 2022 without limiting the starting date of the search period. We searched two databases MEDLINE and CINAHL (see Supplementary S1 for search terms and search string). We also checked references from prior systematic reviews [11,12,13], and searched for references in the articles identified for inclusion, using Web of Science. If a study was not indexed in Web of Science (http://webofscience.help.clarivate.com/) (Accessed on 30 June 2022), we used Citationchaser [18] or Google scholar (https://scholar.google.com/intl/en/scholar/about.html) (Accessed on 30 June 2022). All published observational studies with controls, case reports (more than 10 reported cases), and case series were included. Studies in English and from any country were accepted, and we included studies with any length of follow-up. We excluded conference abstracts and review articles. In addition, we consulted a GP in London, an expert of patient complexity management. He gave us some advice without additional papers.
2.3. Stage 3: Study Selection
We selected studies following the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) flow diagram. We used the PCC framework. Two of three reviewers (HK, AH and SYK) selected the abstracts independently. One reviewer (HK) selected the full texts and another (AH or SYK) confirmed the decision. Any differences in opinion were resolved by discussion and, if resolution was not possible, through arbitration by a third researcher (YK). We also conducted a citation search for included articles using several search data bases, as we stated in the previous section.
2.4. Stage 4: Charting the Data
Data extraction was carried out by one researcher (HK) using standard data extraction forms, including the names of authors, year, and country in which each screening tool was developed. HK also extracted the population screened by the tool, the patient outcome (e.g., probability of a person being hospitalized within one year), impact on healthcare providers/systems, and the costs. The other two researchers (AH and SYK) confirmed the data extraction. If there were disagreements, they were resolved through discussion and extraction of related data.
2.5. Stage 5: Collating, Summarizing and Reporting the Results
We extracted the complexity measurement tools and the studies that examined patient outcomes, healthcare costs, and impacts on healthcare providers using complexity measurement tools. For each complexity management tool, we counted and listed the number of studies that examined patient outcomes, healthcare costs, and impacts on healthcare providers. Patient outcomes were classified into 19 detailed categories, as shown in Supplementary S2. We listed categories measured in correlation with complexity in the outcome measurement column of Table 1. If a significant correlation was not observed, related categories on the list were underlined. While the intervention studies that implicitly investigated correlations between patient complexity and outcomes were included, we counted them as no correlations with complexity.
2.6. Differences between Protocol and Review
We expanded the scope of the participants from the general adult population to the general adult population or patients because we recognized that studies related purely to the former, namely community dwelling citizens, were few in number. We defined general adult patients as those not with a specific disease and not in specific settings, such as the ICU or home healthcare. We reduced and clearly defined the complexity measurement tools included in this review by excluding tools such as comorbidity/multimorbidity measurement tools, frequent visit/hospitalization prediction tools, end-of-life/palliative specific measures, general condition panel-weighting management tools, machine learning prediction tools/algorithms, and software tools.
3. Results
3.1. Patient Complexity Measurement Tools Used with the General Adult Population or Patients
After removing duplicates, we identified 933 records via databases. After title, abstract, and full text review, eight studies were identified, which referred to 17 complexity measurement tools. We identified 18 original tool development studies related to the extracted 17 complexity measurement tools (Table 1). If there were unidentified complexity measurement tools, we searched the tool development studies. Finally, we identified 21 complexity management tools and the corresponding 22 tool development studies (Table 2). Nine out of the 21 tools were used predominately in primary care settings, such as with outpatients or community dwelling people. The other 12 tools were used only for inpatients. Regarding the format of the tools, 11 were designed for face-to-face interviews, 7 were intended to extract data from medical records, and the other 3 were self-assessments.
3.2. Correlations between Patient Complexity and Patients’ Health Related Outcomes
In the next step, we conducted a citation search of the 22 tool development studies. We identified 290 studies. Among them, we identified 25 studies (eight tools) that evaluated correlations between patient complexity and outcomes (including criteria-related validity; Figure 1, Table 1). Six out of 25 outcome studies were interventional [21,27,31,35,38,41] (See Supplementary S3 for excluded studies during the process of full-text review). While 8 out of 22 complexity measurement tools were used to stratify participants by complexity profiles to measure correlations between complexity and outcomes, the other 14 tools have not yet been used for that purpose (Table 3). We identified 19 health-related patient outcomes. INTERMED and its derived tools were associated with studies that considered 17 outcomes. The outcome coverage of COMPRI, PCAM, SWAAT, Bandini’s tool, and COMPLEXedex were 4, 3, 3, 2 and 1, respectively. The length of hospital stay was the outcome most examined through complexity measurement tools, which directly referenced patient health condition. Following the length of hospital stay, mental health and health-related quality of life were the variables most frequently examined (Supplementary S2).
3.3. Correlations between Patient Complexity and Impacts on Healthcare Providers/Systems
PCAM is the only measurement tool that was used to describe the burden of health-related staff. IM-E and IM-E-SA were used to evaluate healthcare costs related to complex patients.
4. Discussion
This scoping review identified 21 complexity measurement tools. While 9 out of 21 tools were used in primary care settings, the others were used only for inpatients. We identified 25 studies that investigated the correlation between patients’ complexity and health-related outcomes [19,20,21,22,23,24,25,26,27,29,30,31,32,33,34,35,36,37,38,40,41,42,43]. Two studies investigated the correlation between patients’ complexity with healthcare related costs [25,28]. We found only one study that investigated the correlation between patients’ complexity and impacts on healthcare providers [39].
To the best of our knowledge, this is the first scoping review to comprehensively search for patient complexity measurement tools and reveal the correlation between patients’ complexity with outcomes, healthcare costs, and impact on healthcare providers in the general population or patients. We found three previous reviews that investigated complexity measurement tools. Marcoux et al. (2017) focused on the risk of high use of healthcare services associated with complex healthcare needs by using complexity screening tools [13]. They identified 14 different screening tools. We included 4 out of these 14 tools, namely IM, IM-E, IM-SA and IM-E-SA. We excluded the other 10 tools, which were specialized for predicting high frequency of healthcare use or targeted specific populations, such as homeless persons, rather than the general population or patients. Hawker et al. (2021) identified complexity tools that focused on rheumatic disease [12]. They identified two complexity measurement tools for patients with rheumatic disease, namely INTERMED and SLENQ. SLENQ is specialized for patients with systemic lupus erythematosus. Nicolaus et al. (2022) reviewed definitions of patient complexity, which included complexity defined with complexity measurement tools or conceptual models [11]. They concluded that there exists considerable heterogeneity in definition of patient complexity. None of the three review articles focused on tools used in general populations or patients, or outcomes (health-related patient outcomes, healthcare costs, and impacts on healthcare providers).
Theoretically, and especially in primary care settings, the concept of complexity can depict actual clinical situations better than the conventional biomedical model [11]; however, the evidence is insufficient for clinical implementation in that the correlations between patient complexity and outcomes were not clear, and any intervention protocols were not definitely effective for patients stratified by complexity measurement tools. While specialists are concerned essentially with identifying whether the patient has the disease in which they are expert, primary care physicians as generalists must recognize a broad range of problems [14]. This partially explains why the challenges faced by primary care physicians in their workday are not captured sufficiently by conventional biomedical scales. Therefore, complexity measurement tools are required, especially in primary care settings, to determine problems and opportunities [7].
We found that primary care clinicians do not currently have sufficient supportive evidence to routinely measure complexity to improve health-related outcomes based on identified studies. We identified 6 intervention studies and 19 observational studies that examined the correlation between patient complexity and health-related outcomes. In the observational studies, health-related outcomes were generally correlated with complexity (Table 1); however, insufficiently many studies found clear correlations between patient complexity and health-related outcomes. Although the number of studies were limited, we found some studies that showed greater complexity was related to longer duration of hospital stays, worse health-related quality of life, and more mental health problems. Fewer studies focused on various other exposures such as nutrition, substance misuse, and oral health (Supplementary S2).
Regarding intervention studies, although some theoretical approaches have been attempted [9], there are currently no practical, effective intervention protocols for patients’ complexity. We included six intervention studies [21,27,31,35,38,41] that tracked the time development of outcomes related to complex patients stratified by tools. Four out of six studies [27,31,35,41] did not meaningfully improve health-related outcomes. One found a reduction of the length of hospital stays merely for aged 65 years or older, while no significant effect on that was found for the whole sample [21]. The other one suggested that clinical informational workflows that incorporate social determinants of health data can improve outcomes for patients; however, practical intervention protocol was not proposed [38].
Researchers should conduct additional studies to investigate the correlation between complexity and outcomes in the primary care setting, where the number of complex patients is increasing because of various factors, such as an aging population, increasing prevalence of multimorbidity, and polypharmacy. Researchers should also pay attention to primary care physicians’ situation, in that they usually work understaffed and sometimes must treat a range of complex patients. Some interview tools have been developed specific to the primary care setting [57,58,61]; however, no studies to date have evaluated outcomes. One reason for this may be the time required for measurement.
To develop further research by reducing the burden of measurement, self-administered tools might be preferable than interviewing. For example, one of the interview tools (MCAM) was found to be cumbersome and time-consuming [44]. We included three self-assessment tools in this review, namely IMSA, IM-E-SA and CONECT-6. The IM-E-SA can be utilized in the primary care setting, albeit for older adults [30]. The INTERMED self-administered (IMSA) [49] is another candidate for evaluating outcome, even in primary care settings and not only for older persons. IMSA was developed with reference to IM-E-SA, which was developed and validated four years prior to IMSA. CONECT-6 is used only in emergency department settings. In addition, no study has investigated the correlation with outcomes.
Regarding the area to be investigated in the future, these tools will likely be used to investigate the association between patient complexity and outcomes. In particular, focusing on healthcare costs and impacts on healthcare providers are warranted. To determine effective intervention protocols, a multidisciplinary approach is essential; however, no studies have considered sharing information or action plans with multidisciplinary teams while measuring outcomes for complex patients stratified using tools. Studies of the usefulness of tools to measure complexity would be valuable for information sharing and action planning with multidisciplinary teams.
There are several limitations in our study. First, to develop the search formula in this review, we focused on words related to “complexity.” Because we did not search for constructs related to the concept of complexity, we could have omitted complexity tools that described the concept of complexity but did not include words related to “complexity” per se. Second, only two databases were searched, and only papers in English were selected. This might miss some relevant articles. To overcome this problem, we conducted a citation search with multiple search engines. Third, we conducted a scoping review; therefore, this research does not calculate correlations between patient complexity and outcomes. Fourth, our review did not include perspectives of other health care providers from a primary care perspective. Further review is warranted for other perspectives.
5. Conclusions
This scoping review found that 8 out of 21 patient complexity tools were used in studies that evaluated correlations with outcomes. Further research is required to reveal the correlations between patient complexity and outcomes in the primary care setting.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph192316113/s1. Supplementary S1: Search strategies; Supplementary S2: Correlation between the measured complexity and the outcomes; Supplementary S3: Characteristics of the studies excluded from the qualitative syntheses.
Author Contributions
Conceptualization, H.K., Y.K. and T.A.; methodology, Y.K. and T.A.; data curation, H.K., A.H. and S.Y.-K.; writing—original draft preparation, H.K.; writing—review and editing, Y.K.; writing—review and comments, T.A., A.H., S.Y.-K., K.S. and H.I.; supervision, Y.K., T.A., K.S. and H.I.; All authors have read and agreed to the published version of the manuscript.
Funding
English editing costs were supported in part by a JSPS Grant-in-Aid for Scientific Research (Grant Number 22K15664) to Y.K.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The datasets generated during the current study are available from the corresponding author on reasonable request.
Acknowledgments
The authors would like to express our gratitude to Arben Rrukaj for his thoughtful advice and kind comments on our work. Arben has encouraged H.K. to conduct this review since the latter conceived it. Arben enabled H.K. to be placed at the CPMG (Complex Patient Management Group) led by the former in Brent, UK; without this support H.K. would not have been able to conceive of this review.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
References
- World Health Organization. Constitution of the World Health Organization. Chron. World Health Organ. 1947, 1, 29–43. [Google Scholar]
- Engel, G.L. The need for a new medical model: A challenge for biomedicine. Science 1977, 196, 129–136. [Google Scholar] [CrossRef] [PubMed]
- Marmot, M. The health gap: The challenge of an unequal world. Lancet 2015, 386, 2442–2444. [Google Scholar] [CrossRef] [PubMed]
- Weiss, K.B. Managing complexity in chronic care: An overview of the VA state-of-the-art (SOTA) conference. J. Gen. Intern. Med. 2007, 22 (Suppl. 3), 374–378. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Grembowski, D.; Schaefer, J.; Johnson, K.E.; Fischer, H.; Moore, S.L.; Tai-Seale, M.; Ricciardi, R.; Fraser, J.R.; Miller, D.; LeRoy, L. A conceptual model of the role of complexity in the care of patients with multiple chronic conditions. Med. Care 2014, 52 (Suppl. 3), S7–S14. [Google Scholar] [CrossRef]
- Loeb, D.F.; Binswanger, I.A.; Candrian, C.; Bayliss, E.A. Primary care physician insights into a typology of the complex patient in primary care. Ann. Fam. Med. 2015, 13, 451–455. [Google Scholar] [CrossRef] [Green Version]
- Grant, R.W.; Ashburner, J.M.; Hong, C.S.; Chang, Y.; Barry, M.J.; Atlas, S.J. Defining patient complexity from the primary care physician’s perspective: A cohort study. Ann. Fam. Med. 2011, 155, 797–804. [Google Scholar] [CrossRef]
- Schaink, A.K.; Kuluski, K.; Lyons, R.F.; Fortin, M.; Jadad, A.R.; Upshur, R.; Wodchis, W.P. A scoping review and thematic classification of patient complexity: Offering a unifying framework. J. Comorbidity 2012, 2, 1–9. [Google Scholar] [CrossRef]
- Martin, C.M.; Sturmberg, J.P. General practice—Chaos, complexity and innovation. Med. J. Aust. 2005, 183, 106–109. [Google Scholar] [CrossRef]
- Safford, M.M.; Allison, J.J.; Kiefe, C.I. Patient complexity: More than comorbidity. the vector model of complexity. J. Gen. Intern. Med. 2007, 22 (Suppl. 3), 382–390. [Google Scholar] [CrossRef] [Green Version]
- Nicolaus, S.; Crelier, B.; Donzé, J.D.; Aubert, C.E. Definition of patient complexity in adults: A narrative review. J. Multimorb. Comorbidity 2022, 12, 263355652210812. [Google Scholar] [CrossRef] [PubMed]
- Hawker, K.; Barnabe, C.; Barber, C.E.H. A scoping Review of tools used to assess patient Complexity in rheumatic disease. Health Expect 2021, 24, 556–565. [Google Scholar] [CrossRef] [PubMed]
- Marcoux, V.; Chouinard, M.-C.; Diadiou, F.; Dufour, I.; Hudon, C. Screening tools to identify patients with complex health needs at risk of high use of health care services: A scoping review. PLoS ONE 2017, 12, e0188663. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Etz, R.; Miller, W.L.; Stange, K.C. Simple Rules That Guide Generalist and Specialist Care. Fam. Med. 2021, 53, 697–700. [Google Scholar] [CrossRef] [PubMed]
- Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L.; et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Peters, M.D.; Godfrey, C.M.; Khalil, H.; McInerney, P.; Parker, D.; Soares, C.B. Guidance for conducting systematic scoping reviews. Int. J. Evid.-Based Healthc. 2015, 13, 141–146. [Google Scholar] [CrossRef] [Green Version]
- Clegg, A.; Young, J.; Iliffe, S.; Rikkert, M.O.; Rockwood, K. Frailty in elderly people. Lancet 2013, 381, 752–762. [Google Scholar] [CrossRef] [Green Version]
- Haddaway, N.R.; Grainger, M.J.; Gray, C.T. citationchaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching. Res. Synth. Methods 2022, 13, 533–545. [Google Scholar] [CrossRef]
- De Jonge, P.; Huyse, F.J.; Stiefel, F.C.; Slaets, J.P.J.; Gans RO, B. INTERMED—A clinical instrument for biopsychosocial assessment. Psychosomatics 2001, 42, 106–109. [Google Scholar] [CrossRef]
- De Jonge, P.; Bauer, I.; Huyse, F.J.; Latou, C.H.M. Medical inpatients at risk of extended hospital stay and poor discharge health status: Detection with COMPRI and INTERMED. Psychosom. Med. 2003, 65, 534–541. [Google Scholar] [CrossRef]
- De Jonge, P.; Latour, C.H.M.; Huyse, F.J. Implementing psychiatric interventions on a medical ward: Effects on patients’ quality of life and length of hospital stay. Psychosom. Med. 2003, 65, 997–1002. [Google Scholar] [CrossRef] [PubMed]
- Lobo, E.; Ventura, T.; Navio, M.; Santabárbara, J.; Kathol, R.; Samaniego, E.; Marco, C.; Lobo, A. Identification of components of health complexity on internal medicine units by means of the INTERMED method. Int. J. Clin. Pract. 2015, 69, 1377–1386. [Google Scholar] [CrossRef] [PubMed]
- De Oliveira, C.A.; Weber, B.; Santos, J.L.F.d.; Zucoloto, M.L.; de Camargo, L.L.; Zanetti, A.C.G.; Rzewuska, M.; de Azevedo-Marques, J.M. Health complexity assessment in primary care: A validity and feasibility study of the INTERMED tool. PLoS ONE 2022, 17, e0263702. [Google Scholar] [CrossRef] [PubMed]
- Da Silva, H.S.; Gutierrez, B.A.O. Care complexity in hospitalized elderly according to cognitive performance. Rev. Bras. Enferm. 2019, 72, 134–139. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wild, B.; Heider, D.; Maatouk, I.; Slaets, J.; König, H.; Niehoff, D.; Saum, K.; Brenner, H.; Söllner, W.; Herzog, W. Significance and costs of complex biopsychosocial health care needs in elderly people: Results of a population-based study. Psychosom. Med. 2014, 76, 497–502. [Google Scholar] [CrossRef] [PubMed]
- Boehlen, F.H.; Herzog, W.; Schellberg, D.; Maatouk, I.; Saum, K.-U.; Brenner, H.; Wild, B. Self-perceived coping resources of middle-aged and older adults—Results of a large population-based study. Aging Ment. Health 2016, 21, 1303–1309. [Google Scholar] [CrossRef] [PubMed]
- Uittenbroek, R.J.; Van Asselt, A.D.I.; Spoorenberg, S.L.W.; Kremer, H.P.H.; Wynia, K.; Reijneveld, S. Integrated and Person-Centered Care for Community-Living Older Adults: A Cost-Effectiveness Study. Health Serv. Res. 2018, 53, 3471–3494. [Google Scholar] [CrossRef]
- Peters, L.L.; Burgerhof, J.G.; Boter, H.; Wild, B.; Buskens, E.; Slaets, J.P. Predictive validity of a frailty measure (GFI) and a case complexity measure (IM-E-SA) on healthcare costs in an elderly population. J. Psychosom. Res. 2015, 79, 404–411. [Google Scholar] [CrossRef]
- Bakker, M.H.; Vissink, A.; Spoorenberg, S.L.W.; Wynia, K.; Visser, A. Self-reported oral health problems and the ability to organize dental care of community-dwelling elderly aged ≥75 years. BMC Oral Health 2020, 20, 185. [Google Scholar] [CrossRef]
- Hoeksema, A.R.; Spoorenberg, S.; Peters, L.; Meijer, H.; Raghoebar, G.; Vissink, A.; Wynia, K.; Visser, A. Elderly with remaining teeth report less frailty and better quality of life than edentulous elderly: A cross-sectional study. Oral Dis. 2017, 23, 526–536. [Google Scholar] [CrossRef]
- Spoorenberg, S.L.; Reijneveld, S.A.; Uittenbroek, R.J.; Kremer, H.P.; Wynia, K. Health-Related Problems and Changes After 1 Year as Assessed With the Geriatric ICF Core Set (GeriatrICS) in Community-Living Older Adults Who Are Frail Receiving Person-Centered and Integrated Care From Embrace. Arch. Phys. Med. Rehabil. 2019, 100, 2334–2345. [Google Scholar] [CrossRef] [PubMed]
- Peters, L.L.; Boter, H.; Slaets, J.P.; Buskens, E. Development and measurement properties of the self assessment version of the INTERMED for the elderly to assess case complexity. J. Psychosom. Res. 2013, 74, 518–522. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dortland, A.; Peters, L.L.; Boenink, A.D.; Smit, J.H.; Slaets, J.P.; Hoogendoorn, A.W.; Joos, A.; Latour, C.H.; Stiefel, F.; Burrus, C.; et al. Assessment of Biopsychosocial Complexity and Health Care Needs: Measurement Properties of the INTERMED Self-Assessment Version. Psychosom. Med. 2017, 79, 485–492. [Google Scholar] [CrossRef] [PubMed]
- Bakker, M.H.; Vissink, A.; Spoorenberg, S.L.; Jager-Wittenaar, H.; Wynia, K.; Visser, A. Are Edentulousness, Oral Health Problems and Poor Health-Related Quality of Life Associated with Malnutrition in Community-Dwelling Elderly (Aged 75 Years and Over)? A Cross-Sectional Study. Nutrients 2018, 10, 1965. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Spoorenberg, S.L.W.; Wynia, K.; Uittenbroek, R.J.; Kremer, H.P.H.; Reijneveld, S. Effects of a population-based, person-centred and integrated care service on health, wellbeing and self-management of community-living older adults: A randomised controlled trial on Embrace. PLoS ONE 2018, 13, e0190751. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Yokokawa, D.; Shikino, K.; Kishi, Y.; Ban, T.; Miyahara, S.; Ohira, Y.; Yanagita, Y.; Yamauchi, Y.; Hayashi, Y.; Ishizuka, K.; et al. Does scoring patient complexity using COMPRI predict the length of hospital stay? A multicentre case–control study in Japan. BMJ Open 2022, 12, e051891. [Google Scholar] [CrossRef] [PubMed]
- Yoshida, S.; Matsushima, M.; Wakabayashi, H.; Mutai, R.; Murayama, S.; Hayashi, T.; Ichikawa, H.; Nakano, Y.; Watanabe, T.; Fujinuma, Y. Validity and reliability of the Patient Centred Assessment Method for patient complexity and relationship with hospital length of stay: A prospective cohort study. BMJ Open 2017, 7, e016175. [Google Scholar] [CrossRef] [Green Version]
- Hewner, S.; Casucci, S.; Sullivan, S.; Mistretta, F.; Xue, Y.; Johnson, B.; Pratt, R.; Lin, L.; Fox, C. Integrating Social Determinants of Health into Primary Care Clinical and Informational Workflow during Care Transitions. EGEMS 2017, 5, 2. [Google Scholar] [CrossRef] [Green Version]
- Yoshida, S.; Matsushima, M.; Wakabayashi, H.; Mutai, R.; Sugiyama, Y.; Yodoshi, T.; Horiguchi, R.; Watanabe, T.; Fujinuma, Y. Correlation of patient complexity with the burden for health-rel ated professions, and differences in the burden between the professions at a Japanese regional hospital: A prospective cohort study. BMJ Open 2019, 9, e025176. [Google Scholar] [CrossRef] [Green Version]
- Sugiyama, Y.; Matsushima, M.; Yoshimoto, H. Association between alcohol consumption/alcohol use disorders and patient complexity: A cross-sectional study. BMJ Open 2020, 10, e034665. [Google Scholar] [CrossRef]
- Hawner, S.; Seo, J.Y.; Gothard, S.E.; Johnson, B.J. Aligning population-based care management with chronic disease complexity. Nurs. Outlook 2014, 62, 250–258. [Google Scholar] [CrossRef] [PubMed]
- Boutin-Foster, C.; Euster, S.; Rolon, Y.; Motal, A.; BeLue, R.; Kline, R.; Charlson, M.E. Social Work Admission Assessment Tool for Identifying Patients in Need of Comprehensive Social Work Evaluation. Health Soc. Work 2005, 30, 117–125. [Google Scholar] [CrossRef]
- Bandini, F.; Guidi, S.; Blaszczyk, S.; Fumarulo, A.; Pierini, M.; Pratesi, P.; Spolveri, S.; Padeletti, M.; Petrone, P.; Zoppi, P.; et al. Complexity in internal medicine wards: A novel screening method and implications for management. J. Eval. Clin. Pract. 2017, 24, 285–292. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Huyse, F.J.; Lyons, J.S.; Stiefel, F.C.; Slaets, J.P.; de Jonge, P.; Fink, P.; Gans, R.O.; Guex, P.; Herzog, T.; Lobo, A.; et al. “INTERMED”: A method to assess health service needs. I. Development and reliability. Gen. Hosp. Psychiatry 1999, 21, 39–48. [Google Scholar] [CrossRef] [PubMed]
- Stiefel, F.C.; de Jonge, P.; Huyse, F.J.; Guex, P.; Slaets, J.P.; Lyons, J.S.; Spagnoli, J.; Vannotti, M. “INTERMED”: A method to assess health service needs. II. Results on its validity and clinical use. Gen. Hosp. Psychiatry 1999, 21, 49–56. [Google Scholar] [CrossRef]
- Wild, B.; Lechner, S.; Herzog, W.; Maatouk, I.; Wesche, D.; Raum, E.; Müller, H.; Brenner, H.; Slaets, J.; Huyse, F.; et al. Reliable integrative assessment of health care needs in elderly persons: The INTERMED for the Elderly (IM-E). J. Psychosom. Res. 2010, 70, 169–178. [Google Scholar] [CrossRef]
- Huyse, F.J.; de Jonge, P.; Slaets, J.P.; Herzog, T.; Lobo, A.; Lyons, J.S.; Opmeer, B.C.; Stein, B.; Arolt, V.; Balogh, N.; et al. COMPRI—An Instrument to Detect Patients With Complex Care Needs: Results From a European Study. J. Psychosom. Res. 2001, 42, 222–228. [Google Scholar] [CrossRef]
- Pratt, R.; Hibberd, C.; Cameron, I.M.; Maxwell, M. The Patient Centered Assessment Method (PCAM): Integrating the Social Dimensions of Health into Primary Care. J. Comorbidity 2015, 5, 110–119. [Google Scholar] [CrossRef]
- Hewner, S.; Wu, Y.W.B.; Castner, J. Comparative Effectiveness of Risk-Stratified Care Management in Reducing Readmissions in Medicaid Adults With Chronic Disease. J. Healthc. Qual. 2016, 38, 3–16. [Google Scholar]
- Hudon, C.; Bisson, M.; Dubois, M.-F.; Chiu, Y.; Chouinard, M.-C.; Dubuc, N.; Elazhary, N.; Sabourin, V.; Vanasse, A. CONECT-6: A case-finding tool to identify patients with complex health needs. BMC Health Serv. Res. 2021, 21, 157. [Google Scholar]
- McDonough, R.P.; McDonough, A.; Doucette, W.R. Use of medication regimen complexity to target services in the community. J. Am. Pharm. Assoc. JAPhA 2021, 62, 750–756. [Google Scholar] [CrossRef] [PubMed]
- Roberson, D.; Connell, M.; Dillis, S.; Gauvreau, K.; Gore, R.; Heagerty, E.; Jenkins, K.; Ma, L.; Maurer, A.; Stephenson, J.; et al. Cognitive Complexity of the Medical Record Is a Risk Factor for Major Adverse Events. Perm. J. 2014, 18, 4–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Groningen, N.V.; Prasad, P.A.; Najafi, N.; Rajkomar, A.; Khanna, R.R.; Fang, M.C. Electronic Order Volume as a Meaningful Component in Estimating Patient Complexity and Resident Physician Workload. J. Hosp. Med. 2018, 13, 829–835. [Google Scholar] [CrossRef]
- Liechti, F.D.; Beck, T.; Ruetsche, A.; Roumet, M.C.; Limacher, A.; Tritschler, T.; Donzé, J.D. Development and validation of a score to assess complexity of general internal medicine patients at hospital discharge: A prospective cohort study. BMJ Open 2021, 11, e041205. [Google Scholar] [CrossRef]
- Mehra, T.; Müller, C.T.B.; Volbracht, J.; Seifert, B.; Moos, R. Predictors of High Profit and High Deficit Outliers under SwissDRG of a Tertiary Care Center. PLoS ONE 2015, 10, e0140874. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Bonizzoni, E.; Gussoni, G.; Agnelli, G.; Incalzi, R.A.; Bonfanti, M.; Mastroianni, F.; Candela, M.; Franchi, C.; Frasson, S.; Greco, A.; et al. The complexity of patients hospitalized in Internal Medicine wards evaluated by FADOI-COMPLIMED score(s). A hypothetical approach. PLoS ONE 2018, 13, e0195805. [Google Scholar] [CrossRef] [Green Version]
- Mount, J.K.; Massanari, R.M.; Teachman, J. Patient care complexity as perceived by primary care physicians. Fam. Syst. Health J. Collab. Fam. Healthc. 2015, 33, 137–145. [Google Scholar] [CrossRef]
- Maxwell, M.; Hibberd, C.; Pratt, R.; Cameron, I.; Mercer, S. Development and Initial Validation of the Minnesota Edinburgh Complexity Assessment Method (MECAM) for Use Within the Keep Well Health Check; NHS Health Scotland: Edinburgh, UK, 2011. [Google Scholar]
- Troigros, O.; Béjot, Y.; Rodriguez, P.M.; Shoaib, F.; Ellis, H.; Wade, D. Measuring complexity in neurological rehabilitation: The Oxford Case Complexity Assessment Measure (OCCAM). Clin. Rehabil. 2013, 28, 499–507. [Google Scholar] [CrossRef]
- Corazza, G.R.; Panel, F.T.C.; Klersy, C.; Formagnana, P.; Lenti, M.V.; Padula, D. A consensus for the development of a vector model to assess clinical complexity. Intern. Emerg. Med. 2017, 12, 1313–1318. [Google Scholar] [CrossRef]
- Salisbury, C.; Lay-Flurrie, S.; Bankhead, C.R.; Fuller, A.; Murphy, M.; Caddick, B.; Ordóñez-Mena, J.M.; Holt, T.; Nicholson, B.D.; Perera, R.; et al. Measuring the complexity of general practice consultations: A Delphi and cross-sectional study in English primary care. Br. J. Gen. Pract. 2020, 71, e423–e431. [Google Scholar] [CrossRef]
Figure 1.
PRISMA flow chart.
Table 1.
Characteristics of included studies.
| Tool | Author (Year) | Study Design | Setting | n | Location | Outcome Measurement |
|---|---|---|---|---|---|---|
| INTERMED | Jonge (2001) [19] | Cohort study | Consecutively admitted patients on a general internal medicine ward | 89 | Netherlands |
|
| Jonge (2003) [20] | Cohort study | Patients from admission to discharge | 275 | Netherlands |
| |
| Jonge (2003) [21] | Interventional study | Inpatients | 193 | Netherlands |
| |
| Lobo (2015) [22] | Cohort study | Internal medicine inpatients | 626 | Spain |
| |
| C.A. Oliveira [23] | Cohort study | All adults who arrived at the reception of one of the health services | 230 | Brazil |
| |
| H.S. da Silva [24] | Cross sectional and analytical study | Inpatient | 382 | Brazil |
| |
| INTERMED for Elderly | B. Wild (2014) [25] | Cohort Study | General health check-up people | 3121 | Germany |
|
| F.H. Boehlen (2017) [26] | Cohort Study | Health check-up patients | 9949 | Germany |
| |
| R.J. Uittenbroek (2018) [27] | RCT | All adults aged 75 and older, listed with these GP | 1456 | Netherlands |
| |
| IM-E-SA | L.L. Peters (2015) [28] | Population-based cohort | General population | 713 | Netherlands | Healthcare costs |
| M.H. Bakker (2020) [29] | Cohort Study | Community-dwelling elderly | 89 | Netherlands | Oral status and oral health | |
| A.R. Hoeksema (2017) [30] | Crosssectional descriptive study | Community-dwelling elderly | 1026 | Netherlands | Oral status and oral health | |
| R.J. Uittenbroek (2018) [27] | RCT | All adults aged 75 and older, listed with GPs | 1456 | Netherlands |
| |
| S.L. Spoorenberg (2019) [31] | RCT | Community (follow-up) | 136 | Netherlands | Geriatric ICF (International Classification of Functioning) Core Set
| |
| L.L. Peters (2013) [32] | Cross-sectional study | General population | 338 | Netherlands |
| |
| Dortland (2017) [33] | Cohort study | Inpatients and outpatients | 850 | Germany France Italy Netherlands Switzerland |
| |
| M.H. Bakker (2018) [34] | Cross sectional observational study | Community-living elderly | 1325 | Netherlands |
| |
| S.L. Spoorenberg (2018) [35] | RCT | Inpatients and outpatients | 850 | Germany France Italy Netherlands Switzerland |
| |
| COMPRI | Jonge (2003) [20] | Cohort study | Patients from admission to discharge | 275 | Netherlands |
|
| Jonge (2003) [21] | Interventional study | Inpatients | 193 | Netherlands |
| |
| D. Yokokawa (2022) [36] | Case control study | Newly hospitalised patients | 33 | Japan | Length of hospital stays | |
| PCAM | S. Yoshida (2017) [37] | Prospective cohort study | Inpatient | 201 | Japan | Length of hospital stays |
| Hewner (2017) [38] | Interventional study | Patients who were discharged to the community | 419 | USA |
| |
| S. Yoshida (2018) [39] | Cohort study | All inpatients admitted to the acute care unit of Hospital | 201 | Japan | Burden for health-related staff | |
| Y. Sugiyama (2020) [40] | Cross-sectional study | Outpatient practices | 426 | Japan | Alcohol misuse | |
| COMPLEXedex | S. Hawner (2014) [41] | Before-and-after study | Regional health plan in 2009 | 411,407 | USA | Hospital inpatient utilization |
| SWAAT | Boutin-Foster (2005) [42] | Cohort | Newly hospitalised patients | 299 | USA |
|
| Bandini’s tool | Bandini (2018) [43] | Observational study | Inpatients | 240 | Italy |
|
Note: [] means reference number. Interventional studies are [21,27,31,35,38,41]. Notation: LOS: Length of hospital stays, HLQOL: Health related Quality of life. While the underlined item in an Outcome Measurement column were not observed correlations with complexities, other items in the column shows correlations with complexity scores.
Table 2.
Complexity measurement tools, intended use, applied setting, and format.
| Complexity Tool | Full Name of Tool | Intended Use of the Tool | Applied Setting | Format of the Tool |
|---|---|---|---|---|
| IM [44,45] | INTERMED | Indication for multidisciplinary care | Inpatients | Face to face interview |
| IMSA Dortland | INTERMED Self-Assessment | Self-assessment version of IM | Inpatients | Self-assessment |
| IM-E [46] | INTERMED for the Elderly | Elderly version of IM | Inpatients, outpatients, community dwelling people | Face to face interview |
| IM-E-SA [32] | INTERMED for the Elderly Self-Assessment | Self-assessment version of IM-E | Inpatients, outpatients, community dwelling people | Self-assessment |
| COMPRI [47] | Complexity Prediction Instrument | Indication for multidimensional assessment and interdisciplinary care coordination | Inpatients | Face to face interview |
| PCAM [48] | Patient Centered Assessment Method | Identify biopsychosocial complexities; make appropriate referrals | Inpatient, Outpatient | Face to face interview |
| COMPLEXedex [49] | Rank individuals with multiple chronic diseases hierarchically into segments | Inpatients, Community dwelling people | Calculated using the amount of data in the medical records | |
| CONECT-6 [50] | COmplex NEeds Case-finding Tool-6 | Identify patients with chronic conditions and complex health needs in emergency Departments | Emergency departments | Self-assessment |
| IMECSs [51] | Iowa Medication Complexity Scores | Describe challenges a patient faces in developing medication regimen | Outpatients | Calculated automatically in real time using clinical documentation system |
| CCMR [52] | Cognitive complexity of the medical record | Use CCMR as a useful surrogate for true patient complexity | Inpatients | Calculated using the amount of data in the medical records |
| Electric order volume [53] | Use electronic order volume a useful surrogate for the workload of residents | Inpatients | Calculated using the data in the medical records | |
| PCA [54] | Patient Complexity Assessment score | Use readily available administrative and clinic data to identify complex inpatients | Inpatients | Calculated using the data in the medical records |
| PCCL [55] | Patient clinical complexity level | Indicate a calculated index of disease burden per patient based on the amount and constellation of secondary diagnoses | Inpatients | Calculated using the data in the medical records |
| FADOI-Complimed [56] | Federation of Associations of Hospital Doctors on Internal Medicine Complimed | Describe the complexity of hospitalized patients as a two-dimensional phenomenon | Inpatients | Face to face interview |
| MCAM [57] | Minnesota Complexity Assessment Method | Identify factors interfering with care; formulate new care plan | Outpatients | Face to face interview |
| MECAM [58] | Minnesota Edinburgh Complexity Assessment Method | Identify factors posing risk to patient well-being | Outpatients | Face to face interview |
| OCCAM [59] | Oxford Case Complexity Assessment Method | Identify factors interfering with care; facilitate care coordination | Inpatients | Face to face interview |
| Corazza’s score [60] | Comprehensive assessment of clinical complexity in which an overall index results from the sum of each vector/domain | Outpatients | Face to face interview | |
| Salisbury’s score [61] | Develop a valid and reliable measure of the complexity of general practice consultations | Outpatients | Face to face interview | |
| SWAAT [42] | Social Work Admission Assessment Tool | SWAAT was developed to identify patients who have complicated discharge planning needs and require early social work evaluation. | Inpatients | Face to face interview |
| Bandini’s tool [43] | To identify patients with clinically complex hospitalization events | Inpatients | Extracted data from the hospital medical records |
Note: [] are the studies which developed the measurement tools.
Table 3.
Correlations between the measured complexity and outcomes.
| Complexity Measurement Tools | ||||||||
|---|---|---|---|---|---|---|---|---|
| INTERMED | INTERMED for Elderly | IM-E-SA | COMPRI | PCAM | COMPLEXedex | SWAAT | Bandini’s Tool | |
| Patient outcomes | 6 [19,20,21,22,23,24] | 3 [25,26,27] | 8 [27,29,30,31,32,33,34,35] | 3 [20,21,36] | 3 [37,38,40] | 1 [41] | 1 [42] | 1 [43] |
| Health care costs | 1 [25] | 1 [28] | ||||||
| Impacts on health care providers | 1 [39] | |||||||
Note: 1–8 means the number of included studies. [] means reference number.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Kaneko, H.; Hanamoto, A.; Yamamoto-Kataoka, S.; Kataoka, Y.; Aoki, T.; Shirai, K.; Iso, H. Evaluation of Complexity Measurement Tools for Correlations with Health-Related Outcomes, Health Care Costs and Impacts on Healthcare Providers: A Scoping Review. Int. J. Environ. Res. Public Health 2022, 19, 16113. https://doi.org/10.3390/ijerph192316113
AMA Style
Kaneko H, Hanamoto A, Yamamoto-Kataoka S, Kataoka Y, Aoki T, Shirai K, Iso H. Evaluation of Complexity Measurement Tools for Correlations with Health-Related Outcomes, Health Care Costs and Impacts on Healthcare Providers: A Scoping Review. International Journal of Environmental Research and Public Health. 2022; 19(23):16113. https://doi.org/10.3390/ijerph192316113
Chicago/Turabian StyleKaneko, Hiromitsu, Akiko Hanamoto, Sachiko Yamamoto-Kataoka, Yuki Kataoka, Takuya Aoki, Kokoro Shirai, and Hiroyasu Iso. 2022. "Evaluation of Complexity Measurement Tools for Correlations with Health-Related Outcomes, Health Care Costs and Impacts on Healthcare Providers: A Scoping Review" International Journal of Environmental Research and Public Health 19, no. 23: 16113. https://doi.org/10.3390/ijerph192316113
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
