Next Article in Journal
Association of Skeletal Muscle and Adipose Tissue Distribution with Histologic Severity of Non-Alcoholic Fatty Liver
Previous Article in Journal
A Laboratory-Based Surveillance Study of Invasive Neisseria meningitidis, Streptococcus pneumoniae, and Haemophilus influenzae Diseases in a Serbian Pediatric Population—Implications for Vaccination
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 11
Issue 6
10.3390/diagnostics11061060
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Long-Term Mortality in Patients with Angina across the Spectrum of Dysglycemia: A Machine Learning Approach

by
Yu-Hsuan Li
1,2,
Wayne Huey-Herng Sheu
1,3,
Wen-Chao Yeh
4,
Yung-Chun Chang
5,6,* and
I-Te Lee
1,3,7,8,*
1
Division of Endocrinology and Metabolism, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan
2
Department of Computer Science & Information Engineering, National Taiwan University, Taipei 10617, Taiwan
3
School of Medicine, National Yang-Ming University, Taipei 11221, Taiwan
4
Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu 30013, Taiwan
5
Graduate Institute of Data Science, Taipei Medical University, Taipei 10675, Taiwan
6
Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei 10675, Taiwan
7
School of Medicine, Chung Shan Medical University, Taichung 40201, Taiwan
8
College of Science, Tunghai University, Taichung 40704, Taiwan
*
Authors to whom correspondence should be addressed.
Diagnostics 2021, 11(6), 1060; https://doi.org/10.3390/diagnostics11061060
Submission received: 19 April 2021 / Revised: 2 June 2021 / Accepted: 5 June 2021 / Published: 9 June 2021
(This article belongs to the Section Machine Learning and Artificial Intelligence in Diagnostics)
Browse Figures
Versions Notes

Abstract

:
We aimed to develop and validate a model for predicting mortality in patients with angina across the spectrum of dysglycemia. A total of 1479 patients admitted for coronary angiography due to angina were enrolled. All-cause mortality served as the primary endpoint. The models were validated with five-fold cross validation to predict long-term mortality. The features selected by least absolute shrinkage and selection operator (LASSO) were age, heart rate, plasma glucose levels at 30 min and 120 min during an oral glucose tolerance test (OGTT), the use of angiotensin II receptor blockers, the use of diuretics, and smoking history. This best performing model was built using a random survival forest with selected features. It had a good discriminative ability (Harrell’s C-index: 0.829) and acceptable calibration (Brier score: 0.08) for predicting long-term mortality. Among patients with obstructive coronary artery disease confirmed by angiography, our model outperformed the Global Registry of Acute Coronary Events discharge score for mortality prediction (Harrell’s C-index: 0.829 vs. 0.739, p < 0.001). In conclusion, we developed a machine learning model to predict long-term mortality among patients with angina. With the integration of OGTT, the model could help to identify a high risk of mortality across the spectrum of dysglycemia.

1. Introduction

Angina is frequently encountered in patients with ischemic heart diseases [1]. Based on coronary angiography, ischemic heart disease can be classified as obstructive coronary artery disease (CAD) and non-obstructive CAD [1,2]. Obstructive CAD has an established association with increased cardiovascular mortality and is the target of treatment in contemporary practice [3,4]. However, a growing body of evidence has revealed that 50~70% of patients with angina have non-obstructive CAD on coronary angiography [5,6], and they are also associated with a high risk of all-cause mortality and cardiovascular events [7,8]. Since there are two entities of patients with angina, the European Society of Cardiology has published associated guidelines to address differences in their diagnosis and prognosis [9]. However, the optimal management strategy for patients with angina is still under debate, and a risk stratification model to identify high-risk patients and to tailor personal management is warranted [10,11,12].
Current risk stratification models for patients with ischemic heart disease are mainly based on patients with documented obstructive CAD, ranging from stable CAD to acute coronary syndrome [13]; however, a predictive model focused on patients with angina, including both obstructive and non-obstructive CAD, is still lacking. In addition, glucose perturbation represents a well-documented risk factor for atherosclerosis and is commonly seen in patients with ischemic heart disease [14]. A recent survey showed that dysglycemia detected by an oral glucose tolerance test (OGTT) is prevalent in patients with CAD [15]. However, contemporary predictive models for patients with ischemic heart disease seldom integrate glucose indices into their parameters. For example, the Global Registry of Acute Coronary Events (GRACE) discharge score [16], which is a widely used risk score for patients with acute coronary syndromes and has recently had its predictive ability validated for patients with CAD [17], has long been criticized for not including glucose indices [18]. Herein, we aimed to use data collected from patients with angina undergoing coronary angiography and OGTT to build a machine learning model to predict long-term mortality.

2. Materials and Methods

2.1. Setting and Participants

Data were obtained from a prospective, observational study conducted at Taichung Veterans General Hospital. The study enrolled adult patients admitted for coronary angiography between April 2009 and December 2018 due to symptoms of angina and under suspicion of ischemic heart disease by cardiologists. Patients were excluded if they had (a) a history of diabetes before this coronary angiography; (b) a fasting plasma glucose (FPG) ≥ 126 mg/dL during hospitalization; (c) a surgical history of coronary artery bypass graft; (d) acute or chronic infectious diseases; (e) severe systemic diseases, such as malignancies, autoimmune diseases, and psychiatric disorders; (f) an addiction to alcohol or drugs; or (g) pregnancy. All patients underwent OGTT after overnight fasting at an outpatient visit after discharge, and glucose levels were tested at fasting, at 30 min, and at 120 min during the OGTT. Normal glucose regulation was determined as the FPG < 100 mg/dL and the glucose level at 120 min (OGTT 120 min) was <140 mg/dL. Newly diagnosed diabetes was defined as the FPG ≥ 126 mg/dL or the OGTT 120 min ≥ 200 mg/dL. Prediabetes was defined in patients with a glucose regulation between that of normal glucose regulation and diabetes. Obstructive CAD was defined as at least a lesion with ≥50% stenosis, and non-obstructive CAD was defined as a lesion with <50% stenosis in coronary angiography reports. Baseline data collected upon admission, coronary angiography reports, OGTT results, and the medication at the outpatient visit were considered as candidate variables for analysis. Variables with >20% missing values were excluded, and multivariable imputation using the chained equation was applied for the remaining variables [19]. Mortality data up to December 2019 were retrieved from the Collaboration Center of Health Information Application, Department of Health, Executive Yuan, Taiwan, and served as the outcome of interest. The study complied with the Declaration of Helsinki and was approved by the Institutional Review Board of Taichung Veterans General Hospital. Written consent was obtained from each patient before the study procedures were performed (Trial Registration: NCT01198730, ClinicalTrials.gov, last accessed on 7 January 2020).

2.2. Model Development and Evaluation

Model development consisted of two parts. In the first part, we used all available variables to develop prediction models. We chose different machine learning methods, including random survival forest (RSF), gradient boosting machine learning algorithm (XGBoost) for survival analysis, and discrete-time survival model for neural networks [20,21]. RSF is an ensemble tree method for analysis of right censored survival data. It can handle complex interactions among variables, including mixed data types and nonlinear relationships between variables. XGBoost is a novel boosting tree-based ensemble algorithm whose performance is iteratively improved through the optimization of a customized objective function. To handle time-to-event data, the objective function of XGBoost was set as Cox regression. The discrete-time survival model for neural networks was implemented in the Keras deep learning framework and was trained with the maximum likelihood method using minibatch stochastic gradient descent. The likelihood function was used as the loss function, and it naturally incorporated non-proportional hazards. The prediction of these machine learning methods was the hazard ratio of each individual, and their results were compared with those of Cox proportional hazards models.
In the second part, we used features selected by the least absolute shrinkage and selection operator (LASSO)-derived Cox proportional hazards model to construct the predictive model. As the Cox proportional hazards model could not converge and face the risk of overfitting using all the variables those we collected, LASSO was introduced in the Cox model for regularization. LASSO regularization can shrink the estimations and force certain coefficients to zero, thereby keeping only the important features [22]. Significant predictors with p values < 0.05 in the LASSO-derived Cox proportional hazards model were selected as input variables for the machine learning methods mentioned above.
The prediction models were internally validated using a five-fold cross validation. Discrimination ability was assessed using the Harrell’s C-index [23]. Calibration was evaluated using the Brier score [5]. The Brier score measures the mean squared difference between the predicted probability and the actual outcome, with a lower score indicating better calibrated predictions. To explain the contribution of each variable to the best performing model, Shapley values were utilized [24,25]. Based on game theory, Shapley values can explain a model’s prediction by computing the importance of each feature to the prediction.

2.3. Comparison with GRACE Discharge Score

For patients with obstructive CAD confirmed by coronary angiography, we calculated their GRACE discharge score, the components of which included age, heart rate, systolic blood pressure, serum creatinine, the presence of congestive heart failure, cardiac arrest during admission, elevated cardiac enzyme, and ST segment deviation [16]. We compared the performance of GRACE with that of the best performing model. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were used to evaluate the improvement in predictive power of our final model compared with the GRACE discharge score [26]. Analyses were performed using R 3.4 software (The R Project for Statistical Computing, Vienna, Austria) and Python (version 3.6).

3. Results

3.1. Characteristics of Enrolled Patients

A total of 1479 patients were included in the analyses, and 157 patients (10.6%) died during a median follow-up of 6 years (interquartile range, 3.1–9.1 years). Table 1 lists the cohort’s baseline characteristics. Briefly, patients who died tended to be older, with higher heart rates, higher urine albumin-to-creatinine ratios, higher uric acid levels, higher high-density lipoprotein cholesterol levels, higher FPG, and higher OGTT 120 min, whereas their body mass index, diastolic blood pressure, triglycerides, glutamic pyruvic transaminase, hemoglobulin, and estimated glomerular filtration rate were lower compared with patients who survived. Patients who died during follow-up had a lower prevalence of beta-blocker use and a higher prevalence of newly diagnosed diabetes, CAD history before admission, smoking history, and uses of angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers (ARB), alpha blockers, and diuretics compared with patients who survived.

3.2. Feature Importance and Model Performance

The significant predictors selected by the LASSO-derived Cox proportional hazards model included age, heart rate, glucose level at 30 min (OGTT 30 min), OGTT 120 min, CAD history, smoking history, use of ARB, and use of diuretics (Table 2). The performance of the predictive models is shown in Table 3. The RSF model after feature selection had the highest Harrell’s C-index (0.829) and acceptable calibration (Brier score: 0.08). Figure 1 shows the area under the operating curve and the calibration plot of predicted risks at 10 years. Shapley values of the variables in the best performing model are shown in Figure 2.

3.3. Comparison with GRACE Discharge Score

Among patients with obstructive CAD confirmed by angiography during admission, Harrell’s C-index of the best performing model was significantly greater than the GRACE discharge score (0.829 vs. 0.739, respectively; p < 0.001). The NRI (0.328, 95% confidence interval [CI]: 0.096–0.583, p = 0.027) and IDI (0.135, 95% CI: 0.068–0.203, p = 0.007) indices also showed improvement in predictive ability compared with the GRACE discharge score model alone. Predictive ability was also significantly better in the best performing model than in the GRACE discharge score, with OGTT 120 min model. (Table 4).

4. Discussion

In this study, we built a machine learning-based model to predict long term mortality among patients with angina across the spectrum of dysglycemia. With glucose indices obtained from OGTT and other available clinical data, this model showed good discrimination and accuracy in predicting long-term mortality after coronary angiography. To the best of our knowledge, this study is among the first to compare state-of-the-art machine learning methods to predict survival in patients with angina, with an emphasis on OGTT results as important parameters.
For patients with obstructive CAD, several predictive models have been developed to predict major cardiovascular events and mortality. For example, the GRACE discharge score has been recently validated in its accuracy for predicting mortality 2 years after coronary angiography, with an area under the curve of 0.61 for patients with stable CAD [17]. The ABC-CHD score [27], with risk factors identified by the Cox proportional hazards model including age, biomarkers (N-terminal prohormone of brain natriuretic peptide and troponin-T), and clinical histories (smoking, diabetes, and presence of peripheral artery disease), has a good discriminatory ability (Harrell’s C-index: 0.71) and calibration for three-year mortality. However, previous models seldom include patients with non-obstructive CAD. It has been reported that more than half of patients with angina have no obstructive CAD during coronary angiography [28,29,30]. Since a sizable proportion of patients with angina has non-obstructive CAD, our model, which was derived from a cohort in which almost 50% of patients had non-obstructive CAD, is more representative of real-world patients with angina. Even for patients with obstructive CAD, our model outperformed the GRACE discharge score to predict long-term mortality.
The predictors detected by our LASSO-derived Cox proportional hazards models were age, diuretic use, ARB use, heart rate at admission, OGTT 120 min, and OGTT 30 min. Although some are well-established risk factors for mortality and have been included in previous predictive models for patients with CAD, using OGTT results as parameters for risk stratification has not been investigated before. Our study showed high accuracy in mortality prediction with the integration of OGTT results. According to the EUROASPIRE study, OGTT 120 min is a predictor of major cardiovascular events and mortality for patients without diabetes [31]. Chattopadhyay et al. [32] showed that, with the adjustment of OGTT 120 min, the GRACE discharge score has an improved prognostic ability for patients with acute coronary syndrome. Similar to above studies, our model containing OGTT 120 min outperformed the GRACE discharge score for predicting mortality among patients with angina, supporting the importance of OGTT 120 min for mortality prediction among patients with ischemic heart disease. According to our data at baseline, the mean FPG was in the normal range, but OGTT 120 min and glycated hemoglobin (HbA1c) were in the range of prediabetes. Postprandial hyperglycemia has been reported more in Asian patients in comparison to Caucasian patients [33,34]. Moreover, OGTT 120 min is more sensitive than FPG for diagnosing abnormal glucose regulation in patients with CAD [35], and is better than HbA1c and FPG for predicting major cardiovascular events [31].
Our model also revealed potentially unidentified predictors, such as OGTT 30 min. OGTT 30 min has predictive value for developing type 2 diabetes and is associated with inflammatory markers [36,37]; however, its role in mortality prediction has not been previously evaluated. Based on the Shapley value derived from our model, OGTT 30 min also contributed to mortality prediction in patients with angina. Further prospective studies are warranted to elucidate the prognostic value of OGTT 30 min. It is notable that several traditional cardiovascular risks were not selected for our predictive model. In this cohort which included patients with angina, all of the participants had undergone hospitalization where adequate medication and education were performed. Therefore, controlled cholesterol and blood pressure was observed in outpatient visits, and the attenuated contribution to traditional risk factors was close to that in real-world practice [38]. Moreover, the median duration between coronary angiography and outpatient visit was 13 days (interquartile range: 9–17 days) in the present study. Therefore, the low-density lipoprotein cholesterol might be not adequately decreased after a short-term treatment of statins.
There are several clinical applications of this model. Our model is derived from patients with angina. More than half of patients with angina have non-obstructive CAD, and approximately 20~40% of patients with CAD still suffer from angina symptoms after revascularization [4]. There is heterogeneity in prognosis among patients with angina, and it is important to stratify their risk and tailor their management strategy. However, contemporary clinical practice mainly focuses on the prevention and management of obstructive CAD [39], despite the fact that the risk of major cardiovascular events and mortality among patients with non-obstructive CAD is increased [40]. Our model could help to identify patients with angina at a high risk of mortality. In addition, our model emphasized the importance of OGTT. Screening for dysglycemia using OGTT in patients undergoing percutaneous coronary intervention has long been proposed and is also recommended in European Society of Cardiology guidelines [40,41]. However, adhesion to this recommendation is poor [35], partially because the prognostic role of OGTT is less clear. Although early intervention using anti-diabetic drugs did not significantly reduce the risk of a major adverse cardiovascular event in the Acarbose Cardiovascular Evaluation trial [42], other intensive treatment, e.g., high-intensity statins, is highly recommended for patients with high-event risk [43]. Our model, which adds prognostic value to the OGTT for patients with angina, could increase adherence to this recommendation.
The major strength of this study is that we used the LASSO-derived Cox proportional hazards model for feature selection and advanced machine learning methods for model development. Only six variables were needed in our model after utilization of LASSO regularization, and most of them, except OGTT, were available from electronic health records, making it a convenient tool to implement in clinical practice. The best performing machine method in our study was RSF. Previous predictive models for CAD were usually built using Cox proportional hazards models; however, several assumptions must be met before applying the Cox proportional hazards model. Conversely, machine learning methods, such as RSF, can handle non-linear, complex relationships between features without assumptions, thus widening their clinical application. However, there are still some limitations which should be highlighted in the current study. First, coronary angiography was arranged by cardiologists according to clinical judgement, including inconclusive non-invasive tests, unresponsive to medical therapies, or risks of high event. However, we did not examine the microvascular dysfunction using invasive stress tests in the enrolled patients without significant coronary obstruction [9]. Second, we did not follow the changes in medical treatment after baseline assessments. Third, this cohort has been active since 2009, and contemporary anti-diabetic medications, such as sodium glucose co-transporters 2 inhibitors and glucagon-like peptide-1 receptor agonists, which can reduce mortality risk in patients with type 2 diabetes, were seldom prescribed. Finally, our model has not been externally validated with other independent datasets in the present study, so its performance in other datasets is unknown. Further external validation using an independent dataset is scheduled.

5. Conclusions

We developed a machine learning model containing OGTT results and other clinically available parameters to predict all-cause mortality among patients with angina. This model could help to identify patients at a high risk of mortality. With the integration of glucose indices from OGTT, patients with dysglycemia could be identified early, enabling their risk of mortality to be accurately evaluated.

Author Contributions

Conceptualization, Y.-H.L., W.H.-H.S. and I.-T.L.; data curation, I.-T.L.; formal analysis, Y.-H.L. and Y.-C.C.; funding acquisition, W.H.-H.S. and I.-T.L.; investigation, Y.-H.L. and I.-T.L.; methodology, Y.-H.L., W.-C.Y., Y.-C.C. and I.-T.L.; project administration, W.H.-H.S. and I.-T.L.; resources, W.H.-H.S. and I.-T.L.; software, Y.-H.L., W.-C.Y. and Y.-C.C.; supervision, Y.-C.C. and I.-T.L.; validation, Y.-C.C. and I.-T.L.; visualization, Y.-H.L. and Y.-C.C.; writing—original draft, Y.-H.L.; writing—review and editing, I.-T.L.. All authors performed a critical revision of the manuscript for important intellectual content. I.-T.L. is the guarantor of this work and takes full responsibility for the work as a whole, including the study design, access to data, and the decision to submit the manuscript for publication. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Taichung Veterans General Hospital, Taichung, Taiwan (grant number TCVGH-1103502D and TCVGH-1103501B), Ministry of Science and Technology, Taiwan (grant number MOST109–2314-B-075A-004), and National Health Research Institute, Taiwan (grant number NHRI-EX109-10927HT and NHRI-EX110-10927HT).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of Taichung Veterans General Hospital (protocol code: C08215 and date of approval: 9 March 2009).

Informed Consent Statement

Informed consent was obtained from all the subjects involved in the study.

Data Availability Statement

Data available on request due to privacy.

Acknowledgments

The authors acknowledge the Cardiovascular Center of Taichung Veterans General Hospital for its support.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; the collection, analyses, or interpretation of data; the writing of the manuscript, or the decision to publish the results.

References

  1. Ford, T.J.; Berry, C. Angina: Contemporary diagnosis and management. Heart 2020, 106, 387–398. [Google Scholar] [CrossRef]
  2. Ferrari, R.; Camici, P.G.; Crea, F.; Danchin, N.; Fox, K.; Maggioni, A.P.; Manolis, A.J.; Marzilli, M.; Rosano, G.M.C.; Lopez-Sendon, J.L. Expert consensus document: A ‘diamond’ approach to personalized treatment of angina. Nat. Rev. Cardiol. 2018, 15, 120–132. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Sundaram, V.; Bloom, C.; Zakeri, R.; Halcox, J.; Cohen, A.; Bowrin, K.; Briere, J.B.; Banerjee, A.; Simon, D.I.; Cleland, J.G.F.; et al. Temporal trends in the incidence, treatment patterns, and outcomes of coronary artery disease and peripheral artery disease in the UK, 2006–2015. Eur. Heart J. 2020, 41, 1636–1649. [Google Scholar] [CrossRef] [PubMed]
  4. Marzilli, M.; Crea, F.; Morrone, D.; Bonow, R.O.; Brown, D.L.; Camici, P.G.; Chilian, W.M.; DeMaria, A.; Guarini, G.; Huqi, A.; et al. Myocardial ischemia: From disease to syndrome. Int. J. Cardiol. 2020, 314, 32–35. [Google Scholar] [CrossRef]
  5. Brier, G.W. Verification of Forecasts Expressed in Terms of Probability. Mon. Weather. Rev. 1950, 78, 1–3. [Google Scholar] [CrossRef]
  6. Pepine, C.J.; Anderson, R.D.; Sharaf, B.L.; Reis, S.E.; Smith, K.M.; Handberg, E.M.; Johnson, B.D.; Sopko, G.; Bairey Merz, C.N. Coronary microvascular reactivity to adenosine predicts adverse outcome in women evaluated for suspected ischemia results from the National Heart, Lung and Blood Institute WISE (Women’s Ischemia Syndrome Evaluation) study. J. Am. Coll. Cardiol. 2010, 55, 2825–2832. [Google Scholar] [CrossRef] [Green Version]
  7. Jespersen, L.; Hvelplund, A.; Abildstrøm, S.Z.; Pedersen, F.; Galatius, S.; Madsen, J.K.; Jørgensen, E.; Kelbæk, H.; Prescott, E. Stable angina pectoris with no obstructive coronary artery disease is associated with increased risks of major adverse cardiovascular events. Eur. Heart J. 2012, 33, 734–744. [Google Scholar] [CrossRef]
  8. Schulman-Marcus, J.; Hartaigh, B.; Gransar, H.; Lin, F.; Valenti, V.; Cho, I.; Berman, D.; Callister, T.; DeLago, A.; Hadamitzky, M.; et al. Sex-Specific Associations Between Coronary Artery Plaque Extent and Risk of Major Adverse Cardiovascular Events: The CONFIRM Long-Term Registry. JACC Cardiovasc. Imaging 2016, 9, 364–372. [Google Scholar] [CrossRef] [Green Version]
  9. Knuuti, J.; Wijns, W.; Saraste, A.; Capodanno, D.; Barbato, E.; Funck-Brentano, C.; Prescott, E.; Storey, R.F.; Deaton, C.; Cuisset, T.; et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes: The Task Force for the diagnosis and management of chronic coronary syndromes of the European Society of Cardiology (ESC). Eur. Heart J. 2019, 41, 407–477. [Google Scholar] [CrossRef]
  10. Maron, D.J.; Hochman, J.S.; Reynolds, H.R.; Bangalore, S.; O’Brien, S.M.; Boden, W.E.; Chaitman, B.R.; Senior, R.; López-Sendón, J.; Alexander, K.P.; et al. Initial Invasive or Conservative Strategy for Stable Coronary Disease. N. Engl. J. Med. 2020, 382, 1395–1407. [Google Scholar] [CrossRef] [PubMed]
  11. Kunadian, V.; Chieffo, A.; Camici, P.G.; Berry, C.; Escaned, J.; Maas, A.; Prescott, E.; Karam, N.; Appelman, Y.; Fraccaro, C.; et al. An EAPCI Expert Consensus Document on Ischaemia with Non-Obstructive Coronary Arteries in Collaboration with European Society of Cardiology Working Group on Coronary Pathophysiology & Microcirculation Endorsed by Coronary Vasomotor Disorders International Study Group. Eur. Heart J. 2020, 41, 3504–3520. [Google Scholar]
  12. Dweck, M.R.; Newby, D.E. Non-obstructive coronary artery disease can no longer be ignored. Eur. Heart J. Cardiovasc. Imaging 2020, 21, 489–490. [Google Scholar] [CrossRef] [PubMed]
  13. Buccheri, S.; D’Arrigo, P.; Franchina, G.; Capodanno, D. Risk Stratification in Patients with Coronary Artery Disease: A Practical Walkthrough in the Landscape of Prognostic Risk Models. Interv. Cardiol. 2018, 13, 112–120. [Google Scholar] [CrossRef]
  14. Kuhl, J.; Jorneskog, G.; Wemminger, M.; Bengtsson, M.; Lundman, P.; Kalani, M. Long-term clinical outcome in patients with acute coronary syndrome and dysglycaemia. Cardiovasc. Diabetol. 2015, 14, 120. [Google Scholar] [CrossRef] [Green Version]
  15. Ferrannini, G.; De Bacquer, D.; De Backer, G.; Kotseva, K.; Mellbin, L.; Wood, D.; Rydén, L. Screening for Glucose Perturbations and Risk Factor Management in Dysglycemic Patients with Coronary Artery Disease-A Persistent Challenge in Need of Substantial Improvement: A Report from ESC EORP EUROASPIRE V. Diabetes Care 2020, 43, 726–733. [Google Scholar] [CrossRef] [Green Version]
  16. Shuvy, M.; Beeri, G.; Klein, E.; Cohen, T.; Shlomo, N.; Minha, S.; Pereg, D. Accuracy of the Global Registry of Acute Coronary Events (GRACE) Risk Score in Contemporary Treatment of Patients with Acute Coronary Syndrome. Can. J. Cardiol. 2018, 34, 1613–1617. [Google Scholar] [CrossRef] [PubMed]
  17. Zhao, X.; Li, J.; Xian, Y.; Chen, J.; Gao, Z.; Qiao, S.; Yang, Y.; Gao, R.; Xu, B.; Yuan, J. Prognostic value of the GRACE discharge score for predicting the mortality of patients with stable coronary artery disease who underwent percutaneous coronary intervention. Catheter. Cardiovasc. Interv. 2020, 95, 550–557. [Google Scholar] [CrossRef] [PubMed]
  18. Liu, X.J.; Wan, Z.F.; Zhao, N.; Zhang, Y.P.; Mi, L.; Wang, X.H.; Zhou, D.; Wu, Y.; Yuan, Z.Y. Adjustment of the GRACE score by HemoglobinA1c enables a more accurate prediction of long-term major adverse cardiac events in acute coronary syndrome without diabetes undergoing percutaneous coronary intervention. Cardiovasc. Diabetol. 2015, 14, 110. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Azur, M.J.; Stuart, E.A.; Frangakis, C.; Leaf, P.J. Multiple imputation by chained equations: What is it and how does it work? Int. J. Methods Psychiatr. Res. 2011, 20, 40–49. [Google Scholar] [CrossRef]
  20. Chen, T.; Guestrin, C. In Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; Volume 2016, pp. 785–794. [Google Scholar]
  21. Gensheimer, M.F.; Narasimhan, B. A Scalable Discrete-Time Survival Model for Neural Networks. PeerJ 2019, 7, e6257. [Google Scholar] [CrossRef] [PubMed]
  22. Teppola, P.; Taavitsainen, V.M. Parsimonious and robust multivariate calibration with rational function Least Absolute Shrinkage and Selection Operator and rational function Elastic Net. Anal. Chim. Acta 2013, 768, 57–68. [Google Scholar] [CrossRef] [PubMed]
  23. Uno, H.; Cai, T.; Pencina, M.J.; D’Agostino, R.B.; Wei, L.J. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat. Med. 2011, 30, 1105–1117. [Google Scholar] [CrossRef] [Green Version]
  24. Lundberg, S.M.; Erion, G.; Chen, H.; DeGrave, A.; Prutkin, J.M.; Nair, B.; Katz, R.; Himmelfarb, J.; Bansal, N.; Lee, S.-I. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2020, 2, 56–67. [Google Scholar] [CrossRef] [PubMed]
  25. Lundberg, S.M.; Lee, S.-I. A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems; Long Beach, CA, USA, 2017; Volume 2017, pp. 4765–4774. Available online: https://papers.nips.cc/paper/2017 (accessed on 19 April 2021).
  26. Pencina, M.J.; D’Agostino, R.B., Sr.; Steyerberg, E.W. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat. Med. 2011, 30, 11–21. [Google Scholar] [CrossRef]
  27. Lindholm, D.; Lindbäck, J.; Armstrong, P.W.; Budaj, A.; Cannon, C.P.; Granger, C.B.; Hagström, E.; Held, C.; Koenig, W.; Östlund, O.; et al. Biomarker-Based Risk Model to Predict Cardiovascular Mortality in Patients with Stable Coronary Disease. J. Am. Coll. Cardiol. 2017, 70, 813–826. [Google Scholar] [CrossRef]
  28. Arnold, S.V.; Grodzinsky, A.; Gosch, K.L.; Kosiborod, M.; Jones, P.G.; Breeding, T.; Towheed, A.; Beltrame, J.; Alexander, K.P.; Spertus, J.A. Predictors of Physician Under-Recognition of Angina in Outpatients with Stable Coronary Artery Disease. Circ. Cardiovasc. Qual Outcomes 2016, 9, 554–559. [Google Scholar] [CrossRef] [Green Version]
  29. Lanza, G.A.; Crea, F. Primary coronary microvascular dysfunction: Clinical presentation, pathophysiology, and management. Circulation 2010, 121, 2317–2325. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Pepine, C.J.; Ferdinand, K.C.; Shaw, L.J.; Light-McGroary, K.A.; Shah, R.U.; Gulati, M.; Duvernoy, C.; Walsh, M.N.; Bairey Merz, C.N. Emergence of Nonobstructive Coronary Artery Disease: A Woman’s Problem and Need for Change in Definition on Angiography. J. Am. Coll. Cardiol. 2015, 66, 1918–1933. [Google Scholar] [CrossRef]
  31. Shahim, B.; De Bacquer, D.; De Backer, G.; Gyberg, V.; Kotseva, K.; Mellbin, L.; Schnell, O.; Tuomilehto, J.; Wood, D.; Rydén, L. The Prognostic Value of Fasting Plasma Glucose, Two-Hour Postload Glucose, and HbA(1c) in Patients with Coronary Artery Disease: A Report from EUROASPIRE IV: A Survey From the European Society of Cardiology. Diabetes Care 2017, 40, 1233–1240. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Chattopadhyay, S.; George, A.; John, J.; Sathyapalan, T. Adjustment of the GRACE score by 2-h post-load glucose improves prediction of long-term major adverse cardiac events in acute coronary syndrome in patients without known diabetes. Eur. Heart J. 2018, 39, 2740–2745. [Google Scholar] [CrossRef] [Green Version]
  33. Wang, J.S.; Tu, S.T.; Lee, I.T.; Lin, S.D.; Lin, S.Y.; Su, S.L.; Lee, W.J.; Sheu, W.H. Contribution of postprandial glucose to excess hyperglycaemia in Asian type 2 diabetic patients using continuous glucose monitoring. Diabetes Metab. Res. Rev. 2011, 27, 79–84. [Google Scholar] [CrossRef]
  34. Henry, C.J.; Lightowler, H.J.; Newens, K.; Sudha, V.; Radhika, G.; Sathya, R.M.; Mohan, V. Glycaemic index of common foods tested in the UK and India. Br. J. Nutr. 2008, 99, 840–845. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Hu, D.Y.; Pan, C.Y.; Yu, J.M.; China Heart Survey Group. The relationship between coronary artery disease and abnormal glucose regulation in China: The China Heart Survey. Eur. Heart J. 2006, 27, 2573–2579. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Chen, G.; Zou, X.; Yao, J.; Jiang, Q.; Zhang, Y.; Tu, M.; Yang, S.; Xu, S.; Lin, W.; Huang, H.; et al. The correlation between the oral glucose tolerance test 30-minutes plasma glucose and risk factors for diabetes and cardiovascular diseases: A cross-sectional epidemiological study of diabetes in Fujian Province in the South-East of China. J. Endocrinol. Investig. 2011, 34, e115–e120. [Google Scholar] [CrossRef] [PubMed]
  37. Tsuboi, A.; Minato, S.; Yano, M.; Takeuchi, M.; Kitaoka, K.; Kurata, M.; Yoshino, G.; Wu, B.; Kazumi, T.; Fukuo, K. Association of serum orosomucoid with 30-min plasma glucose and glucose excursion during oral glucose tolerance tests in non-obese young Japanese women. BMJ Open Diabetes Res. Care 2018, 6, e000508. [Google Scholar] [CrossRef] [Green Version]
  38. Alagona, P., Jr.; Ahmad, T.A. Cardiovascular disease risk assessment and prevention: Current guidelines and limitations. Med. Clin. N. Am. 2015, 99, 711–731. [Google Scholar] [CrossRef] [PubMed]
  39. Taqueti, V.R.; Everett, B.M.; Murthy, V.L.; Gaber, M.; Foster, C.R.; Hainer, J.; Blankstein, R.; Dorbala, S.; Di Carli, M.F. Interaction of impaired coronary flow reserve and cardiomyocyte injury on adverse cardiovascular outcomes in patients without overt coronary artery disease. Circle 2015, 131, 528–535. [Google Scholar] [CrossRef] [Green Version]
  40. Shahim, B.; Gyberg, V.; De Bacquer, D.; Kotseva, K.; De Backer, G.; Schnell, O.; Tuomilehto, J.; Wood, D.; Rydén, L. Undetected dysglycaemia common in primary care patients treated for hypertension and/or dyslipidaemia: On the need for a screening strategy in clinical practice. A report from EUROASPIRE IV a registry from the EuroObservational Research Programme of the European Society of Cardiology. Cardiovasc. Diabetol. 2018, 17, 21. [Google Scholar]
  41. Grant, P.J.; Cosentino, F. The 2019 ESC Guidelines on diabetes, pre-diabetes, and cardiovascular diseases developed in collaboration with the EASD: New features and the ‘Ten Commandments’ of the 2019 Guidelines are discussed by Professor Peter J. Grant and Professor Francesco Cosentino, the Task Force chairmen. Eur. Heart J. 2019, 40, 3215–3217. [Google Scholar] [PubMed]
  42. Holman, R.R.; Coleman, R.L.; Chan, J.C.N.; Chiasson, J.L.; Feng, H.; Ge, J.; Gerstein, H.C.; Gray, R.; Huo, Y.; Lang, Z.; et al. Effects of acarbose on cardiovascular and diabetes outcomes in patients with coronary heart disease and impaired glucose tolerance (ACE): A randomised, double-blind, placebo-controlled trial. Lancet Diabetes Endocrinol. 2017, 5, 877–886. [Google Scholar] [CrossRef] [Green Version]
  43. Grundy, S.M.; Stone, N.J.; Bailey, A.L.; Beam, C.; Birtcher, K.K.; Blumenthal, R.S.; Braun, L.T.; de Ferranti, S.; Faiella-Tommasino, J.; Forman, D.E.; et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the Management of Blood Cholesterol: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circle 2019, 139, e1082–e1143. [Google Scholar]
Diagnostics 11 01060 g001 550
Figure 1. Area under the curve of the best performing model for predicting all-cause mortality among patients with angina (a). The closer to the dotted line indicates the better prediction in the calibration plot of the best performing model (b).
Figure 1. Area under the curve of the best performing model for predicting all-cause mortality among patients with angina (a). The closer to the dotted line indicates the better prediction in the calibration plot of the best performing model (b).
Diagnostics 11 01060 g001
Diagnostics 11 01060 g002 550
Figure 2. Impact of selected features on model prediction. The impact on prediction was based on the SHAP value, which was calculated from the best performing model. (ARB = angiotensin II receptor blocker, CAD = coronary artery disease, OGTT = oral glucose tolerance test, SHAP = Shapley Additive Explanation values).
Figure 2. Impact of selected features on model prediction. The impact on prediction was based on the SHAP value, which was calculated from the best performing model. (ARB = angiotensin II receptor blocker, CAD = coronary artery disease, OGTT = oral glucose tolerance test, SHAP = Shapley Additive Explanation values).
Diagnostics 11 01060 g002
Table
Table 1. Baseline characteristics of the study population.
Table 1. Baseline characteristics of the study population.
Survival
(N = 1322)		Death
(N = 157)		p
Age (years)58 ± 1072 ± 12<0.001
Female (n, %)238 (18%)20 (13%)0.122
Body height (cm)165 ± 8162 ± 8<0.001
Body weight (kg)71.3 ± 11.766.4 ± 12.2<0.001
BMI (kg/m2)26.1 ± 3.425.2 ± 3.90.004
Waist (cm)90.6 ± 890.5 ± 9.40.923
SBP (mmHg)126 ± 17128 ± 200.226
DBP (mmHg)74 ± 1069 ± 11<0.001
Heart rate (beat/min)71 ± 1175 ± 13<0.001
UACR (mg/g)36 ± 176111 ± 274<0.001
Uric acid (mg/dL)6.6 ± 1.67.3 ± 2.0<0.001
Triglycerides (mg/dL)147 ± 105122 ± 780.004
Total cholesterol (mg/dL)173 ± 39169 ± 350.253
HDL cholesterol (mg/dL)46 ± 1249 ± 150.007
LDL cholesterol (mg/dL)106 ± 34103 ± 310.501
Creatinine (mg/dL)1.06 ± 1.011.01 ±0.570.298
eGFR (mL/min/1.73 m2)82 ± 2263 ± 21<0.001
GPT (U/L)32 ± 3226 ± 190.028
Hemoglobin (g/dL)13.9 ± 1.513.2 ± 1.90.001
WBC (/μL)6898 ± 27726923 ± 21760.914
CK (U/L)151 ± 314180 ± 3700.373
CKMB (U/L)9 ± 1611 ± 160.241
Troponin-T (ng/L)3.9 ± 19.51.4 ± 3.60.209
OGTT (mg/dL)
   Glucose 0 min95 ± 14100 ± 19<0.001
   Glucose 30 min169 ± 32169 ± 360.962
   Glucose 120 min145 ± 50166 ± 59<0.001
HbA1c (%)5.8 ± 0.66.1 ± 0.8<0.001
Glucose status (n, %) <0.001
   Normal glucose regulation428 (32.4%)43 (27.4%)
   Prediabetes637 (48.2%)51 (32.5%)
   Diabetes257 (19.4%)63 (40.1%)
Smoking status (n, %) <0.001
   Non-smoker590 (44.6%)52 (33.1%)
   Smoker318 (24.1%)18 (11.5%)
   Ex-smoker414 (31.3%)87 (55.4%)
Medication (n, %)
   Antiplatelet1226 (92.8%)147 (93.6%)0.830
   ACE inhibitor278 (21.0%)46 (29.3%)0.023
   ARB415 (31.4%)69 (43.9%)0.002
   Alpha blocker54 (4.1%)16 (10.2%)0.001
   Beta blocker368 (27.8%)23 (14.6%)0.001
   CCB689 (52.1%)78 (49.7%)0.622
   Diuretics153 (11.6%)44 (28.0%)<0.001
CAD history * (n, %)140 (10.6%)26 (16.6%)0.035
Grace score90.2±20.3125.3 ± 31<0.001
Left ventricular ejection fraction (%)52 ± 1147 ± 13<0.001
Number of coronary arteries with significant stenosis † (n, %)0.007
   Non-obstructive CAD611 (46.2%)51 (32.5%)
   1356 (26.9%)48 (30.6%)
   2245 (18.5%)42 (26.8%)
   3110 (8.3%)16 (10.2%)
Non-invasive studies before angiography (n, %)
   Treadmill exercise test575 (43.5%)66 (42.0%)0.793
   Myocardial perfusion imaging116 (8.8%)16 (10.2%)0.660
   Echocardiography282 (21.3%)30 (19.1%)0.588
   Rest electrocardiography349 (26.4%)45 (28.7%)0.609
Percutaneous coronary intervention (n, %)
   without stent insertion146 (11.0%)36 (22.9%)<0.001
   with stent insertion542 (41.0%)67 (42.7%)0.640
* CAD history refers to percutaneous coronary intervention for obstructive CAD before admission; † Significant stenosis defined as stenosis ≥ 50%. ACE = angiotensin-converting enzyme, ARB = angiotensin II receptor blocker, BMI = body mass index, BP = blood pressure, eGFR = estimated glomerular filtration rate, FPG = fasting plasma glucose, HDL = high-density lipoprotein, LDL = low-density lipoprotein, OGTT = oral glucose tolerance test, SBP = systolic blood pressure, DBP = diastolic blood pressure, CCB = calcium channel blocker, CAD = coronary artery disease, WBC = white blood cell count, CK = creatine kinase, GPT = glutamic pyruvic transaminase, HbA1c = glycated hemoglobin, UACR = albumin-to-creatinine ratio.
Table
Table 2. Features selected by the LASSO-derived Cox proportional hazards model.
Table 2. Features selected by the LASSO-derived Cox proportional hazards model.
HR95% CIp
Age1.06(1.04–0.08)<0.001
Heart rate1.02(1.01–1.04)<0.001
OGTT 30 min0.98(0.97–0.99)<0.001
OGTT 120 min1.01(1.01–1.02)<0.001
CAD history1.66(1.03–2.66)0.040
Smoking history1.31(1.06–1.65)0.013
ARB use1.74(1.18–2.55)0.008
Diuretic use1.57(1.01–2.38)0.048
ARB = angiotensin II receptor blocker, CAD = coronary artery disease, CI = confidence interval, HR = hazard ratio, LASSO = least absolute shrinkage and selection operator, OGTT = oral glucose tolerance test.
Table
Table 3. Performance of different machine learning methods.
Table 3. Performance of different machine learning methods.
Harrell’s C-IndexBrier Score
With all variables
   Cox regression0.7740.069
   RSF0.8040.082
   XGBoost0.7880.033
   DNN0.7500.102
With selected features
   Cox regression0.7410.076
   RSF0.8290.080
   XGBoost0.7940.056
   DNN0.7960.106
RSF = random survival forest, XGBoost = extreme gradient boosting, DNN = dense neural network.
Table
Table 4. GRACE-score-based models compared to the best-performing model.
Table 4. GRACE-score-based models compared to the best-performing model.
ModelHarrell’s C-Index
(95% CI)		pAbsolute IDI
(95% CI)		pNRI
(95% CI)		p
GRACE score0.739
(0.683, 0.796)		<0.0010.135
(0.068, 0.203)		0.0070.328
(0.096, 0.583)		0.027
GRACE score + OGTT 120 min0.740
(0.685, 0.797)		<0.0010.115
(0.033, 0.224)		0.0270.336
(0.103, 0.646)		0.027
RSF0.829
(0.790, 0.869)
CI = confidence interval; GRACE = Global Registry of Acute Coronary Events; IDI = integrated discrimination improvement; NRI = net reclassification improvement; OGTT = oral glucose tolerance test; RSF = random survival forest with selected features.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Li, Y.-H.; Sheu, W.H.-H.; Yeh, W.-C.; Chang, Y.-C.; Lee, I.-T. Predicting Long-Term Mortality in Patients with Angina across the Spectrum of Dysglycemia: A Machine Learning Approach. Diagnostics 2021, 11, 1060. https://doi.org/10.3390/diagnostics11061060

AMA Style

Li Y-H, Sheu WH-H, Yeh W-C, Chang Y-C, Lee I-T. Predicting Long-Term Mortality in Patients with Angina across the Spectrum of Dysglycemia: A Machine Learning Approach. Diagnostics. 2021; 11(6):1060. https://doi.org/10.3390/diagnostics11061060

Chicago/Turabian Style

Li, Yu-Hsuan, Wayne Huey-Herng Sheu, Wen-Chao Yeh, Yung-Chun Chang, and I-Te Lee. 2021. "Predicting Long-Term Mortality in Patients with Angina across the Spectrum of Dysglycemia: A Machine Learning Approach" Diagnostics 11, no. 6: 1060. https://doi.org/10.3390/diagnostics11061060

APA Style

Li, Y. -H., Sheu, W. H. -H., Yeh, W. -C., Chang, Y. -C., & Lee, I. -T. (2021). Predicting Long-Term Mortality in Patients with Angina across the Spectrum of Dysglycemia: A Machine Learning Approach. Diagnostics, 11(6), 1060. https://doi.org/10.3390/diagnostics11061060

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop