Renin Angiotensin System Blockers and Risk of Mortality in Hypertensive Patients Hospitalized for COVID-19: An Italian Registry
Abstract
:1. Introduction
2. Materials and Methods
3. Results
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Gressens, S.B.; Leftheriotis, G.; Dussaule, J.C.; Flamant, M.; Levy, B.I.; Vidal-Petiot, E. Controversial Roles of the Renin Angiotensin System and Its Modulators during the COVID-19 Pandemic. Front. Physiol. 2021, 12, 624052. [Google Scholar] [CrossRef]
- Angeli, F.; Zappa, M.; Reboldi, G.; Trapasso, M.; Cavallini, C.; Spanevello, A.; Verdecchia, P. The pivotal link between ACE2 deficiency and SARS-CoV-2 infection: One year later. Eur. J. Intern. Med. 2021, 93, 28–34. [Google Scholar] [CrossRef]
- Verdecchia, P.; Cavallini, C.; Spanevello, A.; Angeli, F. The pivotal link between ACE2 deficiency and SARS-CoV-2 infection. Eur. J. Intern. Med. 2020, 76, 14–20. [Google Scholar] [CrossRef] [PubMed]
- Verdecchia, P.; Cavallini, C.; Spanevello, A.; Angeli, F. COVID-19: ACE2centric Infective Disease? Hypertension 2020, 76, 294–299. [Google Scholar] [CrossRef] [PubMed]
- Angeli, F.; Verdecchia, P.; Reboldi, G. Pharmacotherapy for hypertensive urgency and emergency in COVID-19 patients. Expert Opin. Pharmacother. 2021, 1–8. [Google Scholar] [CrossRef] [PubMed]
- Mehra, M.R.; Desai, S.S.; Kuy, S.; Henry, T.D.; Patel, A.N. Cardiovascular Disease, Drug Therapy, and Mortality in COVID-19. N. Engl. J. Med. 2020, 382, e102. [Google Scholar] [CrossRef] [PubMed]
- Zhang, P.; Zhu, L.; Cai, J.; Lei, F.; Qin, J.J.; Xie, J.; Liu, Y.M.; Zhao, Y.C.; Huang, X.; Lin, L.; et al. Association of Inpatient Use of Angiotensin-Converting Enzyme Inhibitors and Angiotensin II Receptor Blockers With Mortality Among Patients With Hypertension Hospitalized With COVID-19. Circ. Res. 2020, 126, 1671–1681. [Google Scholar] [CrossRef] [PubMed]
- Angeli, F.; Bachetti, T.; Maugeri Study Group. Temporal changes in co-morbidities and mortality in patients hospitalized for COVID-19 in Italy. Eur. J. Intern. Med. 2020, 82, 123–125. [Google Scholar] [CrossRef]
- Angeli, F.; Marazzato, J.; Verdecchia, P.; Balestrino, A.; Bruschi, C.; Ceriana, P.; Chiovato, L.; Dalla Vecchia, L.A.; De Ponti, R.; Fanfulla, F.; et al. Joint effect of heart failure and coronary artery disease on the risk of death during hospitalization for COVID-19. Eur. J. Intern. Med. 2021, 89, 81–86. [Google Scholar] [CrossRef]
- Li, T. Diagnosis and clinical management of severe acute respiratory syndrome Coronavirus 2 (SARS-CoV-2) infection: An operational recommendation of Peking Union Medical College Hospital (V2.0). Emerg. Microbes Infect. 2020, 9, 582–585. [Google Scholar] [CrossRef] [Green Version]
- Cosentino, F.; Grant, P.J.; Aboyans, V.; Bailey, C.J.; Ceriello, A.; Delgado, V.; Federici, M.; Filippatos, G.; Grobbee, D.E.; Hansen, T.B.; et al. 2019 ESC Guidelines on diabetes, pre-diabetes, and cardiovascular diseases developed in collaboration with the EASD. Eur. Heart J. 2020, 41, 255–323. [Google Scholar] [CrossRef] [Green Version]
- Knuuti, J.; Wijns, W.; Saraste, A.; Capodanno, D.; Barbato, E.; Funck-Brentano, C.; Prescott, E.; Storey, R.F.; Deaton, C.; Cuisset, T.; et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes. Eur. Heart J. 2020, 41, 407–477. [Google Scholar] [CrossRef]
- Piepoli, M.F.; Hoes, A.W.; Agewall, S.; Albus, C.; Brotons, C.; Catapano, A.L.; Cooney, M.T.; Corra, U.; Cosyns, B.; Deaton, C.; et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts)Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur. Heart J. 2016, 37, 2315–2381. [Google Scholar]
- Verdecchia, P.; Reboldi, G.; Angeli, F. The 2020 International Society of Hypertension global hypertension practice guidelines-key messages and clinical considerations. Eur. J. Intern. Med. 2020, 82, 1–6. [Google Scholar] [CrossRef]
- Zhou, F.; Yu, T.; Du, R.; Fan, G.; Liu, Y.; Liu, Z.; Xiang, J.; Wang, Y.; Song, B.; Gu, X.; et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 2020, 395, 1054–1062. [Google Scholar] [CrossRef]
- Iaccarino, G.; Grassi, G.; Borghi, C.; Ferri, C.; Salvetti, M.; Volpe, M. Investigators, Age and Multimorbidity Predict Death among COVID-19 Patients: Results of the SARS-RAS Study of the Italian Society of Hypertension. Hypertension 2020, 76, 366–372. [Google Scholar] [CrossRef] [PubMed]
- Izcovich, A.; Ragusa, M.A.; Tortosa, F.; Lavena Marzio, M.A.; Agnoletti, C.; Bengolea, A.; Ceirano, A.; Espinosa, F.; Saavedra, E.; Sanguine, V.; et al. Prognostic factors for severity and mortality in patients infected with COVID-19: A systematic review. PLoS ONE 2020, 15, e0241955. [Google Scholar] [CrossRef]
- Peng, Y.; Meng, K.; He, M.; Zhu, R.; Guan, H.; Ke, Z.; Leng, L.; Wang, X.; Liu, B.; Hu, C.; et al. Clinical Characteristics and Prognosis of 244 Cardiovascular Patients Suffering From Coronavirus Disease in Wuhan, China. J. Am. Heart Assoc. 2020, 9, e016796. [Google Scholar] [CrossRef] [PubMed]
- Centers for Disease Control and Prevention. Science Brief: Evidence Used to Update the List of Underlying Medical Conditions that Increase a Person’s Risk of Severe Illness from COVID-19. Available online: https://www.cdc.gov/coronavirus/2019-ncov/science/science-briefs/underlying-evidence-table.html (accessed on 15 May 2021).
- Taneri, P.E.; Gómez-Ochoa, S.A.; Llanaj, E.; Raguindin, P.F.; Rojas, L.Z.; Roa-Díaz, Z.M.; Salvador, D.; Groothof, D.; Minder, B.; Kopp-Heim, D.; et al. Anemia and iron metabolism in COVID-19: A systematic review and meta-analysis. Eur. J. Epidemiol. 2020, 35, 763–773. [Google Scholar] [CrossRef] [PubMed]
- Baral, R.; Tsampasian, V.; Debski, M.; Moran, B.; Garg, P.; Clark, A.; Vassiliou, V.S. Association Between Renin-Angiotensin-Aldosterone System Inhibitors and Clinical Outcomes in Patients With COVID-19: A Systematic Review and Meta-analysis. JAMA Netw. Open 2021, 4, e213594. [Google Scholar] [CrossRef] [PubMed]
- Bavishi, C.; Maddox, T.M.; Messerli, F.H. Coronavirus Disease 2019 (COVID-19) Infection and Renin Angiotensin System Blockers. JAMA Cardiol. 2020, 5, 745–747. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Conversano, A.; Melillo, F.; Napolano, A.; Fominskiy, E.; Spessot, M.; Ciceri, F.; Agricola, E. Renin-Angiotensin-Aldosterone System inhibitors and outcome in patients with SARS-CoV-2 pneumonia. A case series study. Hypertension 2020, 76, e10–e12. [Google Scholar] [CrossRef] [PubMed]
- Danser, A.H.J.; Epstein, M.; Batlle, D. Renin-Angiotensin System Blockers and the COVID-19 Pandemic: At Present There Is No Evidence to Abandon Renin-Angiotensin System Blockers. Hypertension 2020, 75, 1382–1385. [Google Scholar] [CrossRef] [Green Version]
- Esler, M.; Esler, D. Can angiotensin receptor-blocking drugs perhaps be harmful in the COVID-19 pandemic? J. Hypertens. 2020, 38, 1–2. [Google Scholar] [CrossRef]
- Kjeldsen, S.E.; Narkiewicz, K.; Burnier, M.; Oparil, S. Potential protective effects of antihypertensive treatments during the COVID-19 pandemic: From inhibitors of the renin-angiotensin system to beta-adrenergic receptor blockers. Blood Press 2021, 30, 1–3. [Google Scholar] [CrossRef] [PubMed]
- Mehta, N.; Kalra, A.; Nowacki, A.S.; Anjewierden, S.; Han, Z.; Bhat, P.; Carmona-Rubio, A.E.; Jacob, M.; Procop, G.W.; Harrington, S.; et al. Association of Use of Angiotensin-Converting Enzyme Inhibitors and Angiotensin II Receptor Blockers With Testing Positive for Coronavirus Disease 2019 (COVID-19). JAMA Cardiol. 2020, 5, 1020–1026. [Google Scholar] [CrossRef] [PubMed]
- Reynolds, H.R.; Adhikari, S.; Pulgarin, C.; Troxel, A.B.; Iturrate, E.; Johnson, S.B.; Hausvater, A.; Newman, J.D.; Berger, J.S.; Bangalore, S.; et al. Renin-Angiotensin-Aldosterone System Inhibitors and Risk of COVID-19. N. Engl. J. Med. 2020, 382, 2441–2448. [Google Scholar] [CrossRef]
- Richardson, S.; Hirsch, J.S.; Narasimhan, M.; Crawford, J.M.; McGinn, T.; Davidson, K.W.; Barnaby, D.P.; Becker, L.B.; Chelico, J.D.; Cohen, S.L.; et al. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area. JAMA 2020, 323, 2052–2059. [Google Scholar] [CrossRef] [PubMed]
- Lahens, A.; Mullaert, J.; Gressens, S.; Gault, N.; Flamant, M.; Deconinck, L.; Joly, V.; Yazdanpanah, Y.; Lescure, F.X.; Vidal-Petiot, E. Association between renin-angiotensin-aldosterone system blockers and outcome in coronavirus disease 2019: Analysing in-hospital exposure generates a biased seemingly protective effect of treatment. J. Hypertens. 2021, 39, 367–375. [Google Scholar] [CrossRef]
- Angeli, F.; Reboldi, G.; Spanevello, A.; De Ponti, R.; Visca, D.; Marazzato, J.; Zappa, M.; Trapasso, M.; Masnaghetti, S.; Fabbri, L.M.; et al. Electrocardiographic features of patients with COVID-19: One year of unexpected manifestations. Eur. J. Intern. Med. 2022, 95, 7–12. [Google Scholar] [CrossRef]
- Angeli, F.; Spanevello, A.; De Ponti, R.; Visca, D.; Marazzato, J.; Palmiotto, G.; Feci, D.; Reboldi, G.; Fabbri, L.M.; Verdecchia, P. Electrocardiographic features of patients with COVID-19 pneumonia. Eur. J. Intern. Med. 2020, 78, 101–106. [Google Scholar] [CrossRef]
- Verdecchia, P.; Angeli, F.; Reboldi, G. Angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers and coronavirus. J. Hypertens. 2020, 38, 1190–1191. [Google Scholar] [CrossRef]
- Mongelli, A.; Barbi, V.; Gottardi Zamperla, M.; Atlante, S.; Forleo, L.; Nesta, M.; Massetti, M.; Pontecorvi, A.; Nanni, S.; Farsetti, A.; et al. Evidence for Biological Age Acceleration and Telomere Shortening in COVID-19 Survivors. Int. J. Mol. Sci. 2021, 22, 6151. [Google Scholar] [CrossRef]
- Ferrario, C.M.; Jessup, J.; Chappell, M.C.; Averill, D.B.; Brosnihan, K.B.; Tallant, E.A.; Diz, D.I.; Gallagher, P.E. Effect of angiotensin-converting enzyme inhibition and angiotensin II receptor blockers on cardiac angiotensin-converting enzyme 2. Circulation 2005, 111, 2605–2610. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ishiyama, Y.; Gallagher, P.E.; Averill, D.B.; Tallant, E.A.; Brosnihan, K.B.; Ferrario, C.M. Upregulation of angiotensin-converting enzyme 2 after myocardial infarction by blockade of angiotensin II receptors. Hypertension 2004, 43, 970–976. [Google Scholar] [CrossRef] [Green Version]
- Karram, T.; Abbasi, A.; Keidar, S.; Golomb, E.; Hochberg, I.; Winaver, J.; Hoffman, A.; Abassi, Z. Effects of spironolactone and eprosartan on cardiac remodeling and angiotensin-converting enzyme isoforms in rats with experimental heart failure. Am. J. Physiol. Heart Circ. Physiol. 2005, 289, H1351–H1358. [Google Scholar] [CrossRef]
- Ocaranza, M.P.; Godoy, I.; Jalil, J.E.; Varas, M.; Collantes, P.; Pinto, M.; Roman, M.; Ramirez, C.; Copaja, M.; Diaz-Araya, G.; et al. Enalapril attenuates downregulation of Angiotensin-converting enzyme 2 in the late phase of ventricular dysfunction in myocardial infarcted rat. Hypertension 2006, 48, 572–578. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kreutz, R.; Algharably, E.A.E.H.; Azizi, M.; Dobrowolski, P.; Guzik, T.; Januszewicz, A.; Persu, A.; Prejbisz, A.; Riemer, T.G.; Wang, J.G.; et al. Hypertension, the renin-angiotensin system, and the risk of lower respiratory tract infections and lung injury: Implications for COVID-19. Cardiovasc. Res. 2020, 116, 1688–1699. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Sharma, R.K.; Stevens, B.R.; Obukhov, A.G.; Grant, M.B.; Oudit, G.Y.; Li, Q.; Richards, E.M.; Pepine, C.J.; Raizada, M.K. ACE2 (Angiotensin-Converting Enzyme 2) in Cardiopulmonary Diseases: Ramifications for the Control of SARS-CoV-2. Hypertension 2020, 76, 651–661. [Google Scholar] [CrossRef]
- Angeli, F.; Reboldi, G.; Verdecchia, P. Ageing, ACE2 deficiency and bad outcome in COVID-19. Clin. Chem. Lab. Med. 2021, 59, 1607–1609. [Google Scholar] [CrossRef]
- Angeli, F.; Reboldi, G.; Verdecchia, P. SARS-CoV-2 infection and ACE2 inhibition. J. Hypertens. 2021, 39, 1555–1558. [Google Scholar] [CrossRef]
- Angeli, F.; Reboldi, G.; Trapasso, M.; Verdecchia, P. Hypertension after COVID-19 vaccination. G. Ital. Cardiol. 2022, 23, 10–14. [Google Scholar]
- Verdecchia, P.; Reboldi, G.; Cavallini, C.; Mazzotta, G.; Angeli, F. ACE-inhibitors, angiotensin receptor blockers and severe acute respiratory syndrome caused by coronavirus. G. Ital. Cardiol. 2020, 21, 321–327. [Google Scholar]
- Angeli, F.; Verdecchia, P.; Reboldi, G. RAAS Inhibitors and Risk of COVID-19. N. Engl. J. Med. 2020, 383, 1990–1991. [Google Scholar]
- Wu, L.; Iwai, M.; Nakagami, H.; Chen, R.; Suzuki, J.; Akishita, M.; de Gasparo, M.; Horiuchi, M. Effect of angiotensin II type 1 receptor blockade on cardiac remodeling in angiotensin II type 2 receptor null mice. Arterioscler. Thromb. Vasc. Biol. 2002, 22, 49–54. [Google Scholar] [CrossRef] [Green Version]
- Wu, L.; Iwai, M.; Nakagami, H.; Li, Z.; Chen, R.; Suzuki, J.; Akishita, M.; de Gasparo, M.; Horiuchi, M. Roles of angiotensin II type 2 receptor stimulation associated with selective angiotensin II type 1 receptor blockade with valsartan in the improvement of inflammation-induced vascular injury. Circulation 2001, 104, 2716–2721. [Google Scholar] [CrossRef] [PubMed]
- De Gasparo, M.; Catt, K.J.; Inagami, T.; Wright, J.W.; Unger, T. International union of pharmacology. XXIII. The angiotensin II receptors. Pharmacol. Rev. 2000, 52, 415–472. [Google Scholar]
- Horiuchi, M.; Akishita, M.; Dzau, V.J. Recent progress in angiotensin II type 2 receptor research in the cardiovascular system. Hypertension 1999, 33, 613–621. [Google Scholar] [CrossRef] [Green Version]
- Ran, J.; Song, Y.; Zhuang, Z.; Han, L.; Zhao, S.; Cao, P.; Geng, Y.; Xu, L.; Qin, J.; He, D.; et al. Blood pressure control and adverse outcomes of COVID-19 infection in patients with concomitant hypertension in Wuhan, China. Hypertens. Res. 2020, 43, 1267–1276. [Google Scholar] [CrossRef]
Variable | Overall (n = 566) | ACE-Is (n = 162) | ARBs (n = 147) | Other BP-Lowering Drugs (n = 257) | p |
---|---|---|---|---|---|
Age (years) | 75 ± 11 | 74 ± 12 | 76 ± 10 | 76 ± 11 | 0.060 |
Sex (male, %) | 54 | 62 | 50 | 50 | 0.047 |
BMI (Kg/m2) | 27.3 ± 5.7 | 27.6 ± 6.5 | 26.6 ± 4.2 | 27.5 ± 5.9 | 0.334 |
History | |||||
COPD (%) | 16 | 12 | 15 | 20 | 0.112 |
Type 2 diabetes (%) | 31 | 30 | 31 | 31 | 0.936 |
Dyslipidemia (%) | 32 | 28 | 36 | 33 | 0.330 |
Previous cardiac event (%) | 29 | 30 | 22 | 32 | 0.144 |
Neoplasm (%) | 11 | 11 | 15 | 9 | 0.139 |
Hospitalization | |||||
Hydroxychloroquine (%) | 58 | 58 | 60 | 56 | 0.715 |
Antiretroviral (%) | 27 | 28 | 25 | 28 | 0.834 |
Macrolides (%) | 32 | 28 | 38 | 30 | 0.298 |
Aspirin (%) | 31 | 32 | 30 | 31 | 0.919 |
NSAIDs or glucocorticoids (%) | 40 | 43 | 45 | 36 | 0.149 |
Oxygen level (%) | 95 ± 3 | 95 ± 3 | 95 ± 3 | 95 ± 3 | 0.715 |
Severe hypotension | 7 | 6 | 7 | 9 | 0.648 |
Haemoglobin (g/dL) | 11.5 ± 1.8 | 11.7 ± 1.8 | 11.2 ± 1.7 | 11.5 ± 1.8 | 0.022 |
Lymphocyte count (× 103) | 1.51 ± 1.01 | 1.51 ± 0.66 | 1.63 ± 1.53 | 1.44 ± 0.76 | 0.238 |
eGFR (mL/min/1.73 m2) | 72 ± 23 | 73 ± 24 | 72 ± 21 | 72 ± 25 | 0.818 |
K+ (ng/mL) | 4.4 ± 0.6 | 4.4 ± 0.6 | 4.3 ± 0.5 | 4.3 ± 0.6 | 0.291 |
Troponin elevation (%) | 17 | 17 | 16 | 19 | 0.649 |
PaO2/FIO2 ratio (mm) | 315 ± 129 | 316 ± 110 | 309 ± 128 | 318 ± 141 | 0.852 |
High-sensitivity CRP (mg/dL) | 11.5 ± 26.4 | 7.7 ± 18.1 | 15.5 ± 41.3 | 11.7 ± 26.4 | 0.068 |
Variable | Comparison | OR | 95% CI | p |
---|---|---|---|---|
Baseline multivariable model | ||||
Age > 80 years | Yes vs. No | 2.95 | 1.68 to 5.19 | <0.0001 |
Severe hypotension | Yes vs. No | 3.77 | 1.68 to 8.45 | 0.001 |
Oxygen saturation ≤ 95% | Yes vs. No | 2.10 | 1.18 to 3.71 | 0.011 |
Lymphocyte count ≤ 1.23 × 103 | Yes vs. No | 3.66 | 2.07 to 6.46 | <0.0001 |
Baseline multivariable model and antihypertensive drug subgroups | ||||
Age > 80 years | Yes vs. No | 2.96 | 1.67 to 5.26 | <0.0001 |
Severe hypotension | Yes vs. No | 4.07 | 1.80 to 9.17 | 0.001 |
Oxygen saturation ≤ 95% | Yes vs. No | 2.15 | 1.21 to 3.82 | 0.009 |
Lymphocyte count ≤ 1.23 × 103 | Yes vs. No | 3.65 | 2.06 to 6.47 | <0.0001 |
BP-lowering drugs | ||||
ACE-Is | Other BP-lowering drugs | 0.73 | 0.38 to 1.40 | 0.339 |
ARBs | Other BP-lowering drugs | 0.37 | 0.17 to 0.80 | 0.012 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Angeli, F.; Verdecchia, P.; Balestrino, A.; Bruschi, C.; Ceriana, P.; Chiovato, L.; Dalla Vecchia, L.A.; Fanfulla, F.; La Rovere, M.T.; Perego, F.; et al. Renin Angiotensin System Blockers and Risk of Mortality in Hypertensive Patients Hospitalized for COVID-19: An Italian Registry. J. Cardiovasc. Dev. Dis. 2022, 9, 15. https://doi.org/10.3390/jcdd9010015
Angeli F, Verdecchia P, Balestrino A, Bruschi C, Ceriana P, Chiovato L, Dalla Vecchia LA, Fanfulla F, La Rovere MT, Perego F, et al. Renin Angiotensin System Blockers and Risk of Mortality in Hypertensive Patients Hospitalized for COVID-19: An Italian Registry. Journal of Cardiovascular Development and Disease. 2022; 9(1):15. https://doi.org/10.3390/jcdd9010015
Chicago/Turabian StyleAngeli, Fabio, Paolo Verdecchia, Antonella Balestrino, Claudio Bruschi, Piero Ceriana, Luca Chiovato, Laura Adelaide Dalla Vecchia, Francesco Fanfulla, Maria Teresa La Rovere, Francesca Perego, and et al. 2022. "Renin Angiotensin System Blockers and Risk of Mortality in Hypertensive Patients Hospitalized for COVID-19: An Italian Registry" Journal of Cardiovascular Development and Disease 9, no. 1: 15. https://doi.org/10.3390/jcdd9010015
APA StyleAngeli, F., Verdecchia, P., Balestrino, A., Bruschi, C., Ceriana, P., Chiovato, L., Dalla Vecchia, L. A., Fanfulla, F., La Rovere, M. T., Perego, F., Scalvini, S., Spanevello, A., Traversi, E., Visca, D., Vitacca, M., & Bachetti, T. (2022). Renin Angiotensin System Blockers and Risk of Mortality in Hypertensive Patients Hospitalized for COVID-19: An Italian Registry. Journal of Cardiovascular Development and Disease, 9(1), 15. https://doi.org/10.3390/jcdd9010015