*Article* **Validation of a Novel Predictive Algorithm for Kidney Failure in Patients Suffering from Chronic Kidney Disease: The Prognostic Reasoning System for Chronic Kidney Disease (PROGRES-CKD)**

**Francesco Bellocchio 1,\*, Caterina Lonati 2, Jasmine Ion Titapiccolo 1, Jennifer Nadal 3, Heike Meiselbach 4, Matthias Schmid 3, Barbara Baerthlein 5, Ulrich Tschulena 6, Markus Schneider 3, Ulla T. Schultheiss 7,8, Carlo Barbieri 6, Christoph Moore 6, Sonja Steppan 6, Kai-Uwe Eckardt 4,9, Stefano Stuard <sup>6</sup> and Luca Neri <sup>1</sup>**

	- 91054 Erlangen, Germany; Barbara.Baerthlein@uk-erlangen.de

**Abstract:** Current equation-based risk stratification algorithms for kidney failure (KF) may have limited applicability in real world settings, where missing information may impede their computation for a large share of patients, hampering one from taking full advantage of the wealth of information collected in electronic health records. To overcome such limitations, we trained and validated the Prognostic Reasoning System for Chronic Kidney Disease (PROGRES-CKD), a novel algorithm predicting end-stage kidney disease (ESKD). PROGRES-CKD is a naïve Bayes classifier predicting ESKD onset within 6 and 24 months in adult, stage 3-to-5 CKD patients. PROGRES-CKD trained on 17,775 CKD patients treated in the Fresenius Medical Care (FMC) NephroCare network. The algorithm was validated in a second independent FMC cohort (*n* = 6760) and in the German Chronic Kidney Disease (GCKD) study cohort (*n* = 4058). We contrasted PROGRES-CKD accuracy against the performance of the Kidney Failure Risk Equation (KFRE). Discrimination accuracy in the validation cohorts was excellent for both short-term (stage 4–5 CKD, FMC: AUC = 0.90, 95%CI 0.88–0.91; GCKD: AUC = 0.91, 95% CI 0.86–0.97) and long-term (stage 3–5 CKD, FMC: AUC = 0.85, 95%CI 0.83–0.88; GCKD: AUC = 0.85, 95%CI 0.83–0.88) forecasting horizons. The performance of PROGRES-CKD was non-inferior to KFRE for the 24-month horizon and proved more accurate for the 6-month horizon forecast in both validation cohorts. In the real world setting captured in the FMC validation cohort, PROGRES-CKD was computable for all patients, whereas KFRE could be computed for complete cases only (i.e., 30% and 16% of the cohort in 6- and 24-month horizons). PROGRES-CKD accurately predicts KF onset among CKD patients. Contrary to equation-based scores, PROGRES-CKD extends to patients with incomplete data and allows explicit assessment of prediction robustness in case of missing values. PROGRES-CKD may efficiently assist physicians' prognostic reasoning in real-life applications.

**Citation:** Bellocchio, F.; Lonati, C.; Ion Titapiccolo, J.; Nadal, J.; Meiselbach, H.; Schmid, M.; Baerthlein, B.; Tschulena, U.; Schneider, M.; Schultheiss, U.T.; et al. Validation of a Novel Predictive Algorithm for Kidney Failure in Patients Suffering from Chronic Kidney Disease: The Prognostic Reasoning System for Chronic Kidney Disease (PROGRES-CKD). *Int. J. Environ. Res. Public Health* **2021**, *18*, 12649. https://doi.org/10.3390/ ijerph182312649

Academic Editor: Paul B. Tchounwou

Received: 28 October 2021 Accepted: 25 November 2021 Published: 30 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Keywords:** chronic kidney disease (CKD); end-stage kidney disease (ESKD); kidney replacement therapy (KRT); risk prediction; artificial intelligence; machine learning; naïve Bayes classifiers; precision medicine

#### **1. Introduction**

Multiple behavioral and pharmacological interventions have proven effective in reducing the burden of risk factors for chronic kidney disease (CKD) progression [1–4]. Furthermore, timely transition management (i.e., vascular access creation and training) for patients needing Kidney Replacement Therapy (KRT) is associated with prolonged survival and reduced complication rates once on dialysis, while delayed referrals are associated with increased morbidity, mortality, and healthcare costs [5], as well as worse patient quality of life [6]. Therefore, early identification of high risk patients is an essential prerequisite of personalized clinical decision making [7–9].

Several prediction models were developed to assist physicians in forecasting CKD progression [10]. However, most of them have not been consistently implemented in clinical practice [9,11,12]. Indeed, the majority of published risk scores lack external validation [11,13,14], leading to suboptimal discrimination in external populations [12] and limited generalizability to clinical settings [11]. One prominent exception is represented by the Kidney Failure Risk Equations (KFREs) developed by Tangri and colleagues [15], which showed stable discrimination in different validation studies [16–18]. However, KFREs do not provide short-term forecasts, are not calculable for patients with incomplete data, and need re-calibration when applied to CKD populations with risk factor distributions departing from those of the original derivation dataset.

To overcome such limitations, we developed the Prognostic Reasoning System for Chronic Kidney Disease (PROGRES-CKD), a risk score application for adult patients suffering from CKD stages 3–5. PROGRES-CKD is based on a naïve Bayes Classifier (NBC) algorithm and it was trained on a large-multinational clinical dataset, reflecting real-world clinical practice. The application includes PROGRES-CKD-6 for 6-month forecasting and PROGRES-CKD-24 for 24-month forecasting.

In the present study, we reported the training and validation of both PROGRES-CKD-6 and PROGRES-CKD-24 in two independent samples of CKD patients: the FMC Nephro-Care cohort (European Clinical Database, EuCliD®, [19,20]) and the German Chronic Kidney Disease (GCKD) study cohort [21]. Moreover, we compared the PROGRES-CKD discrimination accuracy and suitability for clinical practice against the KFREs equations.

#### **2. Materials and Methods**

In reporting PROGRES-CKD training and validation studies we adhered to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement [22] and to the Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research [23].
