**1. Introduction**

Arteriovenous fistula (AVF) represents the gold standard vascular access (VA) for haemodialysis (HD). Over time, AVFs may develop dysfunction and lower blood flow due to a series of biological changes that can lead to the formation of a stenosis and subsequent thrombosis. This event has a severe impact on the clinical status of dialysis patients; in the best scenario, endovascular and surgical interventions can restore a satisfactory AVF flow; if not, a central venous catheter (CVC) needs to be placed for interim dialysis access.

Considering the strong negative impact of AVF failure on patient survival, morbidity and quality of life, recent guidelines focused on potential strategies for AVF preservation.

**Citation:** Peralta, R.; Garbelli, M.; Bellocchio, F.; Ponce, P.; Stuard, S.; Lodigiani, M.; Fazendeiro Matos, J.; Ribeiro, R.; Nikam, M.; Botler, M.; et al. Development and Validation of a Machine Learning Model Predicting Arteriovenous Fistula Failure in a Large Network of Dialysis Clinics. *Int. J. Environ. Res. Public Health* **2021**, *18*, 12355. https:// doi.org/10.3390/ijerph182312355

Academic Editor: Tim Hulsen

Received: 15 October 2021 Accepted: 22 November 2021 Published: 24 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

The National Kidney Foundation's (NKF) (KDOQI) Guidelines [1], recommend AVF periodical physical examination (PE), or ultrasound evaluation as primary monitoring methods to detect access dysfunction. However, there is no evidence on the advantages to routine AVF surveillance by measuring intra access blood flow (Qa) [1,2] to improve access patency; nevertheless, its assessment should be considered [3,4].

The controversy concerning the best surveillance strategy to ascertain and evaluate venous stenoses has not yet been solved [5]. The rationale for surveillance is based on the hypothesis that progressive stenosis can be accurately detected by reduced Qa and increased venous pressure (VP) before VA thrombosis occurs [4,6].

Even though both Qa surveillance and ultrasound examination, coupled with preemptive correction of hemodynamically significantly reduces the risk of thrombosis and access loss [7–12], false positive tests would lead to unnecessary intervention procedures [13] which may ultimately promote further neointimal hyperplasia [14]. No current surveillance method is without pitfalls. Major concerns for Qa surveillance relate to low reproducibility in clinical practice which corresponds to a minimal detectable change as large as 25%, questionable cost-effectiveness as the sole surveillance strategy [15] and suboptimal inter-rater agreement across different measurement techniques [16]. Furthermore, the accuracy in identifying stenosis with Qa varies according to patient characteristics and location [15,17]. On the other hand, ultrasound examination requires significant operator training and skill, may not be readily available in all clinical contexts and may not yield conclusive indications for interventions [18,19]. Structured physical examination has been proposed as a convenient alternative monitoring method. The assessment of PE accuracy in detecting and locating AVF stenosis has shown mixed results; whereas few studies have shown acceptable accuracy in either the diagnosis of outflow and of inflow stenosis [20,21] compared with angiography; few others [22,23] reached opposite conclusions. In addition, a metaanalysis of randomized control trial (RCT) studies showed that blood flow measurement was superior in predicting outcomes [24–26]. Furthermore, PE is operator-dependent [27], and has limited long-term prediction power thus explaining why, in a large majority of the cases, many patients may need more frequent surveillance when assuming a rapid AVF deterioration. Taken together, the impact of PE alone on actual prevention of thrombosis is limited [28].

An excellent surveillance method should be quick, easy, accurate, non-invasive, nonoperator-dependent and cost-effective. It is clear, that none of the existing methods can fulfil such expectations alone and a one-fits-all approach is not be able to adequately capture the diversity of AVF functional trajectories between and within patients.

In principle, an automatic triage system based on routinely recorded data requiring no additional effort by healthcare professionals may be used to personalize surveillance strategies based on expected risk stratification.

To this end, we sought to develop and validate a risk model based on the machine learning methods predicting the occurrence of AVF failure within three months.

### **2. Materials and Methods**

#### *2.1. General Description of the Arteriovenous Fistula Failure Model (AVF-FM)*

The AVF Failure Model (AVF-FM) aims at predicting the occurrence of a composite AVF failure endpoint (see, Endpoint Definition below) within three months based on routinely recorded clinical information readily available in health information systems for dialysis patients.

The model is based on the XGBoost algorithm, an iterative method where, at each iteration, a new sub-model is added to correct the prediction error of the previous iteration. Each sub-model is an ensemble of decision trees. A decision tree can be roughly described as a flowchart-like structure in which each internal node represents a "discrimination test" on a given attribute (e.g., any clinical parameter or demographic characteristics); each branch of the decision tree represents the result of the discrimination test (i.e., passed

or not), and each leaf node represent the probability of the outcome. This probability represents the prevalence of events occurring in each leaf in the training set.

The iterative process ends in accordance with a pre-specified stopping rule (e.g., maximum number of iterations or minimal acceptable average prediction error). The structure of the model is computed as a function optimization process combining the minimization of both training error and model complexity.

We selected XGBoost since it is characterized by a good prediction accuracy in a broad variety of problems coupled with short computational time. Furthermore, SHapley Additive exPlanations (SHAP) analysis [29] enables intuitive model interpretation through an accurate and efficient estimation of the contribution of each input variable to the risk.

### *2.2. AVF-FM Training*

The AVF-FM was derived using the information collected in the European Clinical Database (EuCliD®, Fresenius Medical Care, Deutschland GmbH, Wendel, Germany), a large, multinational, database including in-centre dialysis patients [30].

We enrolled all HD/HDF adult patients in Italy, Spain, and Portugal with at least five treatments performed using AVF as vascular access, in the period January 2015–October 2019 and at least three months of follow-up. Furthermore, we considered only AVFs with more than three months of maturation. The unit of analysis for model development and testing was the patient-quarter. The final dataset included all eligible patient quarters (January, April, July and October) for each year. The ascertainment period for feature computation is represented in Figure 1. To ensure sufficient data completeness, we excluded patients with less than 90 days of ascertainment period before the index date for computation.

**Figure 1.** Study Design: the diagram represents the ascertainment period design for different groups of variables.
