**4. Discussion**

The wide scale implementation of electronic health record technology has led to an important and unprecedented accumulation of clinical data, and patient information is immediately accessible to computer systems. We exploited the wealth of information stored in the EuCliD® system to derive a machine-learning algorithm for the prediction of AVF failure within three months.

The model showed good discrimination and excellent calibration. To enhance the interpretation and usability of risk estimates yielded by the model we selected three thresholds identifying four distinct risk classes. The largest group was represented by very low risk patients for whom the expected incidence of the composite AVF failure endpoint was remarkably lower than the observed incidence in the whole target population. On the other side of the spectrum there is a small group of patients accounting for less than 1% of the target population with extremely high risk of clinically significant AVF disfunctions within three months. This risk classification can be used to design personalized clinical management workflows. For example, routine monitoring using dialysis parameters and physical examination may suffice for the very low risk group, thus reducing the costs, resource requirements and importantly, unnecessary interventions. Conversely, the very high-risk patient group may be candidate for a more intensive surveillance and clinical review protocol to rule out conditions deserving immediate interventions. In-between, we found two risk classes with moderate and high risk of AVF failure, respectively. For both such classes, the optimal surveillance strategy could be designed to suit the needs and resources of the local clinic, regions, or larger geography. Importantly, accurate risk estimation makes the process of AVF surveillance optimization transparent and reproducible.

Feature analysis disclosed key information to inspect model functioning and enhance score interpretation. Among the 46 input variables, the main contribution to model performance was the past history of failures for the AVF in use, a condition associated with both constitutional proneness to thrombosis and increased AVF vulnerability due to previous surgical interventions aimed at re-establishing patency [34]. In fact, AVF stenosis are one of the most common reasons for repeated endovascular or surgical intervention and are a well-known problem in AV access maintenance. The high re-intervention rate observed (i.e., 2.46 ± 1.40 procedures/patient/year) [35], clearly explains the importance of past history of failure events as a key variable for our model.

One important finding of our study was that the majority of the 15 most important variables in the model were represented by metrics tapping functional parameters of the AVF under examination, namely recirculation rate, dynamic arterial and venous access pressures, effective blood flow and spKt/V. Access recirculation was the second most important contributing feature to risk estimates in our model. The measurement of access recirculation has been used as a non-invasive method based by ultrasound dilution technique (or dilutional-based method) to determine access blood flow (Qa) [36], and stenosis identification. A high degree of access recirculation is one of the factors more importance to identify AVF inflow problems among HD patients and was routinely used for screening of stenosis in 64% from facilities in northern Italy [37]. Access recirculation and poor HD adequacy assessed by spKt/V, may help indicate AV access dysfunction [1]. A recent study by Robert et al. [38] concluded that routine measurements of spKt/V was a quick and straightforward method for early detection of hemodynamically significant AV fistula stenosis.

Similarly, hemodynamic metrics representing the trajectory of dynamic venous and arterial pressures in the dialysis access circuit along time were strong contributors of risk estimates. Alteration of metrics representing the temporal profile of dynamic venous and arterial pressures suggest a high predictive risk of AVF failure. Abnormal dynamic arterial pressure (DAP) may be suggestive of access inflow problems while alterations of dynamic venous pressure (DVP) is associated with outflow stenosis. The incidence of inflow stenosis in patients with AVF from the cases referred to interventional facilities can reach rates of 40% with significant effects in reducing dialysis blood pump flow (Qb) [39]; therefore, combining several AVF dysfunction predictors during the same surveillance evaluation is of paramount importance.

Of note, all such measures are automatically recorded by sensors installed on HD machines and have been used, alone or in conjunction for AVF monitoring [1]. The great advantage of such metrics over routine access flow measurement (Qa) relates to their continuous, effortless availability, since they are measured without any interruption in the patient's dialysis process, and without time-consuming procedures. Despite Qa has been shown to outperform each of these functional parameters taken alone, this is the first study showing the potential of their combined use for AVF functional assessment. Given that Qa may be consistently available for a minority of patient, we did not include it in the input matrix for model generation. Whether the combination of our risk estimates and Qa provides additional predictive power in selected patients is a matter of further research.

Furthermore, given the strong dependency of risk estimates on AVF functional parameters, our model is sensitive to their changes in AVF and can be used to track risk trajectories over time without any additional data collection burden to the healthcare staff.

Our study has several strengths. The large sample size gathered from multiple dialysis centres across several countries ensured capturing wide diversity in clinical practice and case-mix, two necessary pre-condition for reproducibility and generalizability in machine learning. Additionally, we could leverage on a wide array of clinical variables to characterize patients' health status including laboratory test results, socio-demographic information, medication, dialysis treatment parameters, comorbidities and data continuously recorded by the dialysis machine during each dialysis session. The evidence regarding risk factors associated with AVF patency loss is still limited. Most studies have small sample size, and a limited set of variables was available [40]. On the contrary, we were able to evaluate the association of AVF patency loss with over 100 clinical parameters and their temporal dynamics, an unprecedented wealth of information. One additional benefit of XGBoostbased algorithm is their inherent explainability, which ensures transparency in clinical decision making. For each patient the model produces SHAP metrics which represent the importance of clinical parameters on risk estimates, allowing independent assessment by the attending physician.

On the other hand, we should acknowledge some limitations as well. Our endpoint definition is a composite outcome including thrombosis, switch to another vascular access, interventions aimed at re-establishing patency in outpatient setting and day hospital admission related to intervention to re-establish patency of the AVF. Despite our operational definition is consistent with the endpoint criteria for AVF patency loss described in the *Recommended standards for reports dealing with arteriovenous hemodialysis accesses* issued by the International Society of Vascular Surgery [41], we rely on data reported by healthcare professionals in clinical practice. Therefore, we cannot rule out the possibility that information bias affected our results. Additionally, our definition reflects medical treatment decision and therefore we cannot exclude that inappropriate surgical intervention have been conducted. This may be reflected in our risk estimates (A detailed description of the endpoint definition is reported Supplementary Table S2). Furthermore, all patients included in our analysis received treatment in the NephroCare network. Despite the multicentre, cross-country design of the study, whether the accuracy and calibration of the AVF-FM can be replicated in centres outside the NephroCare network is a matter of further research.

#### **5. Conclusions**

The fundamental principle for performing routine vascular access monitoring and surveillance is timely identification and correction of significant stenosis, thus prolonging patency. Current monitoring and surveillance methods remain operator dependent, may be inefficient and may potentially lead to unnecessary interventions.

The AVF Failure Model has shown promising discrimination performance by combining routinely collected clinical as well as sensor data; therefore, the AVF Failure Model can

potentially enable risk-based personalization of AVF surveillance strategies. Whether the use of the AVF Failure Model in clinical practice would translate in more efficient care and prolonged access survival is a matter of further clinical testing.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/ijerph182312355/s1, Supplementary Table S1: Detailed description of the endpoint definition; Supplementary Table S2: All variables included in the first training iteration; Supplementary Table S3: breakdown of AVF Failure causes in our study; Supplementary Table S4: Distribution of AVF-FM risk classes in 30 re-samplings of the test set.

**Author Contributions:** The first draft of the manuscript was written by R.P. and M.G. M.G. also contributed to data management, data analysis, model development, and approved the final version of the manuscript. L.N. contributed to the study concept and design, model development, interpretation of results, manuscript drafting and oversaw the conduct of the study. F.B. contributed to study concept and design, model development, interpretation of results, manuscript drafting, and approved the final version of the manuscript. M.L. contributed to model development, interpretation of results and approved the final version of the manuscript. D.B. Contributed to study concept, interpretation of results and approved the final version of the manuscript. P.P., S.S., J.F.M., R.R., M.N., M.B., E.S. contributed to interpretation of results and approved the final version of the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding. DB received consulting fees from FMC gmbh in the past 3 years. All remaining authors are full time employees at FMC.

**Institutional Review Board Statement:** The analysis has been conducted in accordance with the declaration of Helsinki. The analysis has been conducted to inform a continuous quality improvement program of health care practice. The Institutional Review Board of FMC-Nephrocare Portugal has confirmed that the study adheres to ethical standards and retrospectively approved the study protocol on 11 October 2021 (see Supplementary Material: Ethics Committee Approval).

**Informed Consent Statement:** All patients included in the study consented their data be used in pseudo-anonymized form for continuous quality improvement and scientific research at their registration to dialysis centers belonging to the Nephrocare network.

**Data Availability Statement:** The datasets used and/or analysed during the current study are personal health information obtained during provision of healthcare services and cannot be shared to protect their confidentiality in compliance with GDPR regulation.

**Conflicts of Interest:** D.B. received consulting fees from FMC in the past 3 years. R.P., M.G., L.N., F.B., M.L., P.P., S.S., M.B., E.S., M.N., J.F.M., R.R. are full time employees at FMC.

### **References**

