*2.6. Performance Metrics*

A comparison between manual and automatic masks was carried out to assess RENFAST's performance in the segmentation of kidney blood vessels and fibrosis. Manual annotations of blood vessels were generated using a custom graphical user interface based on MATLAB. Since fibrosis segmentation can be a long and demanding task, we designed a semi-automatic pipeline to help the pathologist during the generation of the manual mask (Appendix B). Several pixel-based metrics, such as balanced accuracy, precision, recall, and F1SCORE, were evaluated for both blood vessel and fibrosis segmentation. Balanced accuracy (BalACCURACY) is a common metric used in segmentation problems to deal with imbalanced datasets (TP vs. TN). BalACCURACY is calculated as the average of the correct predictions of each class individually. Precision is employed to evaluate the false detection of ghost shapes; recall quantifies the missed detection of ground truth objects; and finally, the F1SCORE is defined as the harmonic mean between precision and recall.

Accurate segmentation of blood vessel borders is fundamental for a correct evaluation of vascular damage. For this reason, we also evaluated the Dice coefficient (DSC) and the Hausdorff distance for all the true-positive vascular structures. Specifically, we computed the 95th percentile Hausdorff distance (HD95), which is defined as the maximum distance of a set (manual boundary) to the nearest point in the other set (automatic boundary). This metric is more robust towards a very small subset of outliers because it is based on the calculation of the 95th percentile of distances. During fibrosis assessment, the pathologist computes the ratio between fibrotic tissue and the whole tissue area. For each image, the absolute error (AE) between manual and automatic estimation was calculated as

$$AE = \left| \left( \frac{fibrosis\_{AREA}}{tissue\_{AREA}} \right)\_{MAXILAL} - \left( \frac{fibrosis\_{AREA}}{tissue\_{AREA}} \right)\_{RENEAST} \right| \tag{5}$$

where (·)*MANUAL* and (·)*RENFAST* denote the manual and the automatic annotations, respectively.
