A CT-Based Clinical, Radiological and Radiomic Machine Learning Model for Predicting Malignancy of Solid Renal Tumors (UroCCR-75)
Abstract
:1. Introduction
2. Materials and Methods
2.1. Patients
2.2. CT Examinations
2.3. CT
2.4. Statistical Analyses
3. Results
3.1. Patients and Tumors
3.2. Conventional and Enhanced Radiological Features
3.3. Three-Dimensional Radiomic Features
3.4. Machine Learning Predictive Models
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
2D | two-dimensional |
3D | three-dimensional |
AUC | area under the receiver operating characteristic curve |
ECOG | Eastern Cooperative Oncology Group |
fpAML | fat-poor angiomyolipoma |
MCE-CT | multiphasic contrast-enhanced computed tomography |
ML | machine learning |
OR | odds ratio |
RCC | renal cell carcinoma |
SVM | support vector machine |
TNM | tumor–node–metastasis |
References
- Moch, H.; Cubilla, A.L.; Humphrey, P.A.; Reuter, V.E.; Ulbright, T.M. The 2016 WHO Classification of Tumours of the Urinary System and Male Genital Organs—Part A: Renal, Penile, and Testicular Tumours. Eur. Urol. 2016, 70, 93–105. [Google Scholar] [PubMed]
- Marconi, L.; Dabestani, S.; Lam, T.B.; Hofmann, F.; Stewart, F.; Norrie, J.; Bex, A.; Bensalah, K.; Canfield, S.E.; Hora, M.; et al. Systematic Review and Meta-analysis of Diagnostic Accuracy of Percutaneous Renal Tumour Biopsy. Eur. Urol. 2016, 69, 660–673. [Google Scholar] [CrossRef] [PubMed]
- Patel, H.; Druskin, S.C.; Rowe, S.P.; Pierorazio, P.M.; Gorin, M.A.; Allaf, M.E. Surgical histopathology for suspected oncocytoma on renal mass biopsy: A systematic review and meta-analysis. BJU Int. 2017, 119, 661–666. [Google Scholar] [CrossRef] [Green Version]
- Marcelin, C.; Ambrosetti, D.; Bernhard, J.; Roy, C.; Grenier, N.; Cornelis, F. Percutaneous image-guided biopsies of small renal tumors: Current practice and perspectives. Diagn. Interv. Imaging 2017, 98, 589–599. [Google Scholar] [CrossRef]
- Cornelis, F.; Grenier, N. Multiparametric Magnetic Resonance Imaging of Solid Renal Tumors: A Practical Algorithm. Semin. Ultrasound CT MRI 2017, 38, 47–58. [Google Scholar] [CrossRef]
- Dyer, R.; DiSantis, D.J.; McClennan, B.L. Simplified Imaging Approach for Evaluation of the Solid Renal Mass in Adults. Radiology 2008, 247, 331–343. [Google Scholar] [CrossRef]
- Gill, I.S.; Aron, M.; Gervais, D.A.; Jewett, M.A. Small Renal Mass. N. Engl. J. Med. 2010, 362, 624–634. [Google Scholar] [CrossRef]
- Young, J.R.; Margolis, D.; Sauk, S.; Pantuck, A.J.; Sayre, J.; Raman, S.S. Clear cell renal cell carcinoma: Discrimination from other renal cell carcinoma subtypes and oncocytoma at multiphasic multidetector CT. Radiology 2013, 267, 444–453. [Google Scholar]
- Ding, J.; Xing, Z.; Jiang, Z.; Chen, J.; Pan, L.; Qiu, J.; Xing, W. CT-based radiomic model predicts high grade of clear cell renal cell carcinoma. Eur. J. Radiol. 2018, 103, 51–56. [Google Scholar] [CrossRef]
- Kocak, B.; Yardimci, A.H.; Bektas, C.T.; Turkcanoglu, M.H.; Erdim, C.; Yucetas, U.; Koca, S.B.; Kilickesmez, O. Textural differences between renal cell carcinoma subtypes: Machine learning-based quantitative computed tomography texture analysis with independent external validation. Eur. J. Radiol. 2018, 107, 149–157. [Google Scholar] [CrossRef]
- Bektas, C.T.; Kocak, B.; Yardimci, A.H.; Turkcanoglu, M.H.; Yucetas, U.; Koca, S.B.; Erdim, C.; Kilickesmez, O. Clear Cell Renal Cell Carcinoma: Machine Learning-Based Quantitative Computed Tomography Texture Analysis for Prediction of Fuhrman Nuclear Grade. Eur. Radiol. 2019, 29, 1153–1163. [Google Scholar] [CrossRef] [PubMed]
- Yang, R.; Wu, J.; Sun, L.; Lai, S.; Xu, Y.; Liu, X.; Ma, Y.; Zhen, X. Radiomics of small renal masses on multiphasic CT: Accuracy of machine learning-based classification models for the differentiation of renal cell carcinoma and angiomyolipoma without visible fat. Eur. Radiol. 2020, 30, 1254–1263. [Google Scholar] [CrossRef] [PubMed]
- Yap, F.Y.; Varghese, B.A.; Cen, S.Y.; Hwang, D.H.; Lei, X.; Desai, B.; Lau, C.; Yang, L.L.; Fullenkamp, A.J.; Hajian, S.; et al. Shape and texture-based radiomics signature on CT effectively discriminates benign from malignant renal masses. Eur. Radiol. 2021, 31, 1011–1021. [Google Scholar] [CrossRef] [PubMed]
- Erdim, C.; Yardimci, A.H.; Bektas, C.T.; Kocak, B.; Koca, S.B.; Demir, H.; Kilickesmez, O. Prediction of Benign and Malignant Solid Renal Masses: Machine Learning-Based CT Texture Analysis. Acad. Radiol. 2020, 27, 1422–1429. [Google Scholar] [CrossRef]
- Sudjai, N.; Siriwanarangsun, P.; Lektrakul, N.; Saiviroonporn, P.; Maungsomboon, S.; Phimolsarnti, R.; Asavamongkolkul, A.; Chandhanayingyong, C. Robustness of Radiomic Features: Two-Dimensional versus Three-Dimensional MRI-Based Feature Reproducibility in Lipomatous Soft-Tissue Tumors. Diagnostics 2023, 13, 258. [Google Scholar] [CrossRef]
- Demircioğlu, A. Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics. Insights Imaging 2021, 12, 172. [Google Scholar] [CrossRef]
- Sun, X.-Y.; Feng, Q.-X.; Xu, X.; Zhang, J.; Zhu, F.-P.; Yang, Y.-H.; Zhang, Y.-D. Radiologic-Radiomic Machine Learning Models for Differentiation of Benign and Malignant Solid Renal Masses: Comparison with Expert-Level Radiologists. Am. J. Roentgenol. 2020, 214, W44–W54. [Google Scholar] [CrossRef]
- Mühlbauer, J.; Egen, L.; Kowalewski, K.-F.; Grilli, M.; Walach, M.T.; Westhoff, N.; Nuhn, P.; Laqua, F.C.; Baessler, B.; Kriegmair, M.C. Radiomics in Renal Cell Carcinoma—A Systematic Review and Meta-Analysis. Cancers 2021, 13, 1348. [Google Scholar] [CrossRef]
- Deng, Y.; Soule, E.; Cui, E.; Samuel, A.; Shah, S.; Lall, C.; Sundaram, C.; Sandrasegaran, K. Usefulness of CT texture analysis in differentiating benign and malignant renal tumours. Clin. Radiol. 2019, 75, 108–115. [Google Scholar] [CrossRef]
- Nie, P.; Yang, G.; Wang, Z.; Yan, L.; Miao, W.; Hao, D.; Wu, J.; Zhao, Y.; Gong, A.; Cui, J.; et al. A CT-based radiomics nomogram for differentiation of renal angiomyo-lipoma without visible fat from homogeneous clear cell renal cell carcinoma. Eur. Radiol. 2020, 30, 1274–1284. [Google Scholar] [CrossRef]
- Ma, Y.; Cao, F.; Xu, X.; Ma, W. Can whole-tumor radiomics-based CT analysis better differentiate fat-poor angiomyolipoma from clear cell renal cell caricinoma: Compared with conventional CT analysis? Abdom. Imaging 2020, 45, 2500–2507. [Google Scholar] [CrossRef] [PubMed]
- Cui, E.-M.; Lin, F.; Li, Q.; Li, R.-G.; Chen, X.-M.; Liu, Z.-S.; Long, W.-S. Differentiation of renal angiomyolipoma without visible fat from renal cell carcinoma by machine learning based on whole-tumor computed tomography texture features. Acta Radiol. 2019, 60, 1543–1552. [Google Scholar] [CrossRef] [PubMed]
- Coy, H.; Hsieh, K.; Wu, W.; Nagarajan, M.B.; Young, J.R.; Douek, M.L.; Brown, M.S.; Scalzo, F.; Raman, S.S. Deep learning and radiomics: The utility of Google TensorFlowTM Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT. Abdom. Radiol. 2019, 44, 2009–2020. [Google Scholar] [CrossRef]
- Coy, H.; Young, J.R.; Douek, M.L.; Brown, M.S.; Sayre, J.; Raman, S.S. Quantitative computer-aided diagnostic algorithm for automated detection of peak lesion attenuation in differentiating clear cell from papillary and chromophobe renal cell carcinoma, oncocytoma, and fat-poor angiomyolipoma on multiphasic multidetector com. Abdom. Radiol. 2017, 42, 1919–1928. [Google Scholar] [CrossRef]
- Li, Y.; Huang, X.; Xia, Y.; Long, L. Value of radiomics in differential diagnosis of chromophobe renal cell carcinoma and renal oncocytoma. Abdom. Radiol. 2020, 45, 3193–3201. [Google Scholar] [CrossRef] [Green Version]
- Lin, F.; Ma, C.; Xu, J.; Lei, Y.; Li, Q.; Lan, Y.; Sun, M.; Long, W.; Cui, E. A CT-based deep learning model for predicting the nuclear grade of clear cell renal cell carcinoma. Eur. J. Radiol. 2020, 129, 109079. [Google Scholar] [CrossRef] [PubMed]
- Kocak, B.; Ates, E.; Durmaz, E.S.; Ulusan, M.B.; Kilickesmez, O. Influence of segmentation margin on machine learning–based high-dimensional quantitative CT texture analysis: A reproducibility study on renal clear cell carcinomas. Eur. Radiol. 2019, 29, 4765–4775. [Google Scholar] [CrossRef] [PubMed]
Variable | Benign (n = 21) | Malignant (n = 111) | Total (n = 132) | Adjusted p-Value |
---|---|---|---|---|
Sex | 0.001 | |||
Female | 16 (76.2%) | 29 (26.1%) | 45 (34.1%) | |
Male | 5 (23.8%) | 82 (73.9%) | 87 (65.9%) | |
Cockcroft clearance | 0.061 | |||
Missing data | 0 | 1 | 1 | |
Mean (SD) | 71.229 (40.212) | 91.758 (39.730) | 88.467 (40.366) | |
Age | 0.099 | |||
Missing data | 1 | 1 | 2 | |
Mean (SD) | 64.450 (11.834) | 57.327 (14.273) | 58.423 (14.122) | |
Body weight | 0.025 | |||
Mean (SD) | 67.476 (16.366) | 79.604 (18.391) | 77.674 (18.567) | |
BMI | 0.119 | |||
Mean (SD) | 24.843 (4.968) | 27.041 (5.567) | 26.691 (5.518) | |
ECOG score | 0.351 | |||
Missing data | 0 | 8 | 8 | |
0 | 20 (95.2%) | 85 (82.5%) | 105 (84.7%) | |
1–3 | 1 (4.8%) | 18 (17.5%) | 19 (15.3%) | |
Cancer history | 0.099 | |||
No | 20 (95.2%) | 81 (73.0%) | 101 (76.5%) | |
Yes | 1 (4.8%) | 30 (27.0%) | 31 (23.5%) | |
Symptoms at diagnosis | 0.240 | |||
No | 20 (95.2%) | 87 (78.4%) | 107 (81.1%) | |
Yes | 1 (4.8%) | 24 (21.6%) | 25 (18.9%) | |
Pathological stage * | 0.103 | |||
T1 | 11 (52.4%) | 84 (75.7%) | 95 (72.0%) | |
T2/T3 | 10 (47.6%) | 27 (24.3%) | 37 (28.0%) | |
Family history of renal cancer | 0.371 | |||
Missing data | 0 | 5 | 5 | |
Yes | 0 | 7 (6.6%) | 7 (5.5%) | |
No | 21 (100%) | 99 (93.4%) | 120 (94.5%) |
Variable Name | Coefficient | Odds Ratio (IC) | p |
---|---|---|---|
(Intercept) | −0.999 | 0.368 | |
delta.ratio.tumor.cortex.AP | −0.192 | 0.826 (0.057, 1) | 0.682 |
Bilateral lesion—Yes (ref: No) | −0.071 | 0.931 (0.583, 1) | 0.332 |
Necrosis—Yes (ref: No) | 0.223 | 1.250 (1, 6.355) | 0.646 |
Intensity.mean.value | −0.006 | 0.994 (0.983, 1) | 0.640 |
Sex—Men (ref: Women) | 0.853 | 2.346 (1.183, 4.384) | 0.987 |
History_cancer—Yes (ref: No) | 0.069 | 1.072 (1, 29.045) | 0.650 |
Variable Name | Benign (n = 21) | Malignant (n = 111) | Total (n = 132) | Adjusted p-Value |
---|---|---|---|---|
Bilateral.lesion | 0.024 | |||
No | 13 (61.9%) | 100 (90.1%) | 113 (85.6%) | |
Yes | 8 (38.1%) | 11 (9.9%) | 19 (14.4%) | |
Calcification | 0.348 | |||
No | 18 (85.7%) | 77 (69.4%) | 95 (72.0%) | |
Yes | 3 (14.3%) | 34 (30.6%) | 37 (28.0%) | |
Contour.regularity | 0.185 | |||
Demarcated | 17 (81.0%) | 67 (60.4%) | 84 (63.6%) | |
Infiltrating | 4 (19.0%) | 44 (39.6%) | 48 (36.4%) | |
Fat | 0.409 | |||
No | 17 (81.0%) | 101 (91.0%) | 118 (89.4%) | |
Yes | 4 (19.0%) | 10 (9.0%) | 14 (10.6%) | |
Homogeneous | 0.714 | |||
No | 14 (66.7%) | 80 (72.1%) | 94 (71.2%) | |
Yes | 7 (33.3%) | 31 (27.9%) | 38 (28.8%) | |
Monofocal.lesion | 0.099 | |||
No | 8 (38.1%) | 17 (15.3%) | 25 (18.9%) | |
Yes | 13 (61.9%) | 94 (84.7%) | 107 (81.1%) | |
Necrosis | 0.025 | |||
No | 13 (61.9%) | 30 (27.0%) | 43 (32.6%) | |
Yes | 8 (38.1%) | 81 (73.0%) | 89 (67.4%) | |
Necrotic.core | 0.099 | |||
No | 16 (76.2%) | 56 (50.5%) | 72 (54.5%) | |
Yes | 5 (23.8%) | 55 (49.5%) | 60 (45.5%) |
Variable Name | Benign (n = 21) | Malignant (n = 111) | Total (n = 132) | Adjusted p-Value |
---|---|---|---|---|
arterial.ROI.intensity.average | 0 | 3 | 3 | 0.680 |
Mean (SD) | 105.033 (47.859) | 97.828 (42.567) | 99.001 (43.355) | |
ratio.tumor.cortex.arterial | 0 | 3 | 3 | 0.562 |
Missing data | 0 | 3 | 3 | |
Mean (SD) | 0.809 (0.361) | 0.847 (0.299) | 0.841 (0.309) | |
ratio.tumor.cortex.portal | 0 | 1 | 1 | 0.014 |
Missing data | 0 | 1 | 1 | |
Mean (SD) | 0.740 (0.157) | 0.612 (0.177) | 0.632 (0.180) | |
delta.intensity.average.AP | 0 | 3 | 3 | 0.024 |
Missing data | 0 | 3 | 3 | |
Mean (SD) | 0.600 (1.275) | 0.107 (0.537) | 0.187 (0.727) | |
delta.ratio.tumor.cortex.AP | 0 | 3 | 3 | 0.008 |
Missing data | 0 | 3 | 3 | |
Mean (SD) | 0.055 (0.492) | −0.217 (0.309) | −0.172 (0.357) |
Variable Name | Benign (n = 21) | Malignant (n = 111) | Total (n = 132) | Adjusted p-Value |
---|---|---|---|---|
X90th.discretized.intensity.percentile | 22.906 (3.446) | 20.435 (4.551) | 20.828 (4.475) | 0.025 |
Area.density…aligned.bounding.box | 0.544 (0.054) | 0.532 (0.042) | 0.534 (0.044) | 0.680 |
Area.density…convex.hull | 1.045 (0.086) | 1.027 (0.053) | 1.030 (0.060) | 0.933 |
Area.density…oriented.bounding.box | 0.572 (0.047) | 0.561 (0.036) | 0.563 (0.038) | 0.680 |
Center.of.mass.shift.cm. | 0.076 (0.061) | 0.112 (0.113) | 0.106 (0.107) | 0.360 |
Cluster.shade | −91.862 (274.983) | −80.688 (251.410) | −82.466 (254.236) | 0.680 |
Correlation | 0.675 (0.134) | 0.655 (0.149) | 0.658 (0.146) | 0.714 |
Flatness | 0.776 (0.105) | 0.763 (0.104) | 0.765 (0.104) | 0.680 |
Global.intensity.peak | 163.910 (40.505) | 140.206 (38.455) | 143.977 (39.597) | 0.025 |
Gray.level.variance..GLDZM. | 28.405 (8.614) | 24.046 (7.029) | 24.739 (7.442) | 0.105 |
High.dependence.high.gray.level.emphasis | 14,606.667 (10,457.428) | 11,350.721 (5853.078) | 11,868.712 (6847.744) | 0.632 |
High.dependence.low.gray.level.emphasis | 0.167 (0.295) | 0.479 (1.360) | 0.429 (1.257) | 0.140 |
Intensity.histogram.coefficient.of.variation | 0.214 (0.084) | 0.210 (0.064) | 0.210 (0.067) | 1.000 |
Intensity.mean.value | 118.633 (43.093) | 83.399 (25.630) | 89.005 (31.661) | 0.008 |
Intensity.based.interquartile.range..Original.Data. | 43.410 (14.082) | 41.619 (15.683) | 41.904 (15.402) | 0.599 |
Inverse.elongation | 0.860 (0.070) | 0.861 (0.096) | 0.861 (0.092) | 0.680 |
Large.distance.low.gray.level.emphasis | 0.204 (0.215) | 0.428 (0.665) | 0.392 (0.621) | 0.099 |
Local.intensity.peak | 135.500 (48.979) | 100.684 (40.422) | 106.223 (43.608) | 0.016 |
Max.value | 227.190 (50.281) | 242.550 (169.494) | 240.106 (156.655) | 0.180 |
Min.value..Original.Data. | −54.714 (59.865) | −68.604 (44.202) | −66.394 (47.051) | 0.714 |
Number.of.compartments.GMM. | 3.333 (1.390) | 3.333 (1.231) | 3.333 (1.252) | 1.000 |
Number.of.gray.levels | 218.524 (75.883) | 219.757 (102.488) | 219.561 (98.485) | 0.714 |
Skewness..Original.Data. | −0.258 (0.587) | 0.022 (1.025) | −0.023 (0.972) | 0.105 |
Small.distance.emphasis | 0.496 (0.135) | 0.433 (0.115) | 0.443 (0.120) | 0.099 |
Small.distance.high.gray.level.emphasis | 167.005 (50.245) | 135.899 (62.734) | 140.847 (61.810) | 0.051 |
Small.distance.low.gray.level.emphasis | 0.005 (0.005) | 0.004 (0.004) | 0.004 (0.004) | 0.714 |
Small.zone.emphasis | 0.586 (0.033) | 0.570 (0.031) | 0.573 (0.032) | 0.099 |
Spherical.disproportion | 1.128 (0.105) | 1.113 (0.112) | 1.115 (0.111) | 0.680 |
Volume.at.intensity.fraction.10. | 0.998 (0.004) | 0.999 (0.003) | 0.999 (0.003) | 0.714 |
Volume.at.intensity.fraction.90. | 0.003 (0.006) | 0.001 (0.002) | 0.002 (0.003) | 0.042 |
Volume.density…aligned.bounding.box | 0.466 (0.035) | 0.462 (0.042) | 0.463 (0.041) | 0.919 |
Volume.density…enclosing.ellipsoid | 0.976 (0.014) | 0.975 (0.022) | 0.975 (0.021) | 0.680 |
Volume.density…oriented.bounding.box | 0.504 (0.027) | 0.501 (0.036) | 0.502 (0.035) | 0.919 |
Volume.fraction.difference.between.intensity.fractions | 0.995 (0.009) | 0.997 (0.004) | 0.997 (0.005) | 0.105 |
Zone.size.entropy | 6.610 (0.303) | 6.598 (0.337) | 6.600 (0.330) | 0.919 |
Model | Accuracy | Sensitivity | Specificity | Precision | Brier Score | F1 Score | AUC |
---|---|---|---|---|---|---|---|
Rpart | 0.895 | 0.983 | 0.429 | 0.901 | 0.098 | 0.94 | 0.608 |
C5.0Tree | 0.861 | 0.956 | 0.362 | 0.888 | 0.117 | 0.921 | 0.736 |
Logit-Lasso | 0.855 | 0.966 | 0.267 | 0.874 | 0.119 | 0.918 | 0.721 |
RandomForest | 0.879 | 0.972 | 0.386 | 0.893 | 0.105 | 0.931 | 0.773 |
svmLinear | 0.852 | 0.954 | 0.314 | 0.88 | 0.106 | 0.916 | 0.81 |
wRpart | 0.765 | 0.815 | 0.5 | 0.896 | 0.19 | 0.854 | 0.654 |
wC5.0Tree | 0.867 | 0.95 | 0.424 | 0.897 | 0.114 | 0.923 | 0.739 |
wllasso | 0.811 | 0.865 | 0.524 | 0.906 | 0.159 | 0.885 | 0.705 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Garnier, C.; Ferrer, L.; Vargas, J.; Gallinato, O.; Jambon, E.; Le Bras, Y.; Bernhard, J.-C.; Colin, T.; Grenier, N.; Marcelin, C. A CT-Based Clinical, Radiological and Radiomic Machine Learning Model for Predicting Malignancy of Solid Renal Tumors (UroCCR-75). Diagnostics 2023, 13, 2548. https://doi.org/10.3390/diagnostics13152548
Garnier C, Ferrer L, Vargas J, Gallinato O, Jambon E, Le Bras Y, Bernhard J-C, Colin T, Grenier N, Marcelin C. A CT-Based Clinical, Radiological and Radiomic Machine Learning Model for Predicting Malignancy of Solid Renal Tumors (UroCCR-75). Diagnostics. 2023; 13(15):2548. https://doi.org/10.3390/diagnostics13152548
Chicago/Turabian StyleGarnier, Cassandre, Loïc Ferrer, Jennifer Vargas, Olivier Gallinato, Eva Jambon, Yann Le Bras, Jean-Christophe Bernhard, Thierry Colin, Nicolas Grenier, and Clément Marcelin. 2023. "A CT-Based Clinical, Radiological and Radiomic Machine Learning Model for Predicting Malignancy of Solid Renal Tumors (UroCCR-75)" Diagnostics 13, no. 15: 2548. https://doi.org/10.3390/diagnostics13152548