S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules
Abstract
:1. Introduction
2. Experimental Section
2.1. Patients
- -
- completely cystic lesions,
- -
- lesions with eggshell calcifications,
- -
- lesions with indeterminate (category III or IV according to the Bethesda classification) or non-diagnostic cytology results (category I according to the Bethesda classification), if the histopathological verification was not performed.
2.2. Methods
2.2.1. Ultrasound Examination
2.2.2. Statistical Analysis
2.2.3. Ethical Approval
3. Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Wolinski, K.; Stangierski, A.; Ruchala, M. Comparison of diagnostic yield of core-needle and fine-needle aspiration biopsies of thyroid lesions: Systematic review and meta-analysis. Eur. Radiol. 2017, 27, 431–436. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tumino, D.; Grani, G.; Di Stefano, M.; Di Mauro, M.; Scutari, M.; Rago, T.; Fugazzola, L.; Castagna, M.G.; Maino, F. Nodular Thyroid Disease in the Era of Precision Medicine. Front. Endocrinol. (Lausanne) 2019, 10, 907. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ruchala, M.; Szczepanek, E. Thyroid ultrasound—A piece of cake? Endokrynol. Pol. 2010, 61, 330–344. [Google Scholar] [PubMed]
- Szczepanek-Parulska, E.; Wolinski, K.; Stangierski, A.; Gurgul, E.; Biczysko, M.; Majewski, P.; Rewaj-Losyk, M.; Ruchala, M. Comparison of diagnostic value of conventional ultrasonography and shear wave elastography in the prediction of thyroid lesions malignancy. PLoS ONE 2013, 8, e81532. [Google Scholar] [CrossRef]
- Dobruch-Sobczak, K.; Migda, B.; Krauze, A.; Mlosek, K.; Slapa, R.Z.; Wareluk, P.; Bakula-Zalewska, E.; Adamczewski, Z.; Lewinski, A.; Jakubowski, W.; et al. Prospective analysis of inter-observer and intra-observer variability in multi ultrasound descriptor assessment of thyroid nodules. J. Ultrason. 2019, 19, 198–206. [Google Scholar] [CrossRef]
- Wolinski, K.; Szkudlarek, M.; Szczepanek-Parulska, E.; Ruchala, M. Usefulness of different ultrasound features of malignancy in predicting the type of thyroid lesions: A meta-analysis of prospective studies. Pol. Arch. Med. Wewn. 2014, 124, 97–104. [Google Scholar] [CrossRef] [Green Version]
- Russ, G.; Bonnema, S.J.; Erdogan, M.F.; Durante, C.; Ngu, R.; Leenhardt, L. European Thyroid Association Guidelines for Ultrasound Malignancy Risk Stratification of Thyroid Nodules in Adults: The EU-TIRADS. Eur. Thyroid. J. 2017, 6, 225–237. [Google Scholar] [CrossRef] [Green Version]
- Jin, Z.; Zhu, Y.; Zhang, S.; Xie, F.; Zhang, M.; Zhang, Y.; Tian, X.; Zhang, J.; Luo, Y.; Cao, J. Ultrasound Computer-Aided Diagnosis (CAD) Based on the Thyroid Imaging Reporting and Data System (TI-RADS) to Distinguish Benign from Malignant Thyroid Nodules and the Diagnostic Performance of Radiologists with Different Diagnostic Experience. Med. Sci. Monit. 2020, 26, e918452. [Google Scholar] [CrossRef]
- Yoo, Y.J.; Ha, E.J.; Cho, Y.J.; Kim, H.L.; Han, M.; Kang, S.Y. Computer-Aided Diagnosis of Thyroid Nodules via Ultrasonography: Initial Clinical Experience. Korean J. Radiol. 2018, 19, 665–672. [Google Scholar] [CrossRef]
- Luo, J.; Huang, F.; Zhou, P.; Chen, J.; Sun, Y.; Xu, F.; Wu, L.; Huang, P. Is ultrasound combined with computed tomography useful for distinguishing between primary thyroid lymphoma and Hashimoto’s thyroiditis? Endokrynol. Pol. 2019, 70, 463–468. [Google Scholar] [CrossRef] [Green Version]
- Dobruch-Sobczak, K.S.; Krauze, A.; Migda, B.; Mlosek, K.; Slapa, R.Z.; Bakula-Zalewska, E.; Adamczewski, Z.; Lewinski, A.; Jakubowski, W.; Dedecjus, M. Integration of Sonoelastography Into the TIRADS Lexicon Could Influence the Classification. Front. Endocrinol. (Lausanne) 2019, 10, 127. [Google Scholar] [CrossRef]
- Dobruch-Sobczak, K.; Adamczewski, Z.; Szczepanek-Parulska, E.; Migda, B.; Wolinski, K.; Krauze, A.; Prostko, P.; Ruchala, M.; Lewinski, A.; Jakubowski, W.; et al. Histopathological Verification of the Diagnostic Performance of the EU-TIRADS Classification of Thyroid Nodules-Results of a Multicenter Study Performed in a Previously Iodine-Deficient Region. J. Clin. Med. 2019, 8, 1781. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Castellana, M.; Grani, G.; Radzina, M.; Guerra, V.; Giovanella, L.; Deandrea, M.; Ngu, R.; Durante, C.; Trimboli, P. Performance of EU-TIRADS in malignancy risk stratification of thyroid nodules. A meta-analysis. Eur. J. Endocrinol. 2020. [Google Scholar] [CrossRef] [PubMed]
- Chambara, N.; Ying, M. The Diagnostic Efficiency of Ultrasound Computer-Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis. Cancers (Basel) 2019, 11, 1759. [Google Scholar] [CrossRef] [Green Version]
- Zhao, W.J.; Fu, L.R.; Huang, Z.M.; Zhu, J.Q.; Ma, B.Y. Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis. Medicine (Baltimore) 2019, 98, e16379. [Google Scholar] [CrossRef]
- Barczynski, M.; Stopa-Barczynska, M.; Wojtczak, B.; Czarniecka, A.; Konturek, A. Clinical validation of S-Detect(TM) mode in semi-automated ultrasound classification of thyroid lesions in surgical office. Gland Surg. 2020, 9, S77–S85. [Google Scholar] [CrossRef]
- Migda, B.; Migda, M.; Migda, M.S.; Slapa, R.Z. Use of the Kwak Thyroid Image Reporting and Data System (K-TIRADS) in differential diagnosis of thyroid nodules: Systematic review and meta-analysis. Eur. Radiol. 2018, 28, 2380–2388. [Google Scholar] [CrossRef] [Green Version]
- Fresilli, D.; Grani, G.; De Pascali, M.L.; Alagna, G.; Tassone, E.; Ramundo, V.; Ascoli, V.; Bosco, D.; Biffoni, M.; Bononi, M.; et al. Computer-aided diagnostic system for thyroid nodule sonographic evaluation outperforms the specificity of less experienced examiners. J. Ultrasound. 2020, 23, 169–174. [Google Scholar] [CrossRef]
- Migda, B.; Migda, M.; Migda, A.M.; Bierca, J.; Slowniska-Srzednicka, J.; Jakubowski, W.; Slapa, R.Z. Evaluation of Four Variants of the Thyroid Imaging Reporting and Data System (TIRADS) Classification in Patients with Multinodular Goitre—Initial study. Endokrynol. Pol. 2018, 69, 156–162. [Google Scholar] [CrossRef] [Green Version]
- Grani, G.; Lamartina, L.; Cantisani, V.; Maranghi, M.; Lucia, P.; Durante, C. Interobserver agreement of various thyroid imaging reporting and data systems. Endocr. Connect. 2018, 7, 1–7. [Google Scholar] [CrossRef]
- Schenke, S.; Klett, R.; Seifert, P.; Kreissl, M.C.; Gorges, R.; Zimny, M. Diagnostic Performance of Different Thyroid Imaging Reporting and Data Systems (Kwak-TIRADS, EU-TIRADS and ACR TI-RADS) for Risk Stratification of Small Thyroid Nodules (≤10 mm). J. Clin. Med. 2020, 9, 236. [Google Scholar] [CrossRef] [Green Version]
- Skowronska, A.; Milczarek-Banach, J.; Wiechno, W.; Chudzinski, W.; Zach, M.; Mazurkiewicz, M.; Miskiewicz, P.; Bednarczuk, T. Accuracy of the European Thyroid Imaging Reporting and Data System (EU-TIRADS) in the valuation of thyroid nodule malignancy in reference to the post-surgery histological results. Pol. J. Radiol. 2018, 83, e579–e586. [Google Scholar] [CrossRef] [PubMed]
- Kim, P.H.; Suh, C.H.; Baek, J.H.; Chung, S.R.; Choi, Y.J.; Lee, J.H. Diagnostic Performance of Four Ultrasound Risk Stratification Systems: A Systematic Review and Meta-Analysis. Thyroid 2020. [Google Scholar] [CrossRef] [PubMed]
- Gitto, S.; Grassi, G.; De Angelis, C.; Monaco, C.G.; Sdao, S.; Sardanelli, F.; Sconfienza, L.M.; Mauri, G. A computer-aided diagnosis system for the assessment and characterization of low-to-high suspicion thyroid nodules on ultrasound. Radiol. Med. 2019, 124, 118–125. [Google Scholar] [CrossRef]
- Kim, H.L.; Ha, E.J.; Han, M. Real-World Performance of Computer-Aided Diagnosis System for Thyroid Nodules Using Ultrasonography. Ultrasound. Med. Biol. 2019, 45, 2672–2678. [Google Scholar] [CrossRef]
- Jeong, E.Y.; Kim, H.L.; Ha, E.J.; Park, S.Y.; Cho, Y.J.; Han, M. Computer-aided diagnosis system for thyroid nodules on ultrasonography: Diagnostic performance and reproducibility based on the experience level of operators. Eur. Radiol. 2019, 29, 1978–1985. [Google Scholar] [CrossRef]
- Chung, S.R.; Baek, J.H.; Lee, M.K.; Ahn, Y.; Choi, Y.J.; Sung, T.Y.; Song, D.E.; Kim, T.Y.; Lee, J.H. Computer-Aided Diagnosis System for the Evaluation of Thyroid Nodules on Ultrasonography: Prospective Non-Inferiority Study according to the Experience Level of Radiologists. Korean J. Radiol. 2020, 21, 369–376. [Google Scholar] [CrossRef]
- Park, V.Y.; Han, K.; Seong, Y.K.; Park, M.H.; Kim, E.K.; Moon, H.J.; Yoon, J.H.; Kwak, J.Y. Diagnosis of Thyroid Nodules: Performance of a Deep Learning Convolutional Neural Network Model vs. Radiologists. Sci. Rep. 2019, 9, 17843. [Google Scholar] [CrossRef]
- Choi, Y.J.; Baek, J.H.; Park, H.S.; Shim, W.H.; Kim, T.Y.; Shong, Y.K.; Lee, J.H. A Computer-Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment. Thyroid 2017, 27, 546–552. [Google Scholar] [CrossRef]
- Xia, S.; Yao, J.; Zhou, W.; Dong, Y.; Xu, S.; Zhou, J.; Zhan, W. A computer-aided diagnosing system in the evaluation of thyroid nodules-experience in a specialized thyroid center. World J. Surg. Oncol. 2019, 17, 210. [Google Scholar] [CrossRef]
Sensitivity | Specificity | PPV | NPV | Accuracy | ||||||
---|---|---|---|---|---|---|---|---|---|---|
Groups | Value | 95% CI | Value | 95% CI | Value | 95% CI | Value | 95% CI | Value | 95% CI |
S-Detect | 89.4 | 79.4–95.6 | 80.6 | 69.1–89.2 | 81.9 | 73.5–88.2 | 88.5 | 79.1–94.0 | 85.0 | 77.7–90.6 |
EU-TIRADS (4 and 5 points) | 90.9 | 81.3–96.6 | 61.2 | 48.5–72.9 | 69.8 | 62.9–75.9 | 87.2 | 75.7–93.8 | 75.9 | 67.8–82.9 |
EU-TIRADS (5 points) | 80.0 | 68.7–88.6 | 79.4 | 67.9–88.3 | 80.0 | 71.2–86.6 | 79.4 | 70.4–86.2 | 79.7 | 72.0–86.1 |
MODEL 1 | 84.9 | 73–92.5 | 88.1 | 77.8–94.7 | 87.5 | 78.4–93.1 | 85.5 | 76.8–91.3 | 86.5 | 79.5–91.8 |
MODEL 2 | 95.5 | 87.3–99.1 | 53.7 | 41.1–66.0 | 67.0 | 61.0–72.6 | 92.3 | 79.5–97.4 | 74.4 | 66.2–81.6 |
MODEL 3 | 93.9 | 85.2–98.3 | 73.1 | 60.9–83.2 | 77.5 | 69.8–83.7 | 92.5 | 82.4–97.0 | 83.5 | 76.0–89.3 |
MODEL 4 | 78.9 | 67.0–87.9 | 89.6 | 79.7–95.7 | 88.1 | 78.5–93.8 | 81.1 | 72.8–87.3 | 84.2 | 76.9–90.0 |
MODEL 5 | 93.9 | 85.2–98.3 | 80.6 | 69.1–89.2 | 82.7 | 74.5–88.6 | 93.1 | 83.8–97.2 | 87.2 | 80.3–92.4 |
EU-TIRADS Scale | S-Detect Classification | ||
---|---|---|---|
MODEL 1 | 4 or 5 points | AND | “possibly malignant” |
MODEL 2 | 4 or 5 points | OR | “possibly malignant” |
MODEL 3 | 5 points | - | - |
3 or 4 points | AND | “possibly malignant” | |
MODEL 4 | 5 points | AND | “possibly malignant” |
MODEL 5 | 5 points | OR | “possibly malignant” |
Number of Patients | Number of Nodules | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | Accuracy (%) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
CAD | Staff | CAD | Staff | CAD | Staff | CAD | Staff | CAD | Staff | |||
Current study *, ** | 88 | 133 | 89.4 | 90.9 | 80.6 | 61.2 | 81.9 | 69.8 | 88.5 | 87.2 | 85.0 | 75.9 |
Gitto S et al. | 62 | 62 | 21.4 | 78.6 | 81.3 | 66.7 | 25.0 | 40.7 | 78 | 91.4 | 67.7 | 69.4 |
Kim HL et al. * | 106 | 218 | 81.4 | 84.9 | 68.2 | 96.2 | 62.5 | 93.6 | 84.9 | 90.7 | 73.4 | 91.7 |
Jeong EY et al. | 85 | 100 | 88.6 | 84.1 | 83.9 | 96.4 | 81.3 | 94.9 | 90.4 | 88.5 | 86.0 | 91.0 |
Chung SR et al. ** | 197 | 197 | 92.0 | 84.0 | 87.9 | 97.9 | 57.5 | 87.5 | 98.4 | 97.2 | 88.5 | 95.8 |
Park VY et al. | 265 | 286 | 90.4–91.0 | 94.2 | 58.5–80.0 | 76.9 | 72.3–84.5 | 83.1 | 83.5–88.1 | 91.7 | 75.9–86.0 | 86.4 |
Choi YJ et al. | 89 | 102 | 90.7 | 88.4 | 74.6 | 94.9 | 72.2 | 92.7 | 91.7 | 91.8 | 81.4 | 92.2 |
Xia S et al. | 171 | 180 | 90.5 | 81.1 | 41.2 | 88.5 | 63.2 | 6.7 | 79.5 | 95.9 | 67.2 | 60.9 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Szczepanek-Parulska, E.; Wolinski, K.; Dobruch-Sobczak, K.; Antosik, P.; Ostalowska, A.; Krauze, A.; Migda, B.; Zylka, A.; Lange-Ratajczak, M.; Banasiewicz, T.; et al. S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules. J. Clin. Med. 2020, 9, 2495. https://doi.org/10.3390/jcm9082495
Szczepanek-Parulska E, Wolinski K, Dobruch-Sobczak K, Antosik P, Ostalowska A, Krauze A, Migda B, Zylka A, Lange-Ratajczak M, Banasiewicz T, et al. S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules. Journal of Clinical Medicine. 2020; 9(8):2495. https://doi.org/10.3390/jcm9082495
Chicago/Turabian StyleSzczepanek-Parulska, Ewelina, Kosma Wolinski, Katarzyna Dobruch-Sobczak, Patrycja Antosik, Anna Ostalowska, Agnieszka Krauze, Bartosz Migda, Agnieszka Zylka, Malgorzata Lange-Ratajczak, Tomasz Banasiewicz, and et al. 2020. "S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules" Journal of Clinical Medicine 9, no. 8: 2495. https://doi.org/10.3390/jcm9082495
APA StyleSzczepanek-Parulska, E., Wolinski, K., Dobruch-Sobczak, K., Antosik, P., Ostalowska, A., Krauze, A., Migda, B., Zylka, A., Lange-Ratajczak, M., Banasiewicz, T., Dedecjus, M., Adamczewski, Z., Slapa, R. Z., Mlosek, R. K., Lewinski, A., & Ruchala, M. (2020). S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules. Journal of Clinical Medicine, 9(8), 2495. https://doi.org/10.3390/jcm9082495