An Explainable Classification Method of SPECT Myocardial Perfusion Images in Nuclear Cardiology Using Deep Learning and Grad-CAM
Abstract
:1. Introduction
2. Literature
3. Materials and Methods
3.1. CAD Dataset
3.2. Research Methodology
3.2.1. Convolutional Neural Networks: Main Aspects
3.2.2. Methodological Framework
- Step 1: Loading dataset
- Step 2: Data preparation
- Data normalization: Data normalization is a common technique in ML classification tasks. This method rescales pixel values by transforming them to the range [0, 1]. This process contributes to the discard of outliers and the effective reduction of computation time.
- Data shuffle: Before inserting into the algorithm, data has to be shuffled so that the extraction of patterns is as unbiased as possible. Therefore, the data shuffle technique was deployed to provide a random order of data insertion.
- Data split: We split the dataset into three parts: validation, training, and testing. More specifically, 15% of the entire dataset was given to testing and the remaining 85% was split into 20% for validation and 80% for training.
- Step 3: Training
- Data augmentation: Data augmentation is usually employed to increase the small number of datasets. It artificially generates various versions of the existing dataset, utilizing specific data augmentation techniques. In our case, we selected flipping and scaling strategies to achieve generalization and avoid overfitting [33].
- Define CNN architecture and activation functions: A detailed analysis was conducted to determine the best CNN architecture. During the experimentation, various values were applied for image size (pixels), batch size, number of nodes and layers for convolutional layers, and number of nodes for fully connected layers. The selection of the activation function is highly crucial since it corresponds to the type of classification problem. For example, the sigmoid function is proposed in binary classification problems, extracting values between 0 and 1 based on a default threshold that categorizes images. On the other hand, softmax is applied in multi-class classification problems, providing probabilities for each possible output, the sum of which is equal to 1 [33,34]
- Train CNN: In the training process, the gradient backward propagation technique was utilized to find the minimized error by adjusting the weights. In this process, CNN also extracted patterns from input images, which will be used in future classification tasks with unknown data. Furthermore, the loss function and the optimizer must be selected for training the CNN.
- Step 4: Validation
- Step 5: Testing
- Step 6: Explainability through Grad-CAM
- Step 7: Inference phase
4. Results
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
References
- Naghavi, M.; Abajobir, A.A.; Abbafati, C.; Abbas, K.M.; Abd-Allah, F.; Abera, S.F.; Aboyans, V.; Adetokunboh, O.; Afshin, A.; Agrawal, A.; et al. Global, Regional, and National Age-Sex Specific Mortality for 264 Causes of Death, 1980–2016: A Systematic Analysis for the Global Burden of Disease Study 2016. Lancet 2017, 390, 1151–1210. [Google Scholar] [CrossRef] [Green Version]
- Mendis, S.; Puska, P.; Norrving, B.; World Health Organization; World Heart Federation. World Stroke Organization Global Atlas on Cardiovascular Disease Prevention and Control/Edited by: Shanthi Mendis; World Heart Federation: Geneva, Switzerland, 2011. [Google Scholar]
- Hammad, M.; Alkinani, M.H.; Gupta, B.B.; Abd El-Latif, A.A. Myocardial Infarction Detection Based on Deep Neural Network on Imbalanced Data. Multimed. Syst. 2021, 1–13. [Google Scholar] [CrossRef]
- Chen, J.-J.; Su, T.-Y.; Chen, W.-S.; Chang, Y.-H.; Lu, H.H.-S. Convolutional Neural Network in the Evaluation of Myocardial Ischemia from CZT SPECT Myocardial Perfusion Imaging: Comparison to Automated Quantification. Appl. Sci. 2021, 11, 514. [Google Scholar] [CrossRef]
- Papandrianos, N.; Papageorgiou, E. Automatic Diagnosis of Coronary Artery Disease in SPECT Myocardial Perfusion Imaging Employing Deep Learning. Appl. Sci. 2021, 11, 6362. [Google Scholar] [CrossRef]
- Liu, H.; Wu, J.; Miller, E.J.; Liu, C.; Yaqiang, N.; Liu, N.; Liu, Y.-H. Diagnostic Accuracy of Stress-Only Myocardial Perfusion SPECT Improved by Deep Learning. Eur. J. Nucl. Med. Mol. Imaging 2021, 48, 2793–2800. [Google Scholar] [CrossRef]
- Mostafapour, S.; Gholamiankhah, F.; Maroofpour, S.; Momennezhad, M.; Asadinezhad, M.; Zakavi, S.R.; Arabi, H. Deep Learning-Based Attenuation Correction in the Image Domain for Myocardial Perfusion SPECT Imaging. arXiv 2021, arXiv:2102.04915. [Google Scholar]
- Kaplan Berkaya, S.; Ak Sivrikoz, I.; Gunal, S. Classification Models for SPECT Myocardial Perfusion Imaging. Comput. Biol. Med. 2020, 123, 103893. [Google Scholar] [CrossRef] [PubMed]
- Ntakolia, C.; Diamantis, D.E.; Papandrianos, N.; Moustakidis, S.; Papageorgiou, E.I. A Lightweight Convolutional Neural Network Architecture Applied for Bone Metastasis Classification in Nuclear Medicine: A Case Study on Prostate Cancer Patients. Healthcare 2020, 8, 493. [Google Scholar] [CrossRef]
- Papandrianos, N.; Papageorgiou, E.I.; Anagnostis, A. Development of Convolutional Neural Networks to Identify Bone Metastasis for Prostate Cancer Patients in Bone Scintigraphy. Ann. Nucl. Med. 2020, 34, 824–832. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep Learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Arabi, H.; AkhavanAllaf, A.; Sanaat, A.; Shiri, I.; Zaidi, H. The Promise of Artificial Intelligence and Deep Learning in PET and SPECT Imaging. Phys. Medica PM Int. J. Devoted Appl. Phys. Med. Biol. Off. J. Ital. Assoc. Biomed. Phys. AIFB 2021, 83, 122–137. [Google Scholar] [CrossRef]
- Nazari, M.; Kluge, A.; Apostolova, I.; Klutmann, S.; Kimiaei, S.; Schroeder, M.; Buchert, R. Explainable AI to Improve Acceptance of Convolutional Neural Networks for Automatic Classification of Dopamine Transporter SPECT in the Diagnosis of Clinically Uncertain Parkinsonian Syndromes. Eur. J. Nucl. Med. Mol. Imaging 2022, 49, 1176–1186. [Google Scholar] [CrossRef]
- Savvopoulos, C.A.; Spyridonidis, T.; Papandrianos, N.; Vassilakos, P.J.; Alexopoulos, D.; Apostolopoulos, D.J. CT-Based Attenuation Correction in Tl-201 Myocardial Perfusion Scintigraphy Is Less Effective than Non-Corrected SPECT for Risk Stratification. J. Nucl. Cardiol. 2014, 21, 519–531. [Google Scholar] [CrossRef] [PubMed]
- Anaya-Isaza, A.; Mera-Jiménez, L.; Zequera-Diaz, M. An Overview of Deep Learning in Medical Imaging. Inform. Med. Unlocked 2021, 26, 100723. [Google Scholar] [CrossRef]
- Singh, A.; Sengupta, S.; Lakshminarayanan, V. Explainable Deep Learning Models in Medical Image Analysis. J. Imaging 2020, 6, 52. [Google Scholar] [CrossRef] [PubMed]
- Gunning, D.; Stefik, M.; Choi, J.; Miller, T.; Stumpf, S.; Yang, G.-Z. XAI-Explainable Artificial Intelligence. Sci. Robot. 2019, 4, eaay7120. [Google Scholar] [CrossRef] [Green Version]
- Otaki, Y.; Singh, A.; Kavanagh, P.; Miller, R.J.H.; Parekh, T.; Tamarappoo, B.K.; Sharir, T.; Einstein, A.J.; Fish, M.B.; Ruddy, T.D.; et al. Clinical Deployment of Explainable Artificial Intelligence of SPECT for Diagnosis of Coronary Artery Disease. JACC Cardiovasc. Imaging 2021, 4, 30. [Google Scholar] [CrossRef]
- Betancur, J.; Hu, L.-H.; Commandeur, F.; Sharir, T.; Einstein, A.J.; Fish, M.B.; Ruddy, T.D.; Kaufmann, P.A.; Sinusas, A.J.; Miller, E.J.; et al. Deep Learning Analysis of Upright-Supine High-Efficiency SPECT Myocardial Perfusion Imaging for Prediction of Obstructive Coronary Artery Disease: A Multicenter Study. J. Nucl. Med. 2019, 60, 664–670. [Google Scholar] [CrossRef] [PubMed]
- Betancur, J.; Commandeur, F.; Motlagh, M.; Sharir, T.; Einstein, A.J.; Bokhari, S.; Fish, M.B.; Ruddy, T.D.; Kaufmann, P.; Sinusas, A.J.; et al. Deep Learning for Prediction of Obstructive Disease From Fast Myocardial Perfusion SPECT. JACC Cardiovasc. Imaging 2018, 11, 1654–1663. [Google Scholar] [CrossRef]
- Zahiri, N.; Asgari, R.; Razavi-Ratki, S.-K.; Parach, A.-A. Deep Learning Analysis of Polar Maps from SPECT Myocardial Perfusion Imaging for Prediction of Coronary Artery Disease. Res. Sq. 2021. [Google Scholar] [CrossRef]
- Papandrianos, N.; Feleki, A.; Papageorgiou, E. Exploring Classification of SPECT MPI Images Applying Convolutional Neural Networks. In Proceedings of the 25th Pan-Hellenic Conference on Informatics, Association for Computing Machinery, New York, NY, USA, 26 August 2021; pp. 483–489. [Google Scholar] [CrossRef]
- Apostolopoulos, I.D.; Papathanasiou, N.D.; Spyridonidis, T.; Apostolopoulos, D.J. Automatic Characterization of Myocardial Perfusion Imaging Polar Maps Employing Deep Learning and Data Augmentation. Hell. J. Nucl. Med. 2020, 23, 125–132. [Google Scholar] [CrossRef] [PubMed]
- Apostolopoulos, I.D.; Apostolopoulos, D.I.; Spyridonidis, T.I.; Papathanasiou, N.D.; Panayiotakis, G.S. Multi-Input Deep Learning Approach for Cardiovascular Disease Diagnosis Using Myocardial Perfusion Imaging and Clinical Data. Phys. Med. PM Int. J. Devoted Appl. Phys. Med. Biol. Off. J. Ital. Assoc. Biomed. Phys. AIFB 2021, 84, 168–177. [Google Scholar] [CrossRef] [PubMed]
- de Souza Filho, E.M.; Fernandes, F.d.A.; Wiefels, C.; de Carvalho, L.N.D.; Dos Santos, T.F.; Dos Santos, A.A.S.M.D.; Mesquita, E.T.; Seixas, F.L.; Chow, B.J.W.; Mesquita, C.T.; et al. Machine Learning Algorithms to Distinguish Myocardial Perfusion SPECT Polar Maps. Front. Cardiovasc. Med. 2021, 8, 741667. [Google Scholar] [CrossRef]
- Nakajima, K.; Kudo, T.; Nakata, T.; Kiso, K.; Kasai, T.; Taniguchi, Y.; Matsuo, S.; Momose, M.; Nakagawa, M.; Sarai, M.; et al. Diagnostic Accuracy of an Artificial Neural Network Compared with Statistical Quantitation of Myocardial Perfusion Images: A Japanese Multicenter Study. Eur. J. Nucl. Med. Mol. Imaging 2017, 44, 2280–2289. [Google Scholar] [CrossRef] [Green Version]
- Ciecholewski, M. Ischemic Heart Disease Detection Using Selected Machine Learning Methods. Int. J. Comput. Math. 2012, 90, 8. [Google Scholar] [CrossRef]
- Otaki, Y.; Tamarappoo, B.; Singh, A.; Sharir, T.; Hu, L.-H.; Gransar, H.; Einstein, A.; Fish, M.; Ruddy, T.; Kaufmann, P.; et al. Diagnostic Accuracy of Deep Learning for Myocardial Perfusion Imaging in Men and Women with a High-Efficiency Parallel-Hole-Collimated Cadmium-Zinc-Telluride Camera: Multicenter Study. J. Nucl. Med. 2020, 61, 92. [Google Scholar]
- Spier, N.; Nekolla, S.; Rupprecht, C.; Mustafa, M.; Navab, N.; Baust, M. Classification of Polar Maps from Cardiac Perfusion Imaging with Graph-Convolutional Neural Networks. Sci. Rep. 2019, 9, 7569. [Google Scholar] [CrossRef] [Green Version]
- Magesh, P.R.; Myloth, R.D.; Tom, R.J. An Explainable Machine Learning Model for Early Detection of Parkinson’s Disease Using LIME on DaTscan Imagery. arXiv 2020, arXiv:200800238. [Google Scholar] [CrossRef] [PubMed]
- Kawauchi, K.; Hirata, K.; Katoh, C.; Ichikawa, S.; Manabe, O.; Kobayashi, K.; Watanabe, S.; Furuya, S.; Shiga, T. A Convolutional Neural Network-Based System to Prevent Patient Misidentification in FDG-PET Examinations. Sci. Rep. 2019, 9, 7192. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Domingues, I.; Pereira, G.; Martins, P.; Duarte, H.; Santos, J.; Abreu, P.H. Using Deep Learning Techniques in Medical Imaging: A Systematic Review of Applications on CT and PET. Artif. Intell. Rev. 2020, 53, 4093–4160. [Google Scholar] [CrossRef]
- Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.W.M.; van Ginneken, B.; Sánchez, C.I. A Survey on Deep Learning in Medical Image Analysis. Med. Image Anal. 2017, 42, 60–88. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Christodoulou, E.; Moustakidis, S.; Papandrianos, N.; Tsaopoulos, D.; Papageorgiou, E. Exploring Deep Learning Capabilities in Knee Osteoarthritis Case Study for Classification. In Proceedings of the IEEE 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), Patras, Greece, 15–17 July 2019; pp. 1–6. [Google Scholar]
- An, J.; Joe, I. Attention Map-Guided Visual Explanations for Deep Neural Networks. Appl. Sci. 2022, 12, 3846. [Google Scholar] [CrossRef]
- Lizzi, F.; Scapicchio, C.; Laruina, F.; Retico, A.; Fantacci, M.E. Convolutional Neural Networks for Breast Density Classification: Performance and Explanation Insights. Appl. Sci. 2022, 12, 148. [Google Scholar] [CrossRef]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int. J. Comput. Vis. 2020, 128, 336–359. [Google Scholar] [CrossRef] [Green Version]
- Selvaraju, R.R.; Das, A.; Vedantam, R.; Cogswell, M.; Parikh, D.; Batra, D. Grad-CAM: Why Did You Say That? arXiv 2017, arXiv:1611.07450. [Google Scholar]
- Chen, X.; Zhou, B.; Xie, H.; Shi, L.; Liu, H.; Holler, W.; Lin, M.; Liu, Y.-H.; Miller, E.J.; Sinusas, A.J.; et al. Direct and Indirect Strategies of Deep-Learning-Based Attenuation Correction for General Purpose and Dedicated Cardiac SPECT. Eur. J. Nucl. Med. Mol. Imaging 2022, 49, 3046–3060. [Google Scholar] [CrossRef] [PubMed]
- Xiao, M.; Zhang, L.; Shi, W.; Liu, J.; He, W.; Jiang, Z. A Visualization Method Based on the Grad-CAM for Medical Image Segmentation Model. In Proceedings of the IEEE 2021 International Conference on Electronic Information Engineering and Computer Science (EIECS), Changchun, China, 23 September 2021; pp. 242–247. [Google Scholar]
- Zhang, Y.C.; Kagen, A.C. Machine Learning Interface for Medical Image Analysis. J. Digit. Imaging 2017, 30, 615–621. [Google Scholar] [CrossRef] [PubMed]
- Papandrianos, N.I.; Feleki, A.; Papageorgiou, E.I.; Martini, C. Deep Learning-Based Automated Diagnosis for Coronary Artery Disease Using SPECT-MPI Images. J. Clin. Med. 2022, 11, 3918. [Google Scholar] [CrossRef] [PubMed]
Clinical Characteristics | Frequency |
---|---|
No | 625 |
Age (mean ± SD) | 62.2 ± 7.8 years |
Sex (male/female) | 65.88%/34.22% |
History of CAD | 40.89% |
Previous Myocardial Infraction | 35.48% |
Previous Stroke | 13.46% |
Hypertension | 70.25% |
Smoking | 58.18% |
Diabetes | 40.56% |
Batch Size | Val Acc (%) | Val Loss | Test Acc (%) | Test Loss | AUC [CI 95%] |
---|---|---|---|---|---|
8 | 87.7 | 0.32 | 84.58 | 0.4 | 0.86 [0.882–0.958] |
16 | 88.54 | 0.39 | 90.62 | 0,26 | 0.92 [0.911–0.977] |
32 | 94.58 | 0.18 | 93.33 | 0.21 | 0.94 [0.919–0.981] |
64 | 84.76 | 0.4 | 82.81 | 0.54 | 0.87 [0.827–0.921] |
Image Size | Convolutional | Dense | Val Acc (%) | Val Loss | Test Acc (%) | Test Loss | AUC [CI 95%] | Sens | Spec |
---|---|---|---|---|---|---|---|---|---|
250 × 250 | 16–32–64 | 128–128 | 92.7 | 0.22 | 89.45 | 0.24 | 0.91 [0.891–0.972] | 0.87 | 0.98 |
250 × 250 | 16–32–64–128 | 128–128 | 90.88 | 0.25 | 90.62 | 0.24 | 0.91 [0.923–0.981] | 1 | 0.97 |
250 × 250 | 16–32–64–128–256 | 128–128 | 92.31 | 0.14 | 92.18 | 0.2 | 0.92 [0.853–0.94] | 0.93 | 0.93 |
300 × 300 | 16–32–64 | 128–128 | 89.84 | 0.31 | 87.89 | 0.35 | 0.9 [0.828–0.921] | 0.93 | 0.9 |
300 × 300 | 16–32–64–128 | 128–128 | 94.58 | 0.18 | 93.33 | 0.21 | 0.9458 [0.938–0.993] | 0.93 | 1 |
300 × 300 | 16–32–64–128–256 | 128–128 | 92.58 | 0.23 | 91.33 | 0.21 | 0.91 [0.936–0.986] | 1 | 0.91 |
350 × 350 | 16–32–64 | 128–128 | 92.96 | 0.19 | 92.14 | 0.22 | 0.92 [0.881–0.957] | 0.9 | 0.97 |
350 × 350 | 16–32–64–128 | 128–128 | 92.03 | 0.17 | 92.16 | 0.17 | 0.93 [0.896–0.976] | 0.875 | 0.97 |
350 × 350 | 16–32–64–128–256 | 128–128 | 93.4 | 0.15 | 92.7 | 0.17 | 0.92 [0.952–0.993] | 1.0 | 0.94 |
Runs | Accuracy (%) | Loss | AUC | CI (95%) | Sens | Spec |
---|---|---|---|---|---|---|
Run 1 | 92.3 | 0.17 | 0.92 | [0.938–0.986] | 1 | 0.94 |
Run 2 | 92.3 | 0.2 | 0.93 | [0.918–0.981] | 0.98 | 0.95 |
Run 3 | 92.3 | 0.27 | 0.94 | [0.916–0.98] | 0.97 | 0.97 |
Run 4 | 94.23 | 0.2 | 0.93 | [0.935–0.985] | 1 | 0.97 |
Run 5 | 94.23 | 0.15 | 0.92 | [0.940–0.987] | 1 | 0.92 |
Run 6 | 98.07 | 0.17 | 0.93 | [0.924–0.979] | 1 | 0.91 |
Run 7 | 92.3 | 0.18 | 0.95 | [0.922–0.978] | 1 | 0.94 |
Run 8 | 94.23 | 0.06 | 0.96 | [0.933–0.987] | 1 | 0.97 |
Run 9 | 94.11 | 0.16 | 0.95 | [0.949–0.993] | 1 | 0.97 |
Run 10 | 90.3 | 0.22 | 0.91 | [0.912–0.978] | 0.87 | 1 |
Average | 93.4 ± 2.45 | 0.17 | 0.93 | [0.929–0.983] | 0.982 | 0.954 |
Data Split (80%–20% Testing) | 10-Fold | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Test Acc | Test Loss | AUC [CI 95%] | Sens | Spec | Test Acc | Test Loss | AUC [CI 95%] | Sens | Spec | |
RGB-CNN | 93.33 | 0.21 | 0.94 [0.945–0.99] | 1 | 0.945 | 93.4 | 0.17 | 0.93 [0.929–0.983] | 0.982 | 0.954 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Papandrianos, N.I.; Feleki, A.; Moustakidis, S.; Papageorgiou, E.I.; Apostolopoulos, I.D.; Apostolopoulos, D.J. An Explainable Classification Method of SPECT Myocardial Perfusion Images in Nuclear Cardiology Using Deep Learning and Grad-CAM. Appl. Sci. 2022, 12, 7592. https://doi.org/10.3390/app12157592
Papandrianos NI, Feleki A, Moustakidis S, Papageorgiou EI, Apostolopoulos ID, Apostolopoulos DJ. An Explainable Classification Method of SPECT Myocardial Perfusion Images in Nuclear Cardiology Using Deep Learning and Grad-CAM. Applied Sciences. 2022; 12(15):7592. https://doi.org/10.3390/app12157592
Chicago/Turabian StylePapandrianos, Nikolaos I., Anna Feleki, Serafeim Moustakidis, Elpiniki I. Papageorgiou, Ioannis D. Apostolopoulos, and Dimitris J. Apostolopoulos. 2022. "An Explainable Classification Method of SPECT Myocardial Perfusion Images in Nuclear Cardiology Using Deep Learning and Grad-CAM" Applied Sciences 12, no. 15: 7592. https://doi.org/10.3390/app12157592