Enhancing Diagnostic Accuracy for Skin Cancer and COVID-19 Detection: A Comparative Study Using a Stacked Ensemble Method
Abstract
:1. Introduction
- Bagging, often referred to as bootstrap aggregating, serves as a commonly employed method for creating ensemble-based algorithms. The core concept behind bagging involves creating a set of independent datasets from the original data. Bagging introduces two key steps to the original models: firstly, the creation of bagging samples and their subsequent presentation to the base-learner models, and secondly, the approach to merge the predictions from multiple predictors.
- Boosting is an ML algorithm that involves training multiple models sequentially, with each subsequent model focusing on correcting the errors made by the previous model. Boosting stands as a robust approach that effectively mitigates overfitting [14].
- The stacked ensemble ML method is a technique that combines multiple classification methods that are homogeneous or heterogeneous, known as base learners, to produce a single superior-performing classification model. The key idea of this method is that the meta-learner generates its final predictions according to the base-learner predictions. The meta-learner is a model that aims to reduce the prediction mistakes of the base learners [15].
Contribution of Research Work
- To ensure a reliable evaluation of the proposed method, a customized distribution strategy is implemented for sampling each dataset. Departing from the standard data split method, the approach involves a balanced division of data at each stage of model development. This tailored strategy demonstrates its efficacy in enhancing the overall performance of the model.
- Additionally, the most effective and high-performing CNN variants with a default input size of 224 × 224 (DenseNet-121, ResNet101, VGG16, and a combination of these networks) are utilized as feature extractors. These pretrained models are configured to exclude their fully connected layers to make them suitable for feature extraction. The extracted high-level features from these models are combined as a feature vector that can be valuable for subsequent classification tasks and that also reduces model complexity due to the removal of computationally and memory-intensive fully connected layers.
- The generated feature vector is then further used as input of base-learner modelsto train them. Five different base-learner models (support vector machine (SVM), linear regression (LR), decision tree (DT), random forest (RF), and naive Bayes (NB)) are used in this work.
- The predictions from the base-learner models are integrated with the initial feature vector and the ensemble validation set to generate a very rich and informative fused vector. This fused feature vector is finally fed to the meta-learner model, which provides a very precise final prediction. Five different meta-learner models (RF, DT, LR, NB, and SVM) are compared by training on the same fused feature vector.
- Most of the state-of-the-art methods apply only the base-learner prediction as the input of the meta-learner. The proposed ensemble technique combines different types and levels of features to improve the generalization capability of the final predictors.
2. Related Work
2.1. COVID-19
2.2. Skin Cancer
2.3. COVID-19 and Skin Cancer
2.4. Ensemble Learning
2.5. Types of Ensemble Learning
- Bagging: It is the first effective and the simplest method of ensemble learning. It was originally designed for classification and is usually applied to decision tree models, but it can be used for regression. The method involves using multiple versions of a training set through the process of bootstrapping. Each of these datasets is used to train a different model. The outputs of the models are combined by averaging (for regression) or voting (for classification) to create a single output. Bagging is only effective for non-linear models.
- Boosting: This model-averaging method is a widely used ensemble technique. It can be used for both classification and regression tasks. In this method, weak classifiers are created iteratively, with each one trained on a dataset where misclassified points are given more weight. The final model is created by combining the outputs of the successive models using voting or averaging. Boosting can sometimes fail to generate a classifier as accurate as a single classifier built from the same data, indicating overfitting.
- Stacking: It is a distinct way of combining multiple models that introduces the concept of a meta-learner. It is less widely used compared to bagging and boosting. Stacking is normally used to combine models of different types. It splits the training set into two disjoint sets. The method trains several base learners on the first part and tests on the second part. The predictions of the base learners are taken as the inputs to train the meta-learners.
- Error-correcting output codes: It is a technique that enhances the performance of classification algorithms in multiclass learning scenarios. In this approach, the multiclass dataset is broken down into multiple independent two-class problems. The algorithm is then applied to each of these problems, and the outputs from the resulting classifiers are combined to make the final prediction.
3. Methodology Overview
3.1. Datasets
3.1.1. COVID-19 Radiography Database
3.1.2. Skin Lesion Images for Melanoma Classification
3.2. Data Splitting
3.3. Data Preprocessing
3.4. Feature Extraction Technique
3.5. Fusion Technique
3.6. Classification Models
3.7. Stacked Ensemble Learning
3.8. Performance Evaluation Metrics
4. Results and Analysis
5. Discussion
6. Conclusions
Future Work
- This work exclusively underwent testing on COVID-19 and skin cancer datasets. Therefore, in order to extend its applicability to further datasets, additional research and investigation are required.
- This study approach relied on five established ML models to construct this stacked ensemble model. Additionally, we utilized pretrained models with a fixed input size of for feature extraction. Nevertheless, in future research, there is potential to broaden the scope of this method by adapting it to various pretrained CNN architectures that may have different input sizes.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Kaliraman, B.; Duhan, M. A new hybrid approach for feature extraction and selection of electroencephalogram signals in case of person recognition. J. Reliab. Intell. Environ. 2021, 7, 241–251. [Google Scholar] [CrossRef]
- Umair, A.; Masciari, E.; Ullah, M.H. Vaccine sentiment analysis using BERT + NBSVM and geo-spatial approaches. J. Supercomput. 2023, 79, 17355–17385. [Google Scholar] [CrossRef]
- Hovorushchenko, T.; Moskalenko, A.; Osyadlyi, V. Methods of medical data management based on blockchain technologies. J. Reliab. Intell. Environ. 2023, 9, 5–16. [Google Scholar] [CrossRef] [PubMed]
- Umair, A.; Masciari, E. Sentimental and spatial analysis of covid-19 vaccines tweets. J. Intell. Inf. Syst. 2023, 60, 1–21. [Google Scholar] [CrossRef] [PubMed]
- Naeem, M.; Coronato, A. An AI-empowered home-infrastructure to minimize medication errors. J. Sens. Actuator Netw. 2022, 11, 13. [Google Scholar] [CrossRef]
- Naeem, M.; Coronato, A.; Paragliola, G. Adaptive treatment assisting system for patients using machine learning. In Proceedings of the 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), Granada, Spain, 22–25 October 2019; IEEE: New York, NY, USA, 2019; pp. 460–465. [Google Scholar]
- Haseeb, A.; Khan, M.A.; Shehzad, F.; Alhaisoni, M.; Khan, J.A.; Kim, T.; Cha, J. Knee Osteoarthritis Classification Using X-Ray Images Based on Optimal Deep Neural Network. Comput. Syst. Sci. Eng. 2023, 47, 2397–2415. [Google Scholar] [CrossRef]
- Shah, S.I.H.; Coronato, A.; Naeem, M.; De Pietro, G. Learning and assessing optimal dynamic treatment regimes through cooperative imitation learning. IEEE Access 2022, 10, 78148–78158. [Google Scholar] [CrossRef]
- Gheisari, M.; Taami, T.; Ghaderzadeh, M.; Li, H.; Sadeghsalehi, H.; Sadeghsalehi, H.; Abbasi, A.A. Mobile applications in COVID-19 detection and diagnosis: An efficient tool to control the future pandemic; a multidimensional systematic review of the state of the art. JMIR MHealth UHealth 2023, 12, e44406. [Google Scholar]
- Fiorino, M.; Naeem, M.; Ciampi, M.; Coronato, A. Defining a Metric-Driven Approach for Learning Hazardous Situations. Technologies 2024, 12, 103. [Google Scholar] [CrossRef]
- Ghaderzadeh, M.; Asadi, F.; Ramezan Ghorbani, N.; Almasi, S.; Taami, T. Toward artificial intelligence (AI) applications in the determination of COVID-19 infection severity: Considering AI as a disease control strategy in future pandemics. Iran. J. Blood Cancer 2023, 15, 93–111. [Google Scholar] [CrossRef]
- Coronato, A.; Naeem, M. A reinforcement learning based intelligent system for the healthcare treatment assistance of patients with disabilities. In Proceedings of the International Symposium on Pervasive Systems, Algorithms and Networks, Naples, Italy, 16–20 September 2019; Springer: Cham, Switzerland, 2019; pp. 15–28. [Google Scholar]
- Xue, D.; Zhou, X.; Li, C.; Yao, Y.; Rahaman, M.M.; Zhang, J.; Chen, H.; Zhang, J.; Qi, S.; Sun, H. An Application of Transfer Learning and Ensemble Learning Techniques for Cervical Histopathology Image Classification. IEEE Access 2020, 8, 104603–104618. [Google Scholar] [CrossRef]
- Ganaie, M.A.; Hu, M.; Malik, A.; Tanveer, M.; Suganthan, P. Ensemble deep learning: A review. Eng. Appl. Artif. Intell. 2022, 115, 105151. [Google Scholar] [CrossRef]
- Shekar, B.; Hailu, H. An efficient stacked ensemble model for the detection of COVID-19 and skin cancer using fused feature of transfer learning and handcrafted methods. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2023, 11, 878–894. [Google Scholar] [CrossRef]
- Rajagopal, R. Comparative analysis of COVID-19 X-ray images classification using convolutional neural network, transfer learning, and machine learning classifiers using deep features. Pattern Recognit. Image Anal. 2021, 31, 313–322. [Google Scholar] [CrossRef]
- Shaik, N.S.; Cherukuri, T.K. Transfer learning based novel ensemble classifier for COVID-19 detection from chest CT-scans. Comput. Biol. Med. 2022, 141, 105127. [Google Scholar] [CrossRef]
- Grignaffini, F.; Barbuto, F.; Piazzo, L.; Troiano, M.; Simeoni, P.; Mangini, F.; Pellacani, G.; Cantisani, C.; Frezza, F. Machine Learning Approaches for Skin Cancer Classification from Dermoscopic Images: A Systematic Review. Algorithms 2022, 15, 438. [Google Scholar] [CrossRef]
- Tembhurne, J.V.; Hebbar, N.; Patil, H.Y.; Diwan, T. Skin cancer detection using ensemble of machine learning and deep learning techniques. Multimed. Tools Appl. 2023, 82, 1–24. [Google Scholar] [CrossRef]
- Rahman, Z.; Hossain, M.S.; Islam, M.R.; Hasan, M.M.; Hridhee, R.A. An approach for multiclass skin lesion classification based on ensemble learning. Inform. Med. Unlocked 2021, 25, 100659. [Google Scholar] [CrossRef]
- Müller, D.; Soto-Rey, I.; Kramer, F. An analysis on ensemble learning optimized medical image classification with deep convolutional neural networks. IEEE Access 2022, 10, 66467–66480. [Google Scholar] [CrossRef]
- Shinkai, K.; Bruckner, A.L. Dermatology and COVID-19. JAMA 2020, 324, 1133–1134. [Google Scholar] [CrossRef]
- Goyal, M.; Pandey, M. Ensemble-based data modeling for the prediction of energy consumption in hvac plants. J. Reliab. Intell. Environ. 2021, 7, 49–64. [Google Scholar] [CrossRef]
- Rincy, T.N.; Gupta, R. Ensemble learning techniques and its efficiency in machine learning: A survey. In Proceedings of the 2nd International Conference on Data, Engineering and Applications (IDEA), Bhopal, India, 28–29 February 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
- Witten, I.H.; Frank, E.; Hall, M.A.; Pal, C.J.; Data, M. Practical machine learning tools and techniques. In Proceedings of the Data Mining; Elsevier: Amsterdam, The Netherlands, 2005; Volume 2, pp. 403–413. [Google Scholar]
- Chowdhury, M.E.; Rahman, T.; Khandakar, A.; Mazhar, R.; Kadir, M.A.; Mahbub, Z.B.; Islam, K.R.; Khan, M.S.; Iqbal, A.; Al Emadi, N.; et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 2020, 8, 132665–132676. [Google Scholar] [CrossRef]
- Fanconi, C. Skin Cancer: Malignant vs. Benign-Processed Skin Cancer Pictures of the ISIC Archive. 2019. Available online: https://www.kaggle.com/datasets/fanconic/skin-cancer-malignant-vs-benign (accessed on 15 June 2024).
- Silva, P.; Luz, E.; Silva, G.; Moreira, G.; Silva, R.; Lucio, D.; Menotti, D. COVID-19 detection in CT images with deep learning: A voting-based scheme and cross-datasets analysis. Inform. Med. Unlocked 2020, 20, 100427. [Google Scholar] [CrossRef]
- Mahajan, P.; Uddin, S.; Hajati, F.; Moni, M.A. Ensemble Learning for Disease Prediction: A Review. Healthcare 2023, 11, 1808. [Google Scholar] [CrossRef] [PubMed]
Models | VGG16 | DENSENET121 | RESNET101 | DENSENET121 + RESNET101 | DENSENET121 + VGG16 | RESNET101 + VGG16 | DENSENET121 + RESNET101 + VGG16 |
---|---|---|---|---|---|---|---|
SVM | 95.63 | 91.72 | 95.40 | 92.64 | 94.25 | 91.49 | 94.48 |
LR | 96.78 | 94.71 | 96.09 | 95.40 | 96.55 | 94.48 | 95.86 |
DT | 83.91 | 87.13 | 85.06 | 88.74 | 82.53 | 86.44 | 81.15 |
RF | 92.18 | 91.03 | 92.18 | 91.03 | 91.03 | 91.95 | 89.89 |
NB | 91.26 | 84.83 | 90.34 | 90.57 | 91.26 | 90.34 | 91.03 |
Models | VGG16 | DENSENET121 | RESNET101 | DENSENET121 + RESNET101 | DENSENET121 + VGG16 | RESNET101 + VGG16 | DENSENET121 + RESNET101 + VGG16 |
---|---|---|---|---|---|---|---|
SVM | 97.01 | 95.17 | 97.24 | 94.94 | 96.78 | 96.78 | 96.32 |
LR | 97.01 | 94.71 | 96.09 | 95.40 | 96.55 | 96.09 | 96.32 |
DT | 94.02 | 91.03 | 95.17 | 89.43 | 94.25 | 90.57 | 91.49 |
RF | 96.55 | 93.56 | 96.78 | 94.02 | 96.55 | 94.48 | 93.79 |
NB | 96.05 | 92.64 | 93.33 | 90.57 | 93.33 | 90.34 | 95.63 |
Models | VGG16 | DENSENET121 | RESNET101 | DENSENET121 + RESNET101 | DENSENET121 + VGG16 | RESNET101 + VGG16 | DENSENET121 + RESNET101 + VGG16 |
---|---|---|---|---|---|---|---|
SVM | 83.40 | 82.39 | 86.64 | 81.17 | 83.60 | 82.19 | 80.16 |
LR | 82.19 | 83.00 | 87.85 | 84.82 | 80.77 | 84.41 | 81.98 |
DT | 70.04 | 75.91 | 71.05 | 69.84 | 71.26 | 72.27 | 76.32 |
RF | 80.97 | 81.98 | 84.21 | 82.79 | 81.17 | 83.81 | 78.95 |
NB | 79.35 | 77.94 | 79.51 | 78.74 | 78.14 | 78.14 | 76.52 |
Models | VGG16 | DENSENET121 | RESNET101 | DENSENET121 + RESNET101 | DENSENET121 + VGG16 | RESNET101 + VGG16 | DENSENET121 + RESNET101 + VGG16 |
---|---|---|---|---|---|---|---|
SVM | 84.50 | 83.28 | 86.79 | 82.37 | 83.60 | 84.19 | 80.85 |
LR | 82.37 | 83.59 | 87.89 | 85.72 | 82.37 | 83.59 | 81.99 |
DT | 77.81 | 76.90 | 82.67 | 79.94 | 76.90 | 79.94 | 77.81 |
RF | 80.99 | 81.98 | 84.27 | 83.37 | 81.72 | 83.91 | 78.98 |
NB | 54.71 | 54.71 | 54.71 | 54.71 | 54.71 | 54.71 | 54.71 |
Base Model | SARS-CoV-2 CT Scan Database | ISIC-Archive Dataset | ||||||
---|---|---|---|---|---|---|---|---|
ResNet101 | ResNet101 | |||||||
Accuracy | Precision | Recall | F1-Score | Accuracy | Precision | Recall | F1-Score | |
SVM | 97.24 | 97.78 | 96.89 | 97.55 | 86.79 | 87.45 | 86.99 | 86.23 |
LR | 96.09 | 96.67 | 95.84 | 96.27 | 87.89 | 88.56 | 87.29 | 87.92 |
DT | 95.17 | 96.45 | 95.23 | 95.05 | 82.67 | 82.88 | 82.68 | 82.79 |
RF | 96.78 | 97.01 | 96.55 | 96.55 | 84.27 | 84.98 | 84.43 | 84.99 |
NB | 93.33 | 93.99 | 92.56 | 93.23 | 54.71 | 53.21 | 54.27 | 54.23 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Qayyum, H.; Rizvi, S.T.H.; Naeem, M.; Khalid, U.b.; Abbas, M.; Coronato, A. Enhancing Diagnostic Accuracy for Skin Cancer and COVID-19 Detection: A Comparative Study Using a Stacked Ensemble Method. Technologies 2024, 12, 142. https://doi.org/10.3390/technologies12090142
Qayyum H, Rizvi STH, Naeem M, Khalid Ub, Abbas M, Coronato A. Enhancing Diagnostic Accuracy for Skin Cancer and COVID-19 Detection: A Comparative Study Using a Stacked Ensemble Method. Technologies. 2024; 12(9):142. https://doi.org/10.3390/technologies12090142
Chicago/Turabian StyleQayyum, Hafza, Syed Tahir Hussain Rizvi, Muddasar Naeem, Umamah bint Khalid, Musarat Abbas, and Antonio Coronato. 2024. "Enhancing Diagnostic Accuracy for Skin Cancer and COVID-19 Detection: A Comparative Study Using a Stacked Ensemble Method" Technologies 12, no. 9: 142. https://doi.org/10.3390/technologies12090142