Contemporary Study on Deep Neural Networks to Diagnose COVID-19 Using Digital Posteroanterior X-ray Images

Abstract

COVID-19 is a transferable disease inherited from the SARS-CoV-2 virus. A total of 594 million people have been infected, and 6.4 million human beings have died due to COVID-19. The fastest way to diagnose the disease is by radiography. Deep learning has been the most popular technique for image classification during the last decade. This paper aims to examine the contributions of machine learning for the detection of COVID-19 using Deep Learning and explores the overall application of convolutional neural networks of some famous state-of-the-art deep learning pre-trained models. In this research, our objective is to explore the various image classification strategies for CXIs and the application of deep learning models for optimization and feature selection. The study presented in this article shows that the accuracy of deep learning models when detecting COVID-19 on the basis of chest X-ray images ranges from 93 percent to above 99 percent.

1. Introduction

COVID-19 was originally discovered in Wuhan City, China in December 2019. It rapidly spread out of Wuhan to other countries [1]. At that time, treatment and vaccines for COVID-19 were not available. This pandemic situation was faced all over the world. Early diagnoses for initial control were essential in order to better understand COVID-19, because no particular treatment was available at that time.

In this survey, we mainly focus on the detection of COVID-19 using CXIs from Deep Learning (DL) models used in the state-of-the-art studies mentioned in

Table 1. Name of the model and internal architecture details, along with classification strategy.

Ref.	Model Used	Preprocessing	Feature Extraction	Feature Extraction Model	Classification Using Fully Connected Layer	Classification Using Segmentation	Classification Using Support Vector Machine
[2]	MH-COVIDNet and ResNet	Yes	Yes	AlexNet, VGG19, GoogleNet, and ResNet.	No	No	Yes
[3]	MobileNetV2, SqueezeNet	Yes	Yes	MobileNetV2, SqueezeNet	No	No	Yes
[4]	OptiDCNN	Yes	Yes	OptiDCNN	Yes	No	No
[5]	ULNet	No	Yes	ULNet	Yes	No	No
[6]	LSTM and GANs	Yes	Yes	Inception V4	Yes	No	No
[7]	COVIDX-Net	Yes	Yes	DCNN	Yes	No	No
[8]	ConvNet	Yes	Yes	ConvNet	Yes	No	No
[9]	CapsNet	No	Yes	CapsNet	Yes	No	No
[10]	Transfer learning with CNNs	No	Yes	MobileNet v2, Inception, VGG19, Inception ResNet v2	Yes	No	No
[11]	DarkCovidNet	No	Yes	DarkCovidNet	Yes	No	No
[12]	VGG-16	No	Yes	VGG-16	Yes	No	No
[13]	VGG-16	Yes	Yes	VGG-16	Yes	No	No

. These models have been discussed in detail from an architectural perspective, and in terms of their dataset limitations and their future directions. However, to the best of our knowledge, these are the most important issues regarding the classification task when using DL models. The main contributions of this paper are as follows.

• The architecture used in the state-of-the-art studies presented in Table 1 are discussed, describing their internal structure and how classification is achieved using the DL models; for instance, [2,3] do not use classical approaches for the classification task, whereas the rest of the studies use classical approaches from feature extraction for the classification task.
• As is widely known, COVID-19 is a relatively new disease, and datasets are still very limited; most studies use the same datasets. Details about the datasets used by the authors in Section 2.2.
• Results are discussed on the basis of the datasets used by the authors and the methodology used by state-of-the-art studies.

  Table 2. Number of images used by the authors in their studies and their accuracy.

  Ref. #	COVID-19	Healthy	Other Diseases	Total	Accuracy
  [2]	364	364	364	1092	0.99
  [3]	295	68	98	461	0.99
  [4]	420	2000	N/A	2420	0.99
  [5]	219	1314	1345	2878	0.99
  [6]	371	2923	2778	6072	0.99
  [7]	25	25	N/A	50	0.90
  [8]	225	1583	4292	6100	0.98
  [9]	-	-	94,323	-	0.957
  [10]	224	504	700	1428	0.93
  [11]	250	1000	500	1750	0.92
  [12]	250	3520	2753	6523	0.97
  [13]	445	2880	5179	8504	0.984

  shows the number of images used by the authors in their studies and their corresponding accuracies, while Section 3 presents a comparison of the results obtained in recent studies in terms of percentage.
• The discussions presented in the studies are compiled. Section 4 represents the advantages and the future work reported by the authors discussed in this survey.

During the past several years of COVID-19, it has changed into multiple variants, and these variants are different from one another, with their infecting power, symptoms, and ratio of deaths in the current environment being employed differently. All variants are recognized by the World Health Organization (WHO). For this purpose, deep learning on COVID-19 can help experts to diagnose the disease. When studies for the diagnosis of COVID-19 using deep learning are initiated, images created on the basis of computed tomography (CT) and chest X-ray images (CXIs) are used. Conversely, as a COVID-19 diagnostic procedure, real-time RT–PCR is drastically faster, because the entire technique can be carried out inside a closed tube.

The aim of this survey paper includes research on deep learning techniques. CXIs or CT, as routine tools for pneumonia diagnosis, are easy to perform, and offer rapid diagnosis [1]. Imaging techniques like CXIs and CT used to find the characteristics and signs presented in studies on COVID-19 [2]. Additionally, current studies recommend the use of CXIs for the initial screening of COVID-19. Therefore, screening by means of CXIs can be used as an alternative to the PCR method.

Artificial Intelligence (AI) has produced very promising results in this area with the use of imaging-based data. Figure 1 shows several aspects of AI and ML that are successfully employed in the health industry. A study by Apostolopoulos and Mpesiana [10] demonstrated the detection of COVID-19 from digital CXIs, and this was the first study performed using CXIs, which constitute a transfer learning approach for the detection of disease. In this study, several pre-trained models of deep learning were used, i.e., MobileNet V2, VGG19, Inception, and Inception ResNet V2 and Xception, which are the most frequently used.

In this research survey, we attempt to solve the problem of data shortage in COVID-19 by discussing the datasets used in the detection of COVID-19 from CXIs. Additionally, in this study, state-of-the-art architectures are presented in detail, along with their success rate, limitations, and future research directions.

This paper is structured as follows,

• A literature review is provided, discussing the contributions reported in the literature with respect to the detection of COVID-19 using digital CXIs.
  • The datasets used by different authors for the CXIs are also discussed.
  • Proposed and state-of-the-art pre-trained architecture are also discussed.
  • Details of the training, testing, validations, and numbers of images used by different authors in their studies are described. Table 1 gives a summary of the state-of-the-art architectures used by various authors with promisingly high success rates in the classification task.
• The results of different research studies are discussed, and different state-of-the-art architectures are compared. Table 2 presents a summary of the results.
• Conclusion. The future work discussed and recommended by the authors of the short-listed articles is presented.

2. Related Work in State-of-the-Art Studies for the Detection of COVID-19 from CXIs

In this section, a survey of the DL models used in recent years proposed in some of the state-of-the-art studies reported in Table 1 is compiled. In this survey, we review studies dedicated to the use of CXIs in the diagnosis of the COVID-19 virus. This overview is summarized on the basis of the architectural flow of classification criteria. However, the datasets and the related image preprocessing techniques adopted by the authors are also discussed in Section 2.2 in order to achieve the best possible results. It is still challenging to collect enough high-quality COVID-19 CXIs for the models to be able to perform classification related to neural networks for training and testing purposes so far. We will concisely describe each of the CNN architectures in the articles described in Table 1, and highlight their different properties with respect to datasets and given an architectural view of classification and feature extraction strategies used by the authors. As indicated in Table 1, [2,3] use a different scheme for classification tasks, while the remainder use classical approaches.

2.1. Overview

In this review paper, we examined state-of-the-art studies investigating the use of ML/DL models for training and testing with respect to chest X-ray images (CXIs) with the aim of identifying COVID-19. Several studies have implemented transfer learning using publicly available CNN architectures with publicly available datasets. These architectures, along with their parameters and hyperparameter settings, are publicly available. However, some studies [4,5,7,8,9,10,11] went beyond the use of off-the-shelf tools and implemented novel architectures. In the following subsections, we will provide a general overview of the main approaches presented in Table 1, and the datasets (illustrated in Figure 2) that were used in the research works reviewed in this survey will be discussed.

Studies in COVID-19 diagnostics that employ digital CXIs have multiple and binary classifications. Different studies use different data conditions, like raw data or preprocessed image data, while some studies employ feature extraction or image segmentation techniques for the classification of diseases. Among the various different studies, the most ideal approach is the use of a convolutional neural network (CNN) for the early diagnosis of COVID-19 on the basis of CXIs. Researchers have the use of many Machine Learning (ML) models for the detection of COVID-19 on the basis of CXI datasets. Additionally, computer vision techniques can also be used to detect COVID-19. There is no need for manual feature extraction from the CXIs, DL is an effective Artificial Intelligence (AI) research field that can be used to anticipate the expected results on the basis of CXI data. Some common research areas include skin cancer recognition, brain disease diagnosis, tuberculosis detection, and the detection of breast cancer and lung cancer from CXIs, as well as the segmentation of images containing fungus areas. Below are some recent research studies regarding the detection of COVID-19 from CXIs with the approach used by the authors given in Table 1.

The architecture proposed in [4] finds the best combination of hyperparameters, and automatically designs classifiers for COVID-19 detection from CXIs, an architecture that uses a deep convolutional neural network (DCNN) model with minimum convolutional layers. This approach uses an iterative process, and ends when no further enhancement is recorded in term of results (accuracy). The evolution process starts with the optimization of a simple CNN, which is the starting point. After the evolution process, no more than two additional convolutional layers will be added, and the subsequent structure of the CNN will be the result of a further optimization phase. With the application of CXIs, the model becomes progressively deeper, and every phase maximizes the performance of the system. Wu et al. [5] proposed using a CNN and UL-Net together to detect COVID-19 by using CXIs in the proposed structure in a procedure that was developed with the addition of a new down sampling side and skipping the connections between the fully connected layers based on a U-net, because the structure of the system was like UL, which is why it is known as UL Net. Sheykhivand et al. [6] used CXIs to detach two to four classes based on seven dissimilar and purposeful situations with respect to healthy, viral, bacterial, and COVID-19 classes. In the architecture proposed by the authors, LSTM and transfer learning are used with Generative Adversarial Networks (GANs). Murat Canayaz [2] examined the diagnosis of COVID-19 using CXIs. This research produced a contrast-adjusted dataset possessing three classes, i.e., healthy images, COVID-19, and pneumonia. The image contrast algorithm developed by Ying et al. [14] provided contrast enhancement for the production of a new dataset from existing datasets. Hemdan et al. [7] studied COVIDX-Net, a proposed model containing about seven different Methods of Deep Convolutional Neural Networks (DCNN) models, for example, the latest version of the Visual Geometry Group Network (VGG19) and and additional type of Google MobileNet. Every DCNN model is capable of being used to study the normalized intensities of X-ray images in order to categorize a patient’s status as being either a negative or positive COVID-19 case. Togaçar et al. [3] used the DCNN model to detect COVID-19. The researchers used datasets containing three classification classes, i.e., healthy CXIs, pneumonia, and COVID-19. In the first step, preprocessing was applied to the datasets using the Fuzzy Color method, and images were systematized with the original images and were stacked. Feature extraction was performed with the help of SqueezeNet and MobileNetV2 via Social Mimic Optimization (SMO). The last step used a Support Vector Machine (SVM) as a classifier. In [8], Boran Sekeroglu and Ilker Ozsahin proposed an architecture for the detection of the COVID-19 virus from digital CXIs using a CNN-based architecture using state-of-the-art pre-trained networks for transfer learning. Using real-time diagnostics, the proposed architecture for COVID-19 diagnosis was able to save time for doctors as well as patients. Boran Sekeroglu and Ilker Ozsahin [8] was able to COVID-19 using X-ray images using a Capsule Network, referred to as COVID-CAPS, and the proposed architecture had a number of convolutional layers, i.e., four, and a few capsule layers, i.e., three, with a smaller number of trainable parameters. Apostolopoulos et al. [10] used a transfer learning technique on CNN to detect COVID-19 from CXIs. Futuristic CNN architectures proposed in recent years for medical image classification were used to estimate performance. Ozturk et al. [11] used CXIs with a proposed model for the detection of COVID-19. The proposed model possessed an end-to-end framework and returned the diagnosis results without extracting features. Multi-class and binary-class classification was achieved using the DarkNet model. In this study, a model was used as a classifier for YOLO, a real-time object detection system. Experimental testing performed after 5 to 13 days was found to be able to identify positive symptoms in recovered patients [15]. Brunese et al. [12] proposed a model for the differentiation of COVID-19, pneumonia, and healthy CXIs. The proposed strategy consisted of three stages, where the first detected the existence of pneumonia in the image, the second distinguished between pneumonia and COVID-19, and the third targeted the areas in the CXIs indicating COVID-19. Heidari et al. [13] used computer-aided design (CAD) for the detection of COVID-19 from CXIs. In the proposed approach, two preprocessing steps were used before performing the classification task. The first step removed the common diaphragm regions and bilateral low-pass filter and histogram equalization algorithm in order to process the original image. In the second step, a pseudo color image was formed from the raw image with the filtered image. After preprocessing, the image was fed into the transfer learning-based CNN architecture to perform the classification task using three classes, i.e., pneumonia, healthy CXIs, and COVID-19.

2.2. Datasets Used by Authors in State-of-the-Art Studies

Automatic detection of disease directly depends on the datasets used to train the model of the system; different research studies have considered the use of CXIs due to their lower sensitivity. Current research shows that it is possible to detect early symptoms using CXIs [16]. Patients’ X-ray reports indicate abnormality at the initial time of admission, which is 69%; however, symptoms shown after admission in the hospital reach a value of 80%. There is not a massive body of data currently available for study that is related to COVID-19, because COVID-19 is a new disease. For example, in one study mentioned below [7], the authors performed experiments on the datasets presented in [17,18], containing only 50 images. Figure 2 illustrates the contributions of the datasets used by authors in the state-of-the-art studies listed in Table 1.

Khishe et al. [4] used J.P. Cohen’s COVID-19 dataset [17]. The dataset contains training and test samples, numbering 2084 and 3100, respectively, obtained from CXIs and CT images. The original paper, which was released on the 3 May 2020, contained 250 CXIs of COVID-19 patients, among which 203 out of 250 images showed the posterior–anterior (PA) results of patients in a dataset that is being continuously upgraded. Some selected images are desirable for the detection of COVID-19, as per the information given in [1] by certified radiologists. Furthermore, professional radiologists have estimated their knowledge of the collected datasets, considering 184 out of the 203 images to show clear symptoms of COVID-19. To perform the experiment on the dataset, only 100 images were nominated, and the remaining 84 images formed the training set.

Figure 2. Details of authors [2,3,4,5,6,7,8,9,10,11,12,13] of datasets [15,17,18,19,20,21,22,23,24,25,26,27] used in their studies.

Figure 2. Details of authors [2,3,4,5,6,7,8,9,10,11,12,13] of datasets [15,17,18,19,20,21,22,23,24,25,26,27] used in their studies.

Wu et al. [5] used a dataset of CXIs, i.e., the Radiography Dataset of Kaggle [23]. This dataset includes 2905 CXIs, which are divided into three classes, containing 1341 healthy CXIs, 1345 of pneumonia, and 219 CXIs of COVID-19. This image repository also includes three different datasets: CXIs of COVID-19 are collected from the Italian Society of Medical and Interventional Radiology COVID-19 Dataset (SIRM) and from the Novel COVID-19 2019 dataset, which was established by Dr. Cohen at GitHub [17]. The dataset differs from those used in many articles published recently. The pneumonia and healthy CXIs were collected from Kaggle’s Chest X-ray pneumonia dataset [20]. Sheykhivand et al. [6], in their experiment, used six different trustworthy and reliable datasets that were based on CXIs. Recently, these databases frequently been used in studies focusing on the automatic detection of COVID-19 symptoms [18,19,20,21,24,28]. These datasets include posterior–anterior CXIs of patients with pneumonia. These CXIs include four different classes, namely, viral pneumonia, bacterial pneumonia, healthy, and COVID-19. Murat Canayaz [2] used datasets that had been discovered by Joseph Paul Cohen [17] and Kaggle [23]. There were 145 images of COVID-19 in the first dataset, and in the second dataset, there were 219 images of COVID-19; these datasets are easily and openly available. By merging these datasets, a dataset containing 364 images was obtained for COVID-19. A dataset including pneumonia and normal CXIs, which was prepared in [29], was used in this experiment. Hemdan et al. [7] implemented the proposed model on publicly available images on the basis of a dataset provided by Dr. Joseph Cohen [17] and Dr. Adrian Rosebrock [28], which was used in this study for the classification of COVID-19 results as being either negative or positive. There were 50 CXIs in this dataset, which were categorized into two classes, i.e., half of them were positive images, and the remaining half were negative images for COVID-19 in this dataset. Togaçar et al. [3] performed an experiment using three classes of classification, i.e., pneumonia CXIs, normal CXIs, and COVID-19, on the basis of datasets that were easily and publicly available. In their research, the authors used a combination of two datasets that were publicly and easily accessible, and which contained images of COVID-19. The first COVID-19 dataset was found on the GitHub website, and was shared by the researcher Dr. Joseph Paul Cohen [17]. These images were subsequently examined by experts and made accessible on the public website GitHub. Images of MERS, SARS, COVID-19, and other diseases can be found in Dr. Joseph Paul Cohen’s dataset. For this experiment, a dataset with 76 images highlighting COVID-19 was chosen. The CXIs were developed by a team of researchers from Qatar University, medical practitioners from Bangladesh, and partners from Pakistan and Malaysia, and were included in the second COVID-19 dataset. The current COVID-19 dataset comprises 219 CXIs, and is the second dataset to have been made available on Kaggle [8,19]. The 225 CXIs of COVID-19 used by Boran Sekeroglu and Ilker Ozsahin [8] were discovered in [17], and have been made freely available on GitHub [24]. The age of the COVID-19 group was noted as being 58.8 ± 14.9 years, and the dataset consisted of 131 male patients and 64 female patients. Please note that information was missing for some patients; this is because the datasets used did not always contain complete data, it was the first publicly available dataset containing images of CXIs of COVID-19 collection, and it was developed in a very short period of time. This dataset contains four types of labels, i.e., healthy, bacterial, non-COVID viral, and COVID-19. The study aimed to determine the results for COVID-19: either positive or negative. However, the negative class was organized into three labels—healthy, bacterial, and non-COVID viral. Afshar et al. [9] generated a dataset from a publicly available dataset of CXIs [17,24] and performed experiments. Apostolopoulos et al. [10] used a collection of CXIs provided by Dr. Joseph Cohen [17]. The work in [29] employed bacterial-pneumonia CXIs, and a dataset made available on the Internet [21] was examined by the Italian Society of Medical and Interventional Radiology (SIRM), Radiopaedia, and the Radiological Society of North America (RSNA). Ozturk et al. [11] used two CXI datasets for the identification of COVID-19. The dataset contained CXIs which were developed by Dr. Cohen [17] using images from different open contact sources. This dataset is continually being updated with images that have been shared by researchers from around the world. Furthermore, a database of chest X-rays was provided by the researchers in [30], and was used to detect normal and pneumonia images. Using the suggested strategy, Brunese et al. [12] analyzed several datasets belonging to various universities. Furthermore, researchers have used three different CXI datasets: the first was the collection of “Images with infection of COVID-19” [17], and the second dataset [24] was used by authors of the paper “Automated Detection of COVID- 19 Cases Using Deep Neural Networks with X-ray Images” [24], while the third dataset included the collections from the “National Institutes of Health Chest X-ray” [30]. In the end, by merging all of the datasets, it was possible to obtain results of chest X-rays that were linked to several classes. Heidari et al. [13] developed and collected a dataset of CXIs that was easily available in public medical repositories [17,23,29,31]. These data image repositories were initially produced and studied at the Georgetown University Center for Security and Emerging Technology, the National Library of Medicine, the National Institutes of Health, and Microsoft Research, by the Allen Institute of AI and Chan Zuckerberg, coordinated by the White House Office of Science and Technology Policy. Specifically, the dataset used in this study contained 8474 2D images of CXIs in the posterior–anterior chest view. Among them, 415 images showed confirmed COVID-19 disease, 2880 images showed healthy (non-pneumonia) patients, and 5179 images belonged to other community-acquired non-COVID-19-infected pneumonia cases. Table 2 presents a detailed discussion of the above literature review regarding the datasets used by the authors and the classes used for classification, training, testing, and validation. Figure 3, below, illustrates the smaller amount of data used by authors in more recent studies. However, the dataset of Dr. Joseph [17] has been used by almost all researchers for the classification task. Furthermore, the dataset of Dr. Joseph [17] is very famous, and is used by many researchers in current research, as well.

3. Comparison of Results

In this section, the results presented for some state-of-the-art models are discussed in terms of their performance, and some of those models, selected by the authors, are presented in Table 2 with respect to the detection of COVID-19 using CXIs in binary and multi-class classification tasks. In the table below, we compare the results of the state-of-the-art models developed by the mentioned authors. Table 2 provides details of the number of CXIs with COVID-19 infection, healthy, and with other diseases (e.g., viral/bacterial pneumonia).

Every model was evaluated using different standards for classification tasks: sensitivity, i.e., true positive rate, area under the ROC Curve, specificity, and accuracy. Sensitivity or specificity alone are not a benchmark for performance evaluation; however, they can be used specifically in the case of imbalanced data. In this regard, the performance of the model presented in Figure 4 was assessed using each of the aforementioned criteria. Higher ROC AUC scores were more accurate for differentiating between COVID-19 and without COVID-19 in medical applications [8]. Equations (1)–(4) were used to calculate specificity, accuracy, sensitivity, and precision, respectively.

$$\mathrm{Specificity}=\frac{\mathrm{True}\mathrm{Negatives}}{\mathrm{True}\mathrm{Negatives}+\mathrm{False}\mathrm{Positives}}$$

(1)

$$\mathrm{Accuracy}=\frac{\mathrm{True}\mathrm{Negatives}+\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Positives}+\mathrm{True}\mathrm{Negatives}+\mathrm{False}\mathrm{Negatives}}$$

(2)

$$\mathrm{Sensitivity}=\frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Negatives}}$$

(3)

$$\mathrm{Precesion}=\frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Positives}}$$

(4)

Khishe et al. [4], after repeating the training ten times, achieved an accuracy on the validation dataset between 89.22% and 93.98%. Using weight validation accuracy, the ten trained OptiDCNNs were formed into an ensemble on the basis of weight average. DCNN had an accuracy of between 78% and 90%; however, OptiDCNN achieved a validation accuracy of 91.09% and the ensemble achieved an accuracy of 88.3% on the validation dataset. Wu et al. [5] computed the precision, accuracy, recall, and F1-score of each classification class, i.e., COVID-19 and healthy, using K-fold validation; the author and his team used 1 to 5 folds in this study, as shown in

Table 3. Experimental results of the ULNet proposed in [5] in terms of percentage.

Fold	Class	Recall	Precision	F1-Score	Accuracy
1	Infected with COVID-19	97.67	100	98.82	99.68
	Healthy	100	99.63	99.81
2	Infected with COVID-19	100	91.49	95.56	98.72
	Healthy	98.51	100	99.25
3	Infected with COVID-19	100	91.49	95.56	98.75
	Healthy	98.51	100	99.25
4	Infected with COVID-19	100	100	100	100
	Healthy	100	100	100
5	Infected with COVID-19	100	100	100	100
	Healthy	100	100	100

, below.

The authors applied their proposed ULNet on different datasets, i.e., Kaggle’s and the QaTa COVID-19 CXI datasets.

Table 4. Performance of the proposed ULNet on Kaggle’s dataset in terms of percentage.

Dataset	Folds	Recall	Performance	F1-Score	Accuracy
Kaggle’s	1	98.84	99.82	99.32	99.68
	2	99.26	95.75	97.41	98.72
	3	99.26	95.75	97.41	98.72
	4	100	100	100	100
	5	100	100	100	100
	Average	99.47	98.26	98.83	99.42
QaTa COVID-19	1	98.50	98.51	98.50	98.50
	2	99.50	99.51	99.50	99.50
	3	99.50	99.51	99.50	99.50
	4	99.00	99.02	99.00	99.00
	5	100	100	100	100
	Average	99.30	99.31	99.30	99.30

, below, shows the performance of both datasets used by the author.

Sheykhivand et al. [6] used seven different scenarios for their proposed DNN model for the automatic classification of pneumonia on the basis of digital CXIs. All seven scenarios achieved accuracy greater than 90%, except one of them. The proposed model for binary classification classes, i.e., Healthy and COVID-19, achieved accuracy greater than 99%. A scenario including Healthy and COVID-19 had 564 CXIs of COVID-19 in the test set that were properly identified by the model; however, two samples were misidentified. Murat Canayaz [2] used two datasets, i.e., original and enhanced datasets. The table below shows the performance of different state-of-the-art pre-trained models using SVM for the classification of three classes, namely, pneumonia, healthy, and infected with COVID-19. The proposed network produced stable results for both datasets. Training and testing used 70% and 30% of the data, respectively, in every approach. K-fold cross-validation with a value of 5 for k was used to test the consistency of the experiment. In this study, with the original dataset, accuracies of 97.55%, 98.16%, 95.10% and 95.71% were achieved using AlexNet, VGG-19, GoogleNet, and ResNet, respectively. After the original dataset, the enhancement dataset was used for classification with SVM, achieving accuracies of 97.55%, 98.47%, 96.94%, and 96.94% using AlexNet, VGG-19, GoogleNet, and ResNet, respectively. GoogleNet and ResNet achieved the best overall accuracies, at 95.6% and 96.61%, respectively. Each algorithm was evaluated individually, because this shows the properties obtained from the algorithms with respect to the success of the classification.

Table 5. Results of the models used in the enhanced dataset in terms of percentage.

Model	Overall Accuracy (%)
	Cross-Validation	BPSO Method	BPSO Method on Combined Features
			30% Test Data	k-Fold
AlexNet	97.98	99.08	99.08	99.08
VGG-19	98.71	99.38
GoogleNet	95.6	95.71	97.85	97.06
ResNet	96.61	96.94

shows the details of the results and tuning.

The BPSO method uses a total of 499, 488, 488, and 477 features for AlexNet, VGG-19, GoogleNet, and ResNet, respectively. The BPSO method on combined features using k-fold (k = 5) and a total of 987, 965 features for AlexNet and VGG-19, GoogleNet and ResNet, respectively. Hemdan et al. [7] used 80% and 20% of the data for model training and testing purposes, respectively, for the proposed COVIDX-Net. F1-scores of 0.89 and 0.91 were successfully recorded for healthy and COVID-19, respectively, using VGG-19 and DenseNet; however, the worst classification performance of 0.67 for healthy and 0.00 for COVID-19 was recorded using the Inception v3 model. Togaçar et al. [3] used 70% and 30% of the datasets for training and testing purposes, respectively. The experiment consisted of three steps. The k-fold cross-validation method was used. SqueezeNet and MobileNetV2 models were used with SVM as the classifier in the last layer. On the basis of the results of the classification, it was stated that an overall accuracy of 99.27% was achieved.

Table 6. Results of the confusion matrix obtained by the models used in [3].

Model	Overall Acc. (%)
	Original Dataset	Structured Dataset	Stacked Dataset
SqueezeNet	84.56	95.58	97.06
MobileNetV2	96.32	97.05	97.06
Results of the stacked dataset with the 5-fold cross-validation method.
SqueezeNet	-	-	95.85
MobileNetV2	-	-	96.28

, below, shows the metric values of the models used in [3].

A dataset with 1357 features was obtained, and 30% of the feature set was used as test data. A classification accuracy of 99.27% was achieved by combining the efficient features. The Social Mimic Optimization (SMO) algorithm was able to achieve success in classification by combining the selected features. Boran Sekeroglu and Ilker Ozsahin [8] performed 38 experiments for the ConvNet using three groups. These groups were used in combination with COVID-19, with the first group being Healthy, the second group being Pneumonia, and the last group being Healthy and Pneumonia. In each group, the number of COVID-19 images was 225. Validation was performed using K-Fold cross-validation. For the first group, 1583 Healthy patients were trained, and a maximum mean accuracy of 98.34% was obtained; after applying Average Pixel Per Node (APPN), a maximum mean accuracy of 98.23% was achieved. However, original images with different dimensions were used, resulting in a mean accuracy of 99.11%. In the second group, 4292 Pneumonia patients were trained, and a maximum means accuracy of 99.09% was achieved; after applying APPN, a maximum mean accuracy of 98.93% was achieved. However, original images with different dimensions were used, resulting in a mean accuracy of 99.44% being achieved. In the last group, 4292 Pneumonia and 1583 Healthy patients were trained, and a maximum mean accuracy of 95.99% was obtained when using DenseNet121. Afshar et al. [9] stated that the proposed COVID-CAPS was able to achieve an accuracy of 98.3%. The author and his team used a capsule network-based architecture on CXIs to diagnose COVID-19 on the aforementioned dataset without the use of any preprocessing techniques, as described in detail in the previous section. Apostolopoulos et al. [10] used the transfer learning approach on CNN with a success rate of 98.75% without the application of preprocessing techniques on the CXI data. A total of 1427 CXIs were used to classify viral pneumonia, bacterial pneumonia, and COVID-19. The accuracy of transfer learning models, i.e., Xception, MobileNet v2, ResNet v2, VGG, and Inception, however, was recorded at 98.75% for the Inception model in two-class classification, while an accuracy of 93.48% was recorded for three-class classification tasks. Ozturk et al. [11] used the DarkNet model, and an accuracy of 98.08% was recorded when using 17 convolutional layers. However, the performance was evaluated using a 5-fold cross-validation method for triple and binary classification tasks. Brunese et al. [12] achieved an average accuracy of 97%. Heidari et al. [13] used CAD schemes with CNN, and recorded an overall accuracy of 94.5%; however, in two-class classification tasks, an accuracy of 98% was achieved. Meanwhile, without the use of preprocessing, the proposed system with CAD yielded an accuracy of 88%, which is very low compared to the results presented in other state-of-the-art studies, as shown in Figure 4.

4. Discussion

A review of the applications of artificial intelligence ML-based architectures/mechanisms for the detection of COVID-19 on the basis of CXIs is presented in this research. AI plays a vital role in identifying COVID-19 by applying image processing techniques to CXIs.

Table 7. Advantages and future work reported by the authors.

Ref.	Pros	Cons
[2]	Preprocessing before classification is used which reduces computation time. The MH-CovidNet approach used image enhancement in diagnosing COVID-19 using CXIs.	Not all deep learning models can use the suggested method to their advantage. New deep learning models that will guarantee success must be researched.
[3]	The proposed approach is integrated with portable smart devices. MobileNetV2 and SqeezeNet were used in the model. Minimum parameters used as compared with other deep learning models. SMO algorithm also saves time with CNN during the process.	The limited dataset used. A problem with the suggested method of handling low-resolution images. The resolution dimensions of the original photos and the structured images must match when using the stacking technique.
[4]	The benchmarked dataset was used. Results evaluated by comparative studies. The result shows designed model presents competitive results as compared to benchmarked models. The concept of CAM was applied.	Future study should explore how the existing network works. Further improvement on the proposed model regarding different optimization approaches and validation methods.
[5]	The external dataset was used and provided 99.60% accuracy. The proposed model was 99.30% accurate on the benchmarked dataset.	The proposed model will be validated using more X-ray images from other hospitals. The model will use to identify COVID-19 against images of CT. Salience maps also add to improve the accuracy of the model which is also important for clinical practice.
[6]	The proposed model is efficient, quick, and dependable, with promising outcomes.	Dataset is limited. The model is based on the end-to-end learning principle. Not include any handcrafted features.
[7]	The proposed model demonstrated that the VGG19 and DenseNet201 models have the highest deep learning classifier performance scores.	Not mentioned.
[8]	The ability of classification strongly depends on the dataset characteristics. Convergence is achieved well when using ConvNets for fewer classification classes and fewer pictures.	The experiment’s restricted dataset limits the cohorts that can be chosen and could lead to biased results. A large dataset is needed.
[9]	A low number of trainable parameters.	Pre-training was able to further improve the accuracy, specificity, and AUC. Large dataset required.
[10]	Rapid, inexpensive, and automatic diagnosis of the COVID-19 disease. Extracted characteristics carried out by CNNs result in trustworthy biomarkers that aid in COVID-19 identification.	A large dataset is required for more detailed analysis. Future research would concentrate on identifying patients with moderate symptoms as opposed to pneumonia.
[11]	The model has the ability to diagnose the disease of COVID-19 in seconds. CXIs were used for the diagnosis by the proposed model.	More datasets are required. The model should be placed in the cloud to rapidly diagnose patients and aid in their recovery.
[12]	The proposed model aimed to reduce the time (2.5 s).	Large dataset required. Using formal verification techniques can help you get more accurate results.
[13]	The model includes considerable parameters that need to be trained in, a large and diverse image dataset. Dataset is unbalanced in 3 classes. Class weight technique during the training process was applied. Using a histogram zequalization approach to create a contrast-normalized image and a bilateral low-pass filter to zminimize the noise in the image. Image pre-processing applied.	Publicly available dataset of 8474 cases, including 415 COVID-19 cases. A large image dataset has to be used to test and validate the CAD method. The study solely examines and evaluates two image preprocessing techniques. New image processing and segmentation algorithms must be created.

, below, shows the advantages reported by and future work suggested in recent studies. Still, there is a shortage of datasets for the detection of COVID-19 and the training of models. Appropriate feature selection plays a vital role in the detection of COVID-19 using CXIs. Table 3 presents the best results obtained in recent studies.

5. Conclusions

This article reviewed a deep transfer learning model and a deep learning classic model, and independently developed architectures described by various authors were covered in detail with respect to the automatic detection of COVID-19 on the basis of CXIs from among three other classes (bacterial pneumonia, viral pneumonia, and normal CXIs). While this review study examined studies addressing important research questions, there are still a limited number of datasets available for performing the training/testing tasks. However, some recent studies have used the same datasets to perform these tasks, as it is widely accepted that, as this is a new kind of disease, it will require more time to address issues related to datasets and performance. In order to make it easier for researchers, academics, and practitioners to use and access the current methodologies, applications, and datasets, this research also explored and organized the available literature. The most important AI and ML objectives are diagnosis, classification, and prediction. Moreover, the size of the dataset was not sufficient to be able to perform the experiments carried out by some of the authors mentioned in Section 2.2. In this regard, several data augmentation and preprocessing techniques [2,3,4,6,7,8,13] have been used to enhance the dimensions of the data. Murat Canayaz [2] and Togaçar et al. [3] used SVM as a classifier for the classification task; however, the remainder of the studies discussed in this paper used a fully connected layer for the classification task. The transfer learning model, as well as other models discussed in this review, could help doctors to make accurate decisions regarding the diagnosis of COVID-19, which is an alternate option to the PCR test method. Researchers who plan to develop models or architectures for the detection of COVID-19 on the basis of CXIs can reuse the existing models rather than designing them from the scratch, thus saving time, because they are currently able to achieve accuracies greater than 99%.

6. Future Directions

The COVID-19 epidemic has changed our lives to an exceptional extent. However, the contributions of the research community have been vast across various fronts. In this regard, the direction provided in this paper describes an approach for future researchers for the detection of COVID-19 on the basis of digital CXRs, as well as an in-depth discussion of the datasets used by the authors mentioned in Figure 2, as well as the conclusions drawn on the basis of the datasets used in the various state-of-the-art studies reported here, with the hope that in the future, more real data will become available. However, our survey has some limitations. There is a need for further detailed exploration of the internal architectures of the architectures/models employed in the recent state-of-the-art studies described in Section 2.2. Additionally, it is necessary to keep a continuous eye on new publicly available dataset sources, and, importantly, to collaborate with hospitals and radiology diagnosis departments in order to obtain samples related to COVID-19 CXIs. Moreover, CT images are also efficient for the purposes of feature extraction, and have demonstrated promising outcomes.