Deep Learning Models for COVID-19 Detection

Abstract

Healthcare is one of the crucial aspects of the Internet of things. Connected machine learning-based systems provide faster healthcare services. Doctors and radiologists can also use these systems for collaboration to provide better help to patients. The recently emerged Coronavirus (COVID-19) is known to have strong infectious ability. Reverse transcription-polymerase chain reaction (RT-PCR) is recognised as being one of the primary diagnostic tools. However, RT-PCR tests might not be accurate. In contrast, doctors can employ artificial intelligence techniques on X-ray and CT scans for analysis. Artificial intelligent methods need a large number of images; however, this might not be possible during a pandemic. In this paper, a novel data-efficient deep network is proposed for the identification of COVID-19 on CT images. This method increases the small number of available CT scans by generating synthetic versions of CT scans using the generative adversarial network (GAN). Then, we estimate the parameters of convolutional and fully connected layers of the deep networks using synthetic and augmented data. The method shows that the GAN-based deep learning model provides higher performance than classic deep learning models for COVID-19 detection. The performance evaluation is performed on COVID19-CT and Mosmed datasets. The best performing models are ResNet-18 and MobileNetV2 on COVID19-CT and Mosmed, respectively. The area under curve values of ResNet-18 and MobileNetV2 are $\mathbf{0}.\mathbf{89}\%$ and $\mathbf{0}.\mathbf{84}\%$, respectively.

1. Introduction

A new pneumonia-type Coronavirus (COVID-19) was detected in Wuhan China in 2019, ref. [1]. Previous versions of this virus are known as SARS-CoV-2. This virus has a higher infection ability than other viruses. As a result, many people have been infected and require medical care in hospitals. Recent works can be found in [2,3,4,5,6].

A recent survey [7] explains all the different methods for COVID-19 detection. Transcription-polymerase chain reaction (RT-PCR) tests are being used to detect the virus in the human body. Computed tomography scans (CT scans) and X-ray images are other ways of identifying COVID-19 in the human body. X-ray images show SARS-CoV-2 infection areas on the human lungs. Furthermore, CT scans can be used to visualise the 3D lungs of a person to decide on the level of severity.

Convolutional neural networks (CNNs) are known as powerful models for image modelling. These can be used to diagnose COVID-19 on X-rays and CT scans. The main advantage of using CNN models is that they allow faster detection of the virus from images than doctors and radiologists. Today, CNN models are being used to diagnose viruses on medical images. These models have been utilised for skin lesion detection on digital images, refs. [8,9]. Authors have also used CNN models for retinography diagnosis on images, refs. [10,11,12]. Furthermore, CNN models allow the diagnosis of COVID-19 on X-rays and CT scans. The well-known CNN models are AlexNet [13], GoogleNet [14], VGG [15], MobileNetV2 [16], ResNet [17], and DenseNet [18]. These models are mainly trained using an ImageNet dataset [19] and then fine-tuned on medical images.

CNN models are data-hungry, and they require many images for the training process. However, accessing a large number of CT images might not be possible during a pandemic or may require a long time. Therefore, inadequate data might hamper the usage of an artificial intelligence-based model for COVID-19 detection. On the other hand, data-efficient CNN models are built on small sets of available images and allow fast modelling of the disease to diagnose COVID-19. Therefore, data-efficient models might make a significant contribution to the rapid diagnosis of COVID-19 during a pandemic.

In this paper, the data-efficient GAN based CNN method (Figure 1) allows COVID-19 detection from CT scans. This paper presents enhanced data-efficient convolutional neural networks to detect COVID-19 from CT scans. The proposed approach is based on the generation of synthetic and augmented images and then generates CNN models using these datasets. Synthetic images allow more information to be extracted from the CT scans and then modelled by CNN models. The enhanced model performances and augmented data-based CNN models are compared on two publicly datasets. The results show that the enhanced models outperformed the classic CNN models.

The advantages of the proposed method are as follows:

• The proposed model builds on augmented and synthetic CT images of the chests of COVID-19 patients, whereas the classic models only employ augmented images, refs. [20,21,22,23].
• The proposed CNN learns more possible COVID-19 signs from synthetic CT images then the classic CNN models.

The main novelties of this work are:

• The proposed novel method utilises a GAN model to generate unseen COVID-19 and normal CT images from a small database. This approach allows the CNN model to learn all possible image deformations for better modelling of the CT images. In contrast, classic CNN models build on data augmentation techniques for improved performance. However, image augmentation allows the generation of more COVID-19 and normal images with different views and orientations. This is problematic since the deformation of the lungs on CT images is the same in the generated data.
• A method is proposed for fusing synthetic and augmented CT scans for generating enhanced CNN models for COVID-19 detection.
• Data-efficient enhanced ResNet-18, ResNet-50, VGG, MobileNetV2, AlexNet, and DensNet121 models are proposed for the diagnosis of COVID-19.

The content of this paper is as follows. First, information about the related databases is given. Second, the generation of augmented images is described. Then, the GAN model for synthetic CT scan image creation is explained. The architecture of the GAN model is also explained. Furthermore, the enhanced CNN models based on GAN models are described. Finally, the performance of the proposed methods is evaluated and discussed.

2. Related Work

2.1. Convolutional Neural Networks

He et al. [24] and Hu et al. [25] used CNN models to detect COVID-19 on CT images. He et al. [24] proposed a new transfer learning approach to train a CNN on available CT images. Furthermore, Hu et al. [25] proposed the generation of a CNN model using a small number of CT images.

Mei et al. [26] combined a convolutional neural network and a support vector machine to classify COVID-19 related CT images. The new model architecture was described for more accurate COVID-19 identification on CT images.

Harmon et al. [27] proposed a DensNet-121 network for differentiating COVID-19 from pneumonia virus. The classification accuracy was evaluated using several datasets.

Bhandary et al. [28] replaced the last layer of several CNN model architectures and they introduced a support vector machine. The authors evaluated the performance of this new architecture for COVID-19 diagnosis. The proposed network also detected cancer using CT and X-ray images.

Butt et al. [29] use 3D CT scans for classifying COVID-19, and viral pneumonia. The authors processed each of the CT image patches and then used these patches as inputs to the ResNet-18 model to detect the COVID-19 virus.

2.2. Generative Adversarial Networks

Waheed et al. [1] and Loey et al. [30] utilised a convolutional neural network and generative adversarial network (GAN) for the diagnosis of COVID-19. The authors generated synthetic medical images using a GAN model to create a CNN model.

Generative adversarial networks have been used for medical imaging. The well-known GAN models are vanilla GAN, deep convolutional GAN (DCGAN), pix2pix, and CycleGAN, ref. [31]. The authors of [1,32,33,34] mainly used the vanilla GANs and DPGAN models for generating synthetic images.

3. Method

Figure 1 shows the proposed data-efficient convolutional neural network (CNN) model for COVID-19 detection. The proposed novel method uses the deep convolutional generative adversarial network (DCGAN) and the data augmentation technique to increase the small number of available CT scans. First, the data augmentation technique allows for the creation of many images. Then, these images are used as input to the DCGAN model to produce synthetic images. Then, a CNN model is trained on both synthetically generated and augmented images. In the testing part, the created CNN model provides COVID-19 and non-COVID-19 classification on CT scans.

The proposed data-efficient CNN models are generated and tested on COVID19-CT [24] and MosMed [35] datasets.

Table 1. Total number of images in the datasets.

CT-Scan	Dataset	Train	Test
COVID-19	COVID19-CT	324	40
Normal	COVID19-CT	293	37
COVID-19	Mosmed	168	20
Normal	Mosmed	168	20

describes the number of images and related categories.

3.1. COVID19-CT Database

The authors of [24] prepared a COVID19-CT dataset for research purposes. This dataset is comprised of 349 COVID-19 and 397 normal CT images. Figure 2 also presents samples of COVID-19 and normal images.

3.2. Mosmed Database

The authors of [35] collected 1110 COVID-19 and non-COVID-19 CT scans from hospitals in Moscow, Russia. The authors also grouped COVID-19 related CT scans as a normal, mild, moderate, or severe condition. Sample images are presented in Figure 2.

3.3. Augmented Datasets

The DCGAN model allows the production of synthetic versions of the available images in the dataset. However, GAN models require a number of pictures for accurate modelling of the images in the dataset. Therefore, available images are augmented to increase the quantity. Then the DCGAN uses these enlarged images to produce synthetic images.

Table 2. Total number of images in the datasets.

Data	CT-Scan	Dataset	Train	Test
Aug.	COVID-19	COVID19-CT	1393	40
Aug.	Normal	COVID19-CT	1672	37
GAN	COVID-19	COVID19-CT	500	40
GAN	Normal	COVID19-CT	500	37
Aug+GAN	Normal	COVID19-CT	2172	37
Aug+GAN	COVID-19	COVID19-CT	1893	37
Aug	COVID-19	Mosmed	1087	23
Aug	Normal	Mosmed	1069	23
GAN	COVID-19	Mosmed	128	23
GAN	Normal	Mosmed	128	23
Aug+GAN	Normal	Mosmed	1197	23
Aug+GAN	COVID-19	Mosmed	1218	23

shows the number of augmented (Aug.) CT scans for the COVID19-CT and Mosmed datasets. The datasets are split into training and testing sets. Then, CT scans of the training set are rotated to increase their numbers in both datasets. We applied the image rotation technique to augment the training set.

Table 2 also shows the number of synthetically generated (GAN) CT scans. The datasets included a combination of synthetic and augmented images (Aug+GAN) and the number of combined images is also reported in Table 2.

3.4. Synthetic CT Image Generation

Synthetic CT image generation is based on the DCGAN method [34] in Figure 1 to generate synthetic CT images. The DCGAN method is the improved version of the GAN method [33]. Figure 3 also shows the synthetically generated CT scans of the COVID19-CT dataset. This convolutional neural network comprises a generator and discriminator. The discriminator builds on convolution layers while the generator builds on convolutional transpose layers.

This method employs the generator $G\left(x\right)$ to produce synthetic images and then utilises the discriminator $D\left(z\right)$ to classify a given synthetic image of a real image. Latent vectors of distribution are used as inputs to the generator and then the generator outputs synthetic images. These generated images are used as inputs to discriminate together with real photos. Finally, the discriminator classifies the input image as a real or synthetic image. This process is repeated for many latent vectors, and the optimisation function is minimised. This function is defined by

$$\underset{G}{min}\underset{D}{max}V(D,G)=E_x[logD\left(x\right)+E_z\left[log(1-D\left(G\left(z\right)\right)]\right]$$

(1)

where data are denoted by x, generator distribution and noise variables are denoted by $p_g$ and $p\left(z\right)$ respectively.

The discriminator employs several convolutions, and this method utilises batch normalisation and LeakyReLU operations after each convolution.

Figure 2. Sample images of COVID19-CT dataset. (a) COVID-19, (b) Normal.

Figure 2. Sample images of COVID19-CT dataset. (a) COVID-19, (b) Normal.

Figure 3. Sythetic images of COVID19-CT dataset. (a) COVID-19, (b) Normal.

Figure 3. Sythetic images of COVID19-CT dataset. (a) COVID-19, (b) Normal.

3.5. Model Generations

The AlexNet, VGG, ResNet-18, ResNet-50, MobileNetV2, and DensNet-121 networks were generated using augmented and synthetic images. All RGB images of 224 × 224 × 3 sizes are used as inputs to fine-tuned convolutional neural network models (CNNs) for training. All networks are pretrained on the Imagenet dataset, and then these models are further trained using augmented and synthetic images. Fine-tuning is achieved by freezing all convolutional layers and adapting the last fully connected layer for COVID-19 and non-COVID-19 classification.

3.6. COVID-19 Prediction

Figure 1 presents the data-efficient deep learning method. Each of the CT images is used as input to the deep network and then CT images are classified as COVID-19 or non-COVID-19.

3.7. Components of the Optimisation Function

Optimisation values are as follows. We use 0.9 and 256 for momentum and the batch size respectively. The learning value is 0.0001. We use 50 epochs for optimizing the function.

3.8. Implementation

A desktop computer was used to run the experiments. This computer is equipped with an Intel Corei7-4790 CPU and NVIDIA GeForce GTX-1080Ti graphics card.

3.9. Softwave

We used Pytorch deep learning library for generating and testing the proposed methodology.

4. Performance Evaluation

The performance evaluation of the classic deep learning method and the proposed data-efficient methods were evaluated using the COVID19-CT and Mosmed datasets.

Performance evaluation was performed using the metrics described as follows. The area under the receiver operating characteristic (ROC) curve (AUC), accuracy (ACC), sensitivity (SE), and specificity (SP) performance merits were used to test the accuracy of the methods. Accuracy, sensitivity, and specificity can be described as:

$$\begin{array}{cc}\hfill \mathrm{Accuracy}& =\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill \mathrm{Sensitivity}& =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \mathrm{Specificity}& =\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}}\hfill \end{array}$$

(4)

where true positive, positive, true negative, false positive, and false negative are denoted as TP, TN, FP, and FN, respectively.

Comparison between Classic Deep Learning Method and Proposed Data-Efficient Method

Table 3. Performance comparisons.

Network	Disease	Data	AUC	ACC	SE	SP
Resnet18	COVID19-CT	Aug	0.77	0.75	0.83	0.71
Resnet18	COVID19-CT	Aug+GAN	0.89	0.74	0.88	0.68
Resnet50	COVID19-CT	Aug	0.71	0.77	0.86	0.72
Resnet50	COVID19-CT	Aug+GAN	0.81	0.73	0.95	0.66
Vgg	COVID19-CT	Aug	0.65	0.75	0.86	0.70
Vgg	COVID19-CT	Aug+GAN	0.67	0.76	0.87	0.70
MobileNetV2	COVID19-CT	Aug	0.71	0.73	0.82	0.69
MobileNetV2	COVID19-CT	Aug+GAN	0.77	0.73	0.84	0.68
Densenet121	COVID19-CT	Aug	0.70	0.74	0.87	0.69
Densenet121	COVID19-CT	Aug+GAN	0.77	0.67	0.92	0.61
AlexNet	COVID19-CT	Aug	0.60	0.67	0.72	0.64
AlexNet	COVID19-CT	Aug+GAN	0.80	0.69	0.88	0.64
AlexNet	MosMed	Aug	0.71	0.70	1.00	0.63
AlexNet	MosMed	Aug+GAN	0.73	0.66	0.89	0.60
MobileNetV2	MosMed	Aug	0.77	0.67	0.69	0.65
MobileNetV2	MosMed	Aug+GAN	0.84	0.62	0.65	0.60
Resnet50	MosMed	Aug	0.74	0.69	0.69	0.69
Resnet50	MosMed	Aug+GAN	0.78	0.69	0.69	0.69
Resnet18	MosMed	Aug	0.70	0.67	0.68	0.66
Resnet18	MosMed	Aug+GAN	0.75	0.69	0.69	0.69
Vgg	MosMed	MA	0.63	0.69	0.71	0.68
Vgg	MosMed	Aug+GAN	0.71	0.66	0.67	0.64
Densenet121	MosMed	Aug	0.60	0.65	0.64	0.65
Densenet121	MosMed	Aug+GAN	0.62	0.61	0.63	0.60

reports the performances of the AlexNet, VGG, ResNet-18, ResNet-50, MobileNetV2, and DensNet-121 deep learning models. These models only build on augmented data of the CT scans. These models were evaluated for both the COVID19-CT and Mosmed datasets.

The performances of the proposed data-efficient models are also evaluated in Table 3. Table 3 also reports the performances of the AlexNet, VGG, ResNet-18, ResNet-50, MobileNetV2, and DensNet-121 deep learning models. These models build on both augmented and synthetic data of the CT scans. These models were evaluated for both the COVID19-CT and Mosmed datasets.

Table 3 also reports a comparison between the classic deep learning method and the proposed data-efficient method. First, the comparisons of the models were performed using the COVID19-CT dataset. All data-efficient AlexNet, VGG, ResNet-18, ResNet-50, MobileNetV2, and DensNet-121 models outperformed the classic convolutional models. Furthermore, the ResNet-18 model built on augmented and synthetic data outperformed the ResNet-18 model, which only builds on augmented data. This model provided a $\mathbf{0}.\mathbf{89}\%$ ROC value. This augmented and synthetic data-based model also outperformed all other models.

The comparisons of the models were performed using the Mosmed dataset. All data-efficient AlexNet, VGG, ResNet-18, ResNet-50, MobileNetV2, and DensNet-121 models outperformed the classic convolutional models. Furthermore, the MobileNetV2 model built on augmented and synthetic data outperformed the MobileNetV2 model, which only builds on augmented data. This model provided a $\mathbf{0}.\mathbf{84}\%$ ROC value. This augmented and synthetic data-based model also outperformed all other models.

5. Discussion

The proposed models are also compared with the recent work of ref. [30]. ResNet-18 and ResNet-50 models give $0.88\%$ and $0.95\%$ sensitivity values. However the authors of ref. [30], report $0.85\%$ sensitivity value for ResNet-50 model. The sensitivity value is the measure of the COVID-19 detection. The proposed method shows that the COVID-19 was detected more accurately than other work. Training models on both synthetic and augmented datasets increase the sensitivity values. Table 3 also presents a comparison of the proposed and other works.

All data-efficient models outperformed the classic convolutional neural networks on the COVID19-CT and Mosmed datasets. These models show that building the CNN model on synthetic and augmented images allows better recognition of COVID-19 on CT scans. The reason is that GAN models produce different synthetic CT images related to different COVID-19 deform CT scans. Since synthetic images cover large variations of COVID-19 disease signs on CT scans, the CNN can capture all details of these signals on the images. When synthetically generated images are used in conjunction with augmented data, CNN models perform better than augmented data-based CNN models.

This study also shows that the different CNN models exhibited varying performance results for the two datasets. The models and their performances are listed in

Table 4. Performance comparisons.

Network	Model	AUC	ACC	SE	SP
[30]	AlexNet	-	75.73	63.83	87.62
[36]	VGG16	-	0.90	0.91	0.80
Proposed Work	ResNet18	0.89	0.74	0.88	0.68

. In this table, the best performing model on the COVID19-CT dataset is ResNet-18, while the best performing model on the Mosmed dataset is MobileNetV2. Both of these models employed augmented and synthetically generated CT images for COVID-19 detection.

6. Conclusions

The proposed data-efficient ResNet-18 and ResNet-50 models give $0.88\%$ and $0.95\%$ sensitivity values. The sensitivity value is the measure of the COVID-19 detection. The proposed method shows that the COVID-19 was detected more accurately than other works. We found that we can generate accurate deep networks using limited data. We achieved this using synthetic and augmentation techniques.

The proposed machine learning-based systems provide faster healthcare services. Doctors and radiologists can also use these systems for collaboration to provide better help to patients.

The main novelty of the proposed method is that CNN networks can be generated from several available CT images during pandemic situations. It is well known that accessing CT images during a pandemic might be problematic. Therefore, this paper presents novel data-efficient networks for the diagnosis of the COVID-19 virus from CT images. This method builds on a convolutional neural network and a generative adversarial network (DGAN). The method utilises the DGAN model to produce unseen COVID-19 and normal CT images from a small dataset. Then, the CNN network uses these synthetically generated CT images to learn all possible virus signs on the images. The experiments prove that the proposed method allows higher accuracy results than the classic CNN networks.