Deep Learning in COVID-19 Diagnosis, Prognosis and Treatment Selection

Abstract

Deep learning is a sub-discipline of artificial intelligence that uses artificial neural networks, a machine learning technique, to extract patterns and make predictions from large datasets. In recent years, it has achieved rapid development and is widely used in numerous disciplines with fruitful results. Learning valuable information from complex, high-dimensional, and heterogeneous biomedical data is a key challenge in transforming healthcare. In this review, we provide an overview of emerging deep-learning techniques, COVID-19 research involving deep learning, and concrete examples of deep-learning methods in COVID-19 diagnosis, prognosis, and treatment management. Deep learning can process medical imaging data, laboratory test results, and other relevant data to diagnose diseases and judge disease progression and prognosis, and even recommend treatment plans and drug-use strategies to accelerate drug development and improve drug quality. Furthermore, it can help governments develop proper prevention and control measures. We also assess the current limitations and challenges of deep learning in therapy precision for COVID-19, including the lack of phenotypically abundant data and the need for more interpretable deep-learning models. Finally, we discuss how current barriers can be overcome to enable future clinical applications of deep learning.

1. Introduction

Pneumonia is an acute inflammation of the lower respiratory tract caused by a variety of infectious agents, including viruses, bacteria, and fungi, and is a moderately common condition that can occur in all populations. Coronaviruses (CoVs) belong to the Nidovirales order, Coronaviridae family, which comprises two subfamilies, namely Orthocoronavirinae and Letovirinae (International Committee on Taxonomy of Viruses) [1]. The serious acute respiratory syndrome coronavirus 2 (SARS-CoV-2), designated as COVID-19 by the World Health Organization (WHO) on February 11, 2020, is one of the extremely pathogenic β coronaviruses that infect humans. SARS-CoV-2 binds to the entry receptor ACE2, triggering the uptake and cleavage of the proteases cathepsin B and TMPRSS2 [2]. A comprehensive analysis of the structure of the SARS-CoV-2 ribonucleic acid (RNA) genome in infected cells will be vital for a comprehensive understanding of viral infection and treatment strategies, and it can facilitate target discovery and development of antiviral drugs [3,4]. COVID-19 is a respiratory syndrome. The basic symptoms are fever, chills, cough, sore throat, dyspnea, etc. (Figure 1) [5,6,7], which are similar to those of most pneumonia patients, making it difficult to distinguish between them. Therefore, the early diagnosis of COVID-19 is the most critical step in treating an infection, and diagnostic tools are often molecular biology methods such as the virus nucleic acid real-time reverse transcription polymerase chain reaction (Real-time RT-PCR) (Figure 1). However, RT-PCR has myriad defects, such as insufficient sensitivity leading to misdiagnosis, and taking patient samples can expose doctors to contaminated environments, which is risky. Another diagnostic method is radiology, which can diagnose COVID-19 by medical imaging methods such as Chest X-rays (CXRs) and computed tomography (CT) since most patients have pulmonary signs, such as ground-glass opacity of the lungs on CXR [7]. CT uses high radiation doses, which limits its use in children and pregnant women, while CXRs use low radiation doses at low cost; thus, CXRs are excellent candidates for lung imaging and may be an effective way to detect COVID-19 early and contribute to the differential diagnosis of COVID-19. Deep learning (DL) is a form of artificial intelligence that is roughly modeled on the structure of neurons in the brain and offers great promise in solving numerous problems in computer vision, medical images, diagnosis, natural language processing, robotics, etc. [8]. Epidemiological studies have increasingly focused on the transmission dynamics of COVID-19 (Figure 1), and understanding its transmission and reliably predicting trends are among the most critical factors in preventing the spread of the pandemic. DL and mathematical models can be used for different purposes, such as predicting future cases, analyzing previous infections, and estimating the basic reproduction number and virus doubling time [9], to generate epidemiological predictions useful for public health decision making and to assess the impact of different intervention strategies [10]. In the clinical treatment of COVID-19 patients, early identification of risk factors for critically ill patients and provision of appropriate supportive care are particularly important, as these can help clinicians formulate intermediate care guidelines, help reduce mortality, and alleviate medical resource shortages. Several common clinical scoring systems can be used for triage, but each of them has certain limitations, such as the need for laboratory variables that are difficult to obtain when hospitalized [11]. Therefore, it is hoped that machine learning can be exploited to develop multi-factor decision support systems with different datasets.

COVID-19 is shifting to precision medicine by the increasing availability and improved integration of multiple types of data [12,13,14,15]. Using and interpreting multiple high-dimensional data types in translational research or clinical tasks requires notoriously scarce and time-consuming expertise. Moreover, integrating multiple data types is more resource-intensive than interpreting a single data type and requires modeling algorithms that can learn from a large number of complex features [15,16]. The use of deep learning to automate these tasks and to support COVID-19 detection (identification of pneumonia caused by COVID-19 and other pathogens) is becoming more common. In the early years of the 2020 COVID-19 pandemic, differential diagnoses based on CXR results were mainly performed by radiologists. In a retrospective multi-institutional study by Stephanie et al., the CXR results of 508 patients were interpreted by four radiologists and showed that sensitivity and specificity varied with disease progression, with sensitivity ranging from 55% to 79% and specificity ranging from 83% to 70% within ≤2 days and 11 days after diagnosis, respectively [17]. DL models have the potential to leverage the complexity to provide meaningful insights and identify relevant, granular features across multiple data types, potentially learning features that have not yet been evaluated by radiologists, which will result in superior performance. In 2020, the first scheme used in the differential diagnosis of COVID-19, COVID-Net, had an accuracy rate of 93.3 and a recall rate of 87.1 [18]. The DL model continues to improve amid researchers’ efforts and growing image data, with the 2022 C-COVIDNet model, for example, registering an accuracy rate of 97.5 and a recall rate of 96.2 [19]. This paper provides an overview of the recent applications of DL in the diagnosis, prognosis, and treatment of COVID-19. We focus on DL applications for imaging data and integration of multiple data types. We briefly describe the current DL approaches to the applications covered in this review. In the following, we discuss specific applications of DL in COVID-19, including differential diagnosis, molecular virology studies, therapeutics, and epidemiological studies of COVID-19. Finally, we examine current challenges and potential strategies to enable the systematic application of deep learning in clinical settings.

2. Deep-Learning Methods

Detailed coverage of all deep-learning methods is beyond the scope of this review. Instead, we provide a summary of the DL approach in COVID-19. Deep learning has made significant progress in addressing the issues that have held back the AI community for years [20,21,22,23,24,25]. It is highly effective at discovering complex structures in high-dimensional data and is therefore suitable for numerous fields of science, business [26,27,28], and government [29]. In addition to breaking records in image recognition, it surpasses other machine-learning techniques in predicting the activity of potential drug molecules [30].

2.1. Basic Neural Network Methods

Neural networks were originally used as a major connectionist model [31], with early models emphasizing biological rationality but latter ones more concerned with the simulation of specific cognitive abilities, such as object recognition and language understanding. A feed-forward neural network (FNN), also known as a multilayer perceptron (MLP), is the earliest invention of a simple neural network with no feedback throughout the network and signals propagating unidirectionally from the input layer to the output layer [8]. Recurrent neural networks (RNNs) are a class of neural networks with short-term memory capabilities that are more in line with the structure of biological neural networks than feed-forward neural networks [8]. Convolutional neural networks (CNNs) are deep neural networks with local connections and weight sharing, inspired by the biological receptor field mechanism [8]. While MLPs perform successfully for general prediction, they are also prone to overfitting. RNNs have powerful computational and representational capabilities to approximate any non-linear dynamical system and Turing machines to solve all computable problems, typically for the analysis of sequential data such as text, speech, or DNA sequences. In contrast, CNNs have a certain degree of translation, scaling, and rotation invariance, and are the mainstream models in the field of computer vision [32]. CNNs have also been adapted to analyze non-image data, such as genomic data represented in vector, matrix, or tensor formats. The use of MLPs, RNNs, and CNNs in multiple areas of biomedical science has been proposed to assist clinicians and researchers, including in the areas of diagnosis, epidemiology, and clinical treatment.

2.2. Convolutional Neural Networks

Convolutional neural networks (CNNs) are inspired by the mechanism of biologically receptive fields [33,34]. The hierarchical model of the visual nervous system which is proposed by Hubel and Wiesel states that the network consists of an input layer (an array of photoreceptors), followed by a cascade of numerous modular structures. Each modular structure consists of two cascaded monocytes named simple cells (S-cells) and complex cells (C-cells). Among them, the S-cells collect feed-forward input, while the C-cells collect the inputs from the S-cells. Inspired by this, Kunihiko Fukushima proposed a multi-layer neural network with convolution and subsampling operations called Neocognitron. However, at that time, there was no backpropagation algorithm, and the new machine was trained by unsupervised learning. LeCun introduced the backpropagation algorithm into a convolutional neural network and had great success with handwritten digital recognition [35]. AlexNet is the first modern deep convolutional network model, which can be said to be the beginning of a real breakthrough in image classification in DL technology [36]. Instead of pre-training and layer-by-layer training, AlexNet uses many modern deep-network techniques for the first time, such as parallel training using GPUs, using ReLU as a non-linear activation function, using Dropout to prevent overfitting, and using data augmentation to improve model accuracy. These techniques have greatly facilitated the development of end-to-end DL models. Compared with fully connected (FC) networks, CNNS has many advantages [37]: (1) local connection—each neuron is only connected to a small number of neurons, reducing parameters and accelerating convergence. (2) Weight sharing—A group of connections can share the same weight. (3) Dimensionality reduction—the pooling layer uses the image local correlation principle to downsample the image, which can reduce the amount of data while retaining useful information. These three appealing features make CNNS one of the most representative algorithms in computer vision.

2.3. Graph Neural Networks

Graph neural networks (GNNs) are the neural networks that extend the idea of message transmission to graph structure data; the more typical ones are graph convolutional neural networks (GCNNs), graph attention neural networks (GATs) and message transmission neural networks (MPNNs). GCNNs generalize CNNs from conventional structures (Euclidean domains) to non-Euclidean domains and are specifically designed to analyze graphical data [38]. CNNs are closely related to recurrent graph neural networks, and instead of iteratively traversing node states using contraction constraints, GCNNs use a fixed number of layers with different weights in each layer and address recurrent interdependencies architecturally. Machine learning has been used to build robust and sustainable drug delivery channels in less time to facilitate the rapid synthesis and analysis of small amounts of compounds, which will help refine developed models and new designs [39]. The GCNN algorithm has had great success in the field of drug discovery [40].

2.4. Deep-Network Visualization

Deep-network visualization is the most straightforward way to explore hidden visual patches in neural units, and several visualization methods have been used to improve the interpretability of DL-based models for COVID-19. As an effective way to explain network decisions, attention maps can help to find the boundaries of the trained network, and researchers have used attention mechanisms to generate visual explanations for classifying models as an explanation tool [41]. To ensure interpretability, some researchers use the Gradient-Weighted Class Activation Mapping (Grad-CAM) algorithm, a technique commonly used to debug deep neural networks [42].

Besides classification and interpretability, the semantic segmentation method can also be applied to medical biometric images. Dividing the input image into regions where information needed for further processing can be extracted is called segmentation and is an important process used for evaluation in medical image processing and analysis to separate regions of interest (ROIs) from redundant pixels or unwanted regions as background [43]. A notable way to perform semantic segmentation is the use of generative adversarial networks (GANs). GANs can provide a data augmentation approach to combat the fragmentation of most available datasets, and GAN images of lung nodules can improve diagnostic efficiency for radiologists. In 2015, researchers proposed the concept of a fully convolutional network and applied the CNN structure to image semantic segmentation for the first time [44]. U-Net is a classical model in medical image processing [45], which extends fully convolutional networks for semantics segmentation (FCN) [44] to complement the usual contract network with successive layers. The pooling operator is replaced by an upsampling operator, allowing it to process very few training images and produce more accurate segmentations. U-Net has shown superior performance in medical image segmentation. The encoder–decoder–hop network structure adopted by U-Net has inspired subsequent research, and different extensions based on U-Net have been proposed in recent years [46,47,48]. UNet++, for example, redesigns skip paths and deep supervision to reduce the semantic gap between feature maps of encoder and decoder subnetworks, resulting in a possibly simpler optimization problem for the optimizer to solve [49] and more accurate segmentation to be achieved, especially for lesions appearing at multiple scales. The U-Net-based architecture is quite groundbreaking and valuable in medical image analysis. The diversity of approaches and DL interpretable methods in biomedical imaging enables clinicians and software developers to gain insight into deep-learning models during the development and deployment phases.

3. Deep Learning in COVID-19

Various deep-learning (DL) methods that utilize a combination of omics data and imaging data have been applied to the diagnosis, prognosis, and treatment options of clinical COVID-19 patients (Figure 2 and

Table 1. Summary of deep-learning methods, their relevant applications and brief technical descriptions of each DL model.

Application	Dl Method	Reference	Description
Diagnosis	CNN	Wang et al. [18]	Training CNN on CXR to diagnose COVID-19.
	CNN & transfer learning	Chouhan et al. [50]	By transfer learning, knowledge from one domain is transferred to the target domain, enabling better learning results in COVID-19 disease management.
	CNN & CapsNets	Afshar et al. [51]	Each layer of the capsule network (CapsNet) consists of several capsules, each representing a specific image instance in a specific location through multiple neurons. The transformation matrix distinguishes the training by backpropagating each EM expansion iteration between adjacent capsule layers.
	CNN & 2D/3D	Bayoudh et al. [52]	Uses the potential synergy between a pre-trained VGG16 model (i.e., 2D CNN) and a shallow 3D CNN.
	Semantic segmentation	Zhang et al. [53]	An improved dense GAN is developed for extended datasets and a multi-layer attention mechanism is proposed in conjunction with U-Net for COVID-19 lung CT image segmentation.
Early warning system	Cox proportional risk model	Dong et al. [54]	Depends on regression analysis to confirm the association between predictive covariates (such as clinical features) and event occurrence risks (such as “death”).
	Semantic segmentation	Sharma and Mishra [55]	The diagnosis is interpreted by activating the gradient-CAM mapping to generate a location-map survey model for each disease.
Prediction	RNN	Kafieh et al. [56]	Information is extracted from COVID-19 data sources and used to train models to predict the time of the next occurrence.
	LASSO	Liang et al. [57]	Regression analysis is performed to identify associations between predictive covariates and the risk of event occurrence, thereby predicting malignant progression of the disease.
Novel coronavirus molecules	PrismNet	Sun et al. [12]	In vivo data is integrated to construct and train deep neural networks to simulate the interaction between RBP and RNA targets.
	DeepTCR	Sidhom and Baras [58]	T cell receptor sequences in immune coding databases are analyzed to understand immune genomic differences and models are used to predict critically ill patients.
Control	RL	Yang et al. [59]	Establishes a proactive surveillance system to slow the spread of COVID-19 by warning individuals in areas of interest.
	CNN	Sethi et al. [60]	The initial face data is segmented by transfer learning technology and then sorted by mask classifier.
Treatment	GCN	Jin et al. [61], Zeng et al. [62]	Combined molecular structure and biological target models predict synergistic drug combinations.
	MLP	Zeng et al. [63]	Using Amazon Web Services computing resources and a deep-learning framework to identify reusable medications.
	MBMS	Roh et al. [64]	Using the Internet of Things and deep learning to detect patients’ medication behavior and feedback to doctors.

). However, even with the emerging deep-learning methods, human intervention is still essential in the clinical diagnosis and treatment of COVID-19 patients. Therefore, the goal of DL is not to surpass or replace humans, but rather to provide decision-support tools to help researchers studying COVID-19 and health professionals in the clinical management of COVID-19 patients.

3.1. Deep Learning for Diagnosis of COVID-19

COVID-19 is traditionally diagnosed by real-time RT-PCR testing of respiratory or blood samples [65] (Figure 1). However, considering the simplicity and sensitivity of the test, CXR radiography has become the mainstay of screening, triaging, and diagnosing varieties of COVID-19. Researchers [5,7] noted that the majority of the COVID-19 positive cases in their study presented bilateral radiographic abnormalities in CXR images, such as ground-glass opacity, bilateral abnormalities, and interstitial abnormalities in CXR and CT images. Indeed, early works on COVID-19 imagery identified the existence of pulmonary lesions in non-severe and even recovered patients [66]. Among various deep-learning classifiers, CNNs, in particular, have been enormously effective in computer vision and medical image analysis tasks. COVID-Net represents one of the earliest convolutional networks designed for detecting COVID-19 cases automatically from CXR images [18]. This architecture design consists of two parts: a human–computer collaborative design approach and a machine-driven design exploration part. A lightweight residual projection–expansion–projection–extension (PEPX) design pattern is used in this architecture. An interpretability-driven check was also performed for decision validation, achieving 87.1 percent recall and 93.3 percent accuracy. To reduce the amount of data and time required for training, transfer learning may be an appropriate solution [67]. Researchers have achieved state-of-the-art performance on pneumonia recognition using ensemble models with pre-trained architectures trained on ImageNet, using AlexNet, DenseNet121, Inception V3, GoogLeNet and ResNet18 to achieve 96.4 percent accuracy and 99.62 percent recall on unseen data from the Guangzhou Women and Children’s Medical Center dataset [50]. COVID CAPS [51], a CNN model based on CapsNets, was able to process small data sets with 95.7 percent accuracy. Most computer vision tasks, such as image classification, semantic segmentation, object recognition, etc., are based on 2D CNN [37]. Since high-dimensional data is difficult for humans to understand, the application of multidimensional CNN in 3D is not common, as described above. Hybrid-COVID [52] is a novel Hybrid 2D/3D framework based on CNN that uses the potential synergy between a pre-trained VGG16 model (i.e., 2D CNN) and a shallow 3D CNN to efficiently and effectively diagnose COVID-19 from CXR images with 96.91 percent accuracy. Researchers proposed a medical image segmentation method for COVID-19 lung CT image segmentation using an advanced high-density GAN dataset combined with a multi-layer attention mechanism approach from U-Net [53].

In addition to CXRs, researchers can use several clinical approaches to diagnose COVID-19. Researchers proposed an explainable classification model to automatically differentiate COVID-19 and community-acquired pneumonia (CAP) from healthy lungs in radiographic images [41]. Lung Ultrasound (LUS) imaging can also be used as an alternative to CXRs and CT to identify lung disease [68].

3.2. Deep Learning for COVID-19 Early Warning System

In addition to the diagnosis of COVID-19, it is also important to predict the malignant progression of COVID-19 [69,70,71,72,73]. The sudden progression to critical illness in patients with COVID-19 is a major concern [74,75], and early identification of the malignant progression of COVID-19 can reduce the heterogeneity of patient stratification, optimize diagnosis and treatment, improve the efficiency of medical resource allocation as well as the response capacity of medical systems to emergencies, and, ultimately, reduce mortality [76]. A study has shown that a deep-learning-based survival model [57] can predict the risk of critical illness in COVID-19 patients based on clinical features at admission. By creating an online computational tool to classify patients on admission, the model in turn identifies patients at high risk of serious illness, ensuring that patients most at risk of serious illness receive early access to appropriate care and allowing for the efficient allocation of health resources.

The Cox proportional risk model (CPH) is a widely used statistical model which is a multivariate linear regression model that depends on regression analysis to confirm the association between predictive covariates (such as clinical features) and event occurrence risks (such as “death”). For example, researchers used least absolute shrinkage and selection operator (LASSO) regression and multivariate Cox regression analysis to identify important factors associated with COVID-19-related hospitalization rates [54]. In this study, hypertension, higher neutrophil-to-lymphocyte ratios, and N-terminal pro-b-type natriuretic peptide (NT-proBNP) values were identified and used to develop prognostic line maps. Linear plots show good discrimination and capability to predict the 14-day and 21-day survival probability of COVID-19 patients and evaluation by C-index, AUC values, and calorimetric plots indicate good performance and high value for clinical use. In biomedical images, COVID-MANet uses a semantic segmentation method to categorize CT images of COVID-19-positive specimens into mild, moderate, severe, and critical based on the proportion of infected disease pixels, with mapping of the activation Grad-CAM used to interpret the diagnosis and generate a map survey model of location for each type of disease, except for the interpretation of the map survey model for COVID-19 infection. This increased understanding of risk factors, and the resulting risk stratification, can help clinicians develop medical guidelines to improve the management of risk-stratified care for COVID-19 patients. In the actual clinical experience, mild cases of COVID-19 are usually self-limited, but severe cases require more medical attention. Online submission of clinical information and medical personnel can be used to predict the risk index for admission of patients with shunt disease, as well as the treatment of patients with corresponding scheduling plans, ensuring that patients receive prompt follow-up treatment as early as possible so that medical resources can be used effectively.

3.3. Deep Learning for COVID-19 Prediction

In the context of the COVID-19 pandemic, it is important to accurately predict the development of the epidemic [77] and to identify and short-term estimate the final size and peak time of the epidemic as early as possible [78] (Figure 1). Early prediction [79,80] using mathematical and statistical models combined with available data can be effective in helping governments develop appropriate prevention [81] and control measures [82]. In the early days of the outbreak, many mathematical models were used to predict the COVID-19 pandemic, such as crowd flow models that reduced infection rates among people through enforcement measures, phenomenological models for short-term prediction of COVID-19, a dynamical model (a developed generalized susceptible-exposed-infected-removed (SEIR) model), a susceptible-infectious-recovered-dead (SIRD) model, gated recurrent units (GRUs), long short-term memory (LSTM), an autoregressive integrated moving average (ARIMA) model, simple RNN, bidirectional LSTM (BiLSTM), variational autoencoder (VAE), a neural network model, and an ensemble model using four machine learning methods, namely, support vector machine, logistic regression, gradient boosted decision tree, and neural network.

Kafieh et al. [56] described a kind of prediction method based on deep learning that could help medical and government agencies in pandemic preparation and adjustment. Three tests were used in the study of this model for predicting COVID-19 disease, including basic information, COVID-19 data, and detailed information for each country. In the study, the relevant information was first extracted and processed from the data sources, and then the COVID-19 data from each country was used to train these different machine learning models. Results of five promising models are reported in experiments using different machine learning models, including random forest (RF), LSTM with regular features (LSTM-R), MLP, multivariate LSTM (M-LSTM), and LSTM with extended features (LSTM-E). After evaluating five models, the results show that M-LSTM is the best network model to identify the true size of the pandemic. This model expects that the prediction in each case is consistent with the previous action, and that any new action will lead to different results. Therefore, if the action has a positive impact, the predicted number of infections will decrease. Negative behavior, on the other hand, leads to an increase in the number of predictions. The underlying assumption of the model is the stability of environmental measurements; however, since we live in an uncontrollable situation, each decision changes the trajectory of the epidemic. Therefore, the goal of this model is to determine the intensity and timing of the peak, the expected total number of cases during the COVID-19 pandemic, and the impact of government policies on the number of infections. By identifying these outcomes, the allocation of resources for primary prevention, secondary prevention, risk communication, and preparedness planning can be improved.

3.4. Deep Learning for Novel Coronavirus Molecules

SARS-CoV-2 is the main cause of the COVID-19 pandemic [3,4]. During the course of the ongoing epidemic, the virus has mutated many times. Structural differences between different SARS-CoV-2 strains may also be responsible for their different infectivity and transmission rates. Therapeutic interventions for COVID-19 can be further refined by studying the viral genome and its interactions with the host. In one study, researchers built PrismNet, a deep-learning tool [12], to predict 42 host proteins that bound to SARS-CoV-2 RNA and drugs. Antisense oligonucleotides targeting structural elements and FDA-approved agents that inhibited SARS-CoV-2 RNA-binding proteins significantly reduced SARS-CoV-2 infection in human liver and lung tumor cells, thus revealing multiple therapeutic candidates for COVID-19 treatment.

As an RNA virus, the RNA genome itself is the regulatory center that controls and enables its function. RNA molecules fold into tanglesome higher-order structures. Therefore, a more full-scale analysis of the structure of the SARS-CoV-2 RNA genome in infected cells is important for a full understanding of the viral infection and treatment strategies. It has been shown that RNA structural data can be evaluated with cutting-edge deep-learning techniques to precisely predict RNA-binding proteins (RBPs)–RNA binding in vivo by integrating neural network models of binding and matching RNA features in the cell. Overall, this study identifies many single-stranded regions in the SARS-CoV-2 genome, reveals and validates structural elements (including coding regions) with strong coevolutionary support throughout the genome, and shows that functional RNA structural elements can be targeted by small molecule compounds to disrupt viral infectiousness. It can facilitate target discovery and development of antiviral drugs. However, there are some limitations to this study. Firstly, the obtained structural information of SARS-CoV-2 RNA is not ideal and requires further transformation. Secondly, the model can only report the RNA structure information of a single nucleotide, but it cannot directly reveal the higher-order structure information (including the tertiary RNA structure). Thirdly, host factor predictions do not take into account cellular background information such as protein abundance and localization data. In conclusion, studies have demonstrated that several FDA-approved drugs can effectually restrain viral infection in different cells through the SARS-CoV-2 RNA trans-complementation system, but the mechanism of action needs to be further investigated.

In another study, researchers analyzed T-cell receptor sequence (TCR-SEQ) data from the open-access Immunocoding Database to understand immunogenomic differences that may contribute to different clinical outcomes [58]. They identified two cohorts in the database with clinical outcome data reflecting disease severity and used DeepTCR, a multi-instance deep-learning repertoire classifier, to predict severe SARS-CoV-2 infection in patients based on their repertoire sequencing. The study demonstrated that severely infected patients had higher T-cell responses connected with SARS-CoV-2 epitopes and authenticated the epitopes responsible for these responses. The results showed that the clinical course of SARS-CoV-2 infection varied greatly and was related to some antigen-specific responses. Of course, there are many limitations to this study. One limitation is the overfitting of the model, which may be one reason for the lack of cross-validation of the model. Another non-trivial limitation is the limitation of the database, which still lacks clinical data in the actual fitting process.

3.5. Deep Learning for COVID-19 Control

During this COVID-19 pandemic, unprecedented public health measures have been taken to control the spread of the SARS-CoV-2 virus. One study used deep reinforcement learning, where algorithms were trained to try to find the optimal public health strategy that maximizes the total return to control the spread of COVID-19 [59,83]. The results of the proposed algorithm are analyzed for realistic times and intensities of lockdown and travel restrictions. Researchers have proposed an elementary data-driven approach that utilizes state-of-the-art deep reinforcement learning (RL) algorithms to discover optimal lockdown and travel restriction policies for some countries and regions to reduce the burden of COVID-19. However, this study has some limitations, and early implementation is difficult due to inconsistent testing and reporting of emerging COVID-19 outbreaks and incomplete data. In addition, the study focused only on the health benefits to the population from controlling the spread of COVID-19, without balancing the negative effects of economic and social consequences.

At the same time, given the lack of effective antiviral drugs and restricted medical resources [61], WHO recommends many measures to control infection rates and avoid exhausting limited medical resources [84]. Wearing a mask is one of the non-pharmaceutical interventions that can be used to cut off the main source of SARS-CoV-2 droplets excreted by an infected person [85]. In one study [60], researchers designed a high-precision and real-time technology, called ResNet 50, which can effectively identify non-masked faces in public places to enforce the wearing of masks. Of course, the study has some limitations. Firstly, it cannot tell the difference between a normal mask and a surgical mask. Secondly, the available datasets are small, which makes model training difficult.

3.6. Deep Learning for COVID-19 Treatment

There is an urgent need for effective treatment for COVID-19. However, the discovery of monotherapies with activity against SARS-CoV-2 has been challenging. Combination therapies play a non-negligible role in antiviral therapy because they improve efficacy and reduce toxicity. In contrast, drug synergies usually occur by inhibiting discrete biological targets. One study has proposed a neural network architecture called ComboNet (similar to protein–protein interaction networks) for the joint learning of drug–target interactions [62,86] and drug–drug synergies [61]. The model consists of two parts; the drug–target interaction module and the target–disease association module. By introducing additional biological information, the model significantly outperforms previous approaches in terms of collaborative prediction accuracy. Despite limited training data on drug combinations, the study empirically validated the model predictions and found two drug combinations: remdesivir and IQ-1S (an effective and specific c-Jun N-Terminal Kinase Inhibitor), and remdesivir and reserpine, both of which showed strong in vitro anti-SARS-CoV-2 synergies.

In addition to combination therapy, drug reuse offers a promising approach to the development of COVID-19 prevention and control strategies. One study reported an integrated, web-based deep-learning approach for identifying reusable drugs for COVID-19 (called Coronavirus-knowledge graph embedding, Cov-KGE) [63]. The researchers built a comprehensive knowledge graph that used Amazon’s Amazon Web Services computing resources and Web-based deep-learning frameworks to identify 41 reusable drugs (including indomethacin, dexamethasone, toremifene, and niclosamide) whose therapeutic association with COVID-19 has been validated using transcriptional and proteomic data from human cells infected with SARS-CoV-2 and data from ongoing clinical trials.

Of the 41 drug candidates, 9 are in or have been in clinical trials for COVID-19, including thalidomide, ribavirin, methylprednisolone, umifenovir, suramin, tetrandrine, dexamethasone, azithromycin, and lopinavir. This study also provides a powerful, integrated deep-learning approach for the rapid identification of reusable potential therapeutics for COVID-19. However, it is important to note that all predictive drugs must be tested in randomized clinical trials before they can be used in patients with COVID-19.

In the COVID-19 era, an intelligent medication behavior monitoring system (MBMS) is also needed to monitor patients stably [64], because many patients cannot easily go to the hospital, or the medical staff in the hospital cannot monitor the patient’s condition in real-time. Similar to the use of the Internet of Things (IoT) for electrocardiogram (ECG) systems to predict cardiovascular disease and electronic medical records [87,88], one study designed a medication behavior monitoring system using IoT and deep learning to effectively detect various activities of patients, avoid perceptual errors, and improve user experience. The system uses a human activity identification scheme to identify the medication status in a timely manner and proactively relay various notices to the patient’s mobile device. Information measured on a patient’s medication status is transmitted to doctors, allowing them to perform remote treatments on a regular basis. Experiments show that the proposed system can automatically detect all medication behaviors of patients and inform doctors effectively, improving the accuracy of monitoring the medication behavior of patients. The system is applied to COVID-19 treatment, which can improve the treatment process of patients and collect a large amount of medical data and patient disease data through various sensors. It can solve various problems such as remote treatment and rapid recovery of patients so that patients can receive treatment quickly and comfortably.

Figure 2. The deep-learning methods for COVID-19. LASSO: least absolute shrinkage and selection operator; RL: reinforcement learning; Cov-KGE: Coronavirus-knowledge graph embedding; MBMS: medication behavior monitoring system; Grad-CAM: Gradient-Weighted Class Activation Mapping; GANs: generative adversarial networks; M-LSTM: multivariate long short-term memory; CPH: the Cox proportional hazards model; DL: deep learning; MLP: a multilayer perceptron; CNN: convolutional neural network; RNN: recurrent neural network; GNN: graph neural network.

Figure 2. The deep-learning methods for COVID-19. LASSO: least absolute shrinkage and selection operator; RL: reinforcement learning; Cov-KGE: Coronavirus-knowledge graph embedding; MBMS: medication behavior monitoring system; Grad-CAM: Gradient-Weighted Class Activation Mapping; GANs: generative adversarial networks; M-LSTM: multivariate long short-term memory; CPH: the Cox proportional hazards model; DL: deep learning; MLP: a multilayer perceptron; CNN: convolutional neural network; RNN: recurrent neural network; GNN: graph neural network.

4. Challenges and Limitations: The Road to Clinical Implementation

Artificial intelligence saves costs and increases agility [14], and it may be superhuman in its ability to interpret complex data [89]. However, when the model is applied to real human health data, there may be some bias, resulting in false information. Challenges to its use include, but are not limited to, difficulties in collecting data, internal and external validation, ethical issues, and interpretability [14]. The utility of these models in clinical applications will be limited without careful consideration of the methods and biases used to train the models and the data available for training the models. Therefore, we highlight the following issues in the clinical application of deep learning to COVID-19.

4.1. AI Interpretability

Artificial intelligence is what many call a “black box” (a system that produces output without any explanation or reason) [32]. Although it has a high degree of accuracy, it is often poor in interpretability because it is difficult or impossible to explain the rationality behind the action [90]. While this is acceptable in many cases, clinical applications are not simple, hence the need for an ideal AI-based system or model that can produce accurate predictions and provide a reasonable explanation for those predictions. Many of the AI-based models and methods described in this article involve some form of explanatory analysis to some extent. However, accurate interpretation of AI outputs can be challenging in the face of complex situations. As a result, it has been proposed to build more interactive interpretable models by “talking” to the AI. This approach has the potential to not only demonstrate the acceptability of AI predictions, but also improve the transparency of doctor–patient communication.

4.2. Data Limitations

The data should be of high quality and reliable [14]; the lack of high-quality, properly labeled training data affects deep-learning models in almost all of the applications we discuss. In the process of model training, we cannot obtain the data that can be used to fully train deep-learning models. However, with the development of COVID-19, the problem of insufficient volume of data can be largely resolved. Nevertheless, data quality issues remain, and it is challenging to train a good deep-learning model on large and diverse data sets. In addition, the temporal nature of the data is also a challenge. Many existing deep-learning models assume inputs based on static vectors [91]; however, since diseases always evolve and change over time in uncertain ways, new solutions need to be developed.

5. Discussion

The emergence of new medical technologies indicates that deep learning is more widely applied. In the above article, we introduced some applications of DL in dealing with COVID-19. However, there is still an unresolved problem in that there is a lack of a comprehensive reference model for the development of COVID-19. Based on the exploration of the other models mentioned above, we propose methods to build this reference model, which is intended to help future researchers research and develop models for the comprehensive detection of COVID-19.

The following aspects should be paid attention to when constructing the model [87]: (1) Information collection: determine the biological signals to be collected and the detailed information on COVID-19, for example, body temperature, respiratory rate, CXR image data, COVID-19 data (including infection, death, cure, etc.), the molecular structure of the circulating strain, the drugs used, etc. (2) Information capture method: select the method used to capture biological signals and information. (3) Experiment scope: define the experiment content so as to evaluate the calculation methods and human resources required for the experiment. (4) Application platform: use existing databases or communication platforms to build models. (5) Model detection: after establishing the model, use the collected data to detect the performance and integrity of the model.

6. Conclusions

Deep learning (DL) has played an important role in many aspects of the fight against COVID-19, helping healthcare systems respond more accurately and quickly to COVID-19. Artificial intelligence and deep learning are expected to be used in the development, validation, and deployment of decision support tools to facilitate the accurate diagnosis and treatment of COVID-19. In this review, we have presented a number of promising applications of DL in various areas of COVID-19, including differential diagnosis, molecular virology research, therapeutics, and epidemiology. Interestingly, as the research matures, the combination of multimodal learning and interoperability can reveal new perspectives. Deep-learning methods will become more widespread as new medical technologies improve the quantity and quality of healthcare. Finally, clinical validation of interpretable DL methods will play an important role in making DL accessible in routine patient care.