1. Introduction
In recent years, artificial intelligence (AI) techniques have been highly successful in assisting mankind in various ways, such as improved healthcare, ambient assisted living, smart services, awareness/forecasting of future events, etc. Three major elements have significantly contributed to the success of AI developments in real-life scenario(s): (i) the availability of big data stemming from diverse sources, (ii) advancements in newer learning models as well as computational power, and (iii) the evolution of deep learning (DL) models and high-performance computing infrastructures [
1]. FL has various market use cases and commercial applications focusing on data science, healthcare, industry, and education [
2]. Despite their many benefits, AI techniques face multiple challenges due to poor quality and unstructured data, the non-availability of data for certain tasks, and/or the inability to handle and process data originating from the real-time domain. Though AI has shown a very huge success rate and many remarkable developments exist worldwide, most domains (e.g., real-time, personal data-driven applications, etc.) are still not in a position to leverage AI techniques commercially due to the following major concerns:
Users are highly concerned about their data privacy, and therefore, acquiring and using personal data is very challenging.
The confidentiality of personal data (also known as users’ data) can be compromised because the data are mostly collected in some central place (e.g., server) for central learning (CL).
Most processing in CL-based environments is performed in a black-box manner. Hence, privacy violations cannot be restricted.
The data concerning an individual can be of multiple types such as spatial-temporal activities, demographics, medical data, and physiological readings, to name a few. Depending on the diversity and size of data, the chances of privacy breaches can be very much higher in CL environments.
As cited above, privacy has been a major concern in the adoption of AI techniques in commercial environments. Recently, most digital applications such as contact tracing apps developed for containing the spread of the novel coronavirus disease 2019 (COVID-19) were not welcomed by many people across the globe due to privacy concerns [
3]. Thanks to the emergence of the federated learning (FL) paradigm, many privacy issues can be addressed proactively [
4]. FL is considered a privacy-aware ML model in which privacy is ensured by not centralizing personal data at some central place (e.g., servers). Specifically, FL is a special case of distributed machine or deep learning (ML or DL), which enables
N clients to jointly build an ML/DL model across decentralized data sources without explicitly aggregating the data at some central place [
5]. The key difference between FL and centralized learning (CL) is given in Equation (
1):
By bringing algorithms close to data, FL is regarded as one of the mainstream solutions for privacy preservation. Due to its core abilities such as breaking data island and data silo problems, FL has received considerable attention from the research community, and FL applications are expanding to many fields, including healthcare, finance, supply chains, smart cities, robotics, and education. Furthermore, FL can work with many advanced techniques such as blockchain, the Internet of Things (IoT), the industrial IoT (IIoT), and edge/fog computing infrastructures. In the coming years, FL will likely replace central ML/DL-based systems and be widely used in many commercial sectors. Recently, FL has emerged as a viable solution for securing critical infrastructures such as the IoT from a privacy preservation point of view [
6]. FL has helped preserve the privacy of healthcare data [
7], cloud computing environments [
8], vehicular environments [
9], edge computing [
10], big industrial data [
11], and medical systems [
12] and has helped in intrusion detection and privacy preservation in IoT systems [
13]. Furthermore, FL has been extensively used to preserve privacy in AI applications, collectively called privacy-preserving AI [
14]. Unfortunately, the FL paradigm has various issues due to its decentralized architecture. For example, the invisibility of training data, centralized aggregation, and the training process on the client’s side can result in some security and privacy issues [
15]. Furthermore, due to decentralized processes, convergence and client retention/selection is also very challenging. Recently, many studies have been designed to advocate the uses of FL in many sectors, and to lower the security and privacy attacks. However, the extended taxonomy of FL’s applications with special emphasis on COVID-19 remained unexplored. In this paper, we investigate the recent advancements in the FL paradigm to highlight the significance of FL in the modern era and the challenges impacting FL’s broad adoption in commercial settings.
Although FL has been widely used in many sectors, the future of healthcare is deeply connected with the success of FL because the privacy preservation of medical data is imperative [
16]. The recent COVID-19 pandemic has also tested the technical strength of FL, and it was widely used to handle many parts of this pandemic [
17]. Some of the noticeable applications of FL in the COVID-19 era are: COVID-19 detection from x-ray images [
18,
19], clinical outcome prediction [
20], mortality prediction [
21], pre-processing of COVID-19 data [
22], COVID-19 vulnerability map construction [
23], privacy-preserving data collection [
24,
25], Ct-image-based COVID-19 detection [
26,
27], infected regions segmentation [
28], and face mask detection in dense crowds [
29]. Considering these innovative applications, the concrete and complete overview of FL in the context of COVID-19 is meaningful and important, which is the main motivation behind this research. We affirm, with the contributions of previous generic surveys [
30,
31,
32,
33,
34], that our focus is on COVID-19-related FL developments, which remained unexplored in the current literature. In addition, none of the previous surveys have highlighted the synergies of FL with other emerging technologies to fulfill its promises in the context of COVID-19. The major contributions of this paper are given as follows:
A review of the applications of FL to COVID-19: This article discusses the technical applications of FL along with model and data details focusing on COVID-19, which can help understand recent state-of-the-art (SOTA) developments of the FL paradigm.
Synergies of FL with other technologies: This work highlights the synergies of FL with other technologies that are imperative for privacy preservation, broadening application horizons, and/or enhancing service scenarios. This extended knowledge assists in understanding the technology stack of the FL paradigm.
Review of open-source FL frameworks: This work analyzes the recent open-source development of FL paradigms which can help in designing scalable and reliable FL models in the medical field by addressing their limitations.
Potential challenges and future research directions: This work suggests valuable technical recommendations to address the key challenges of this SOTA decentralized paradigm.
To the best of the authors’ knowledge, this is the first work centering FL with regard to COVID-19, and we believe this could pave the way to understanding FL’s role in the COVID-19 era.
The rest of this paper is structured as follows:
Section 2 discusses the background of the FL paradigm including working methodology (e.g., clients and server responsibilities), main types, and emerging research areas concerning the FL paradigm.
Section 3 presents technical applications of the Fl in the context of COVID-19.
Section 4 presents the latest synergies of FL with the other emerging technologies to enhance the privacy level as well as the application horizon of FL technology.
Section 5 discusses the open-source implementations of FL with a special focus on medical-related developments.
Section 6 highlights the challenges of FL in modern times and suggests valuable recommendations to address those challenges.
Section 7 compares this work with the existing works. We conclude this paper in
Section 8.
3. Technical Applications of Federated Learning in the Context of COVID-19
FL has demonstrated its effectiveness in many sectors, including supply chains, robotics, finance, smart cities, smart healthcare, natural language processing and modeling, the insurance sector, social networks, and the IoT, to name a few. In this paper, our focus is on healthcare and especially COVID-19, and therefore, we summarize the achievements of FL in the healthcare sector only. Before presenting the detailed applications of FL in the COVID-19 era, we present the overall applications of FL in the healthcare domain in
Figure 4.
As shown in
Figure 4, FL has been contributing significantly to the healthcare sector with diverse applications. The input to these applications are data of patients in the form of electronic health records (EHR), data from wearables, sensor readings, demographics, vital signs, images, X-rays, medical histories, and visuals, audio, and videos of various body organs. FL trains high-quality models for neurological (and other) disease diagnoses. Recently, FL and other AI-based developments have been used to assist doctors in performing various activities in hospitals. In the coming years, FL will be one of the mainstream technologies in performing various operations/services. Some studies explored the use of FL in tumor identification using ultrasound images and compared FL architecture and traditional AI architectures [
39]. In this analysis, FL was proven more effective than the traditional ML/DL-based training architectures. Some studies have explored the usage of FL in hearing aids, survival prediction in patients with lung cancer, and confidentiality-aware data processing [
40,
41,
42]. Ngo et al. [
43] developed a SOTA approach by combining DL and FL for diagnosing cerebellar ataxia (CA) using image data. The proposed approach yields higher diagnosis accuracy without feature engineering and ensures data privacy in real-life deployable scenarios. Islam et al. [
44] proposed an FL-based secure data-collection method from IoT devices using drones and blockchain. The proposed approach yields better results in proof of concept experiments, highlighting multiple benefits such as data collection, storage, privacy preservation, security, and execution time. Similarly, FL has also contributed to lowering the effects of this pandemic on the general public when vaccines were unavailable. At present, there are various commercial deployments of FL to control COVID-19 using a variety of data sources. FL can work with many digital technologies, and therefore, the application/use of FL is more dominant than traditional AI techniques. Through a detailed analysis of the SOTA published in the past three years, we summarize practical examples of FL in
Table 2.
As shown in
Table 2, FL has many practical applications in the context of COVID-19. These applications have helped many entities in lowering the severe effects of COVID-19. Further information about the applications of FL in medical fields can be learned from previous survey [
81,
82,
83,
84]. Based on the extensive analysis, we found that most FL applications in the COVID-19 context are detection, prediction, diagnosis, and forecasting. In addition, the most commonly used data types are X-rays, images, and data from wearables. From the AI model’s point of view, CNN and common ML models were frequently used in experiments. This knowledge can assist researchers in customizing existing developments as well as proposing new models for enhancing accuracy, precision, recall,
score, etc. Apart from the data sources mentioned in
Table 2, some FL applications have used signals data as well in improving medical services focusing on COVID-19 [
85,
86,
87]. The implementation of FL with these heterogenous data sources helped in constraining the spread of the virus in a privacy-preserving way. In addition to FL, many other digital technologies have also contributed to lowering the effects of the pandemic on the general public. In
Figure 5, we summarize the key technologies that have helped mankind mostly in the pre-vaccine era.
As shown in
Figure 5, many technical developments have been made across the globe to combat the virus. In addition, the developments of contactless services have also boosted AI-related developments across the globe. In the post-COVID-19 era, more disruptive technologies will further reshape the industry.
Figure 5.
Overview of key technologies other than FL that have contributed to handling the pandemic (Adopted and enhanced from [
88]).
Figure 5.
Overview of key technologies other than FL that have contributed to handling the pandemic (Adopted and enhanced from [
88]).
4. Recent Synergies of Federated Learning with Other Emerging Technologies in the Context of COVID-19
Due to the distributed nature, the invisibility of training data, and untrustworthy clients’ behavior, FL could not unleash much of its potential [
89]. For example, FL failed to fully protect training data from adversaries because sometimes gradients/parameter sharing can weaken the privacy of participants. Similarly, due to the open nature of training, any party (including malicious entities) can join the system, and corrupt the training process with either wrong data or wrong models. Furthermore, FL cannot guarantee that the number of participants in the initial rounds will remain until the end of the training process. To overcome these challenges, FL has been extensively integrated with other emerging technologies. For example, to protect the privacy of training data, FL has been integrated with the differential privacy [
90]. To further protect personal data in industrial settings, FL has established synergy with the blockchain [
91,
92]. In
Table 3, we highlight the main synergies of FL with other emerging technologies in the context of COVID-19.
Apart from the analysis presented in
Table 3, some recent surveys have highlighted the synergies in one or more aspects of the FL paradigm [
115,
116]. Furthermore, the synergies of FL are increasing data day by day to improve various technical aspects of this technology [
117,
118]. These synergies have also extended the applicability of this technology to many commercial and industrial sectors. Furthermore, some of these integrations are made to lower the communication and computation overheads of this technology [
119]. In addition, some integrations are improving the privacy aspects of this technology [
120]. In the coming years, the synergies of this paradigm with emerging technologies are likely to expand to advance its capabilities.
5. Open Source Implementation Frameworks of Federated Learning
In this section, we discuss the open-source implementations of the FL paradigm that have been experimentally tested on some real-world datasets. Although many open-source frameworks have been developed in the recent past, we present only the main frameworks that are accessible for rapid validation and experimentation in
Figure 6. Most frameworks listed in
Figure 6 can work with any dataset, but only a few provide robust support against attacks (i.e., Privacy FL). The tutorials and documentation about most frameworks are incomplete/partial except for OpenFL and PySft. Only a few frameworks provide support for other libraries and data partitioning. Most frameworks run on traditional CPUs, and only a few can run on large-scale hardware such as graphical processing units (GPUs). In addition to these open source developments, some propriety frameworks such as IBM FL [
121], Substra [
122], and NVIDIA CLARA [
123], etc., have also been developed, which are not yet publicly available for rapid testing and validation. Moreover, there exists an open-source implementation of FL for some other emerging technologies (e.g., the IoT) [
124]. Interestingly, only two frameworks (e.g., OpenFL and Fed-BioMed) provide support for medical applications. By using the FL frameworks listed in
Figure 6, possible risks of exposing patients’ sensitive health-related information can be resolved. In addition, the FL strategy enhances the training performance on medical data by exploiting big and large-scale datasets and offloading most processing to the local devices in a network, which would not be possible with the centralized AI technique.
To provide technical information concerning two medical-related frameworks, we compare both frameworks on technical grounds in
Table 4. The analysis presented in
Table 4 can help to understand and further improve the implantation of these frameworks. Lastly, these implementations have been used as the baseline in most studies and have been rigorously enhanced.
6. Challenges and Recommendations
When it comes to the actual deployment of FL in real-life healthcare settings, there exist multiple challenges. Although some challenges have been described in the previous research, a clear picture from all perspectives is still missing. In this work, we highlight most challenges of the FL paradigm and suggest valuable recommendations to address those challenges. As shown in
Figure 7, we have categorized these FL challenges into nine main categories, which remained unexplored in the current literature.
Apart from the challenges cited in
Figure 7, explainability, transparency, and fairness are also the main challenges of FL in the context of healthcare [
125,
126]. We will present each of these challenges in detail in the following paragraphs.
Client-related challenges: In the FL paradigm, clients are regarded as independent, which means they can perform most activities autonomously. Hence, they can leave the system at any time, which can lead to longer convergence and disturbs the training process [
127]. The prevention of the client’s dropout is a longstanding challenge in FL. In addition, the selection of clients who can contribute good models/data is also a non-trivial task. In some cases, clients make bots with each other to carry out any sort of malicious activities, which makes FL results unreliable or corrupts the training process. In addition, some clients hold up the data/model and delay the convergence speed. All these client-related challenges can degrade the performance of the FL paradigm.
Figure 7.
Technical challenges of the FL paradigm from the perspective of COVID-19.
Figure 7.
Technical challenges of the FL paradigm from the perspective of COVID-19.
Servers-related challenges: In the FL paradigm, the server is responsible for the orchestration of the local models, aggregating models, and sharing the global model. However, in some cases, multiple attacks can be executed on the server by adversaries, which makes the FL system untrustworthy. Since the server is only concerned with the model weights without deep inspection, it cannot filter malicious clients, which degrades the performance of the FL paradigm. In some cases, gradients/parameters are exposed to adversaries during aggregation. All these server-related challenges can degrade the performance of the FL paradigm.
Training-data-related challenges: In the FL paradigm, training data are the most important element because the quality of FL models depends on the training data. There exist multiple challenges with regard to the quality of data. In addition, the privacy of the training data is one of the hot challenges in the FL paradigm [
128]. Recently, the non-i.i.d. nature of the training data poses various technical challenges in the FL paradigm, and their solution has become more urgent than ever. In addition, guaranteeing the quality of data and preventing it from poisoning the paradigm is also one of the main challenges [
129]. To truly benefit from the potential of FL, training-data-related challenges need robust solutions.
Poisoning-attacks-related challenges: In the FL paradigm, two main challenges that make the FL system unreliable in terms of results are: data poisoning and model poisoning. In the former attack, wrong data are used in training the local mode. In the latter attack, wrong local models are being sent to the central server [
130,
131]. Both these attacks have been investigated to enhance the trustworthiness of FL results. Furthermore, many strategies, even such as compromising privacy, have been suggested to eliminate these attacks [
132]. To truly benefit from the potential of FL, both these challenges need a robust solution from the research community.
COVID-19-related challenges: In the COVID-19 era, due to the rapid rise in the amount of data, processing large and high-dimensional datasets has become challenging [
133]. Furthermore, due to privacy concerns, good-quality data cannot be obtained easily. In these circumstances, FL can contribute toward resolving the data winter problem. However, the lack of a well-defined method for deploying FL methods in real life hinders the progress of AI-related methods. In addition, privacy issues such as data reconstruction make the deployment of FL very hard. Furthermore, identifying clients that can contribute good data in the FL paradigm remains challenging. Lastly, processing heterogeneous sources of data and deriving knowledge if it is very challenging. Furthermore, studying all dynamics of COVID-19 is still challenging because good-quality data for some aspects of this pandemic are not available for research purposes.
Apart from the challenges discussed above, handling inference and training time vulnerabilities in the FL paradigm is also very challenging. Luo et al. [
134] discussed the possibility of inference attacks on FL systems through which potential privacy leakages can occur in real-life scenarios. Through this approach, the authors highlighted the need to preserve the privacy of prediction outputs in the vertical FL. Qiu et al. [
135] highlighted the possibility of relation leakage and node leakage, leading to severe privacy breaches from graph data in vertical FL. Ha et al. [
136] highlighted the possibility of inference attacks on the client side in FL systems using the generative adversarial networks (GANs) model. The authors have shown that some DL models can learn “unintended” features that can expose personal information to adversarial participants/clients. Rassouli et al. [
137] have shown that in FL systems, an adversary can perfectly reconstruct a substantial number of features when the number of predictions is large enough. These kinds of data reconstruction attacks enable full training data disclosure in most cases. Zhang et al. [
138] proposed a GAN-enhanced method for launching a membership inference attack in FL systems. The authors achieved a 98% attack accuracy and identified two main reasons (i.e., diversity in training data and overfitted FL models) for the success of such attacks. To address these inference attacks, many defense strategies have also been developed [
139,
140,
141]. Further information about inference attacks and their corresponding defense can be learned from a recent study [
142]. Recently, security and privacy issues have been rigorously investigated by many researchers [
143]. In the future, more defense mechanisms will be needed to provide a solid defense against many emerging inference attacks (e.g., feature detection, extraction, feature disclosure, label disclosure, data reconstruction, membership inference, unintended features, feature information, etc.). Recently, addressing statistical heterogeneity in training data across clients/devices has also become one of the hot challenges in the FL paradigm [
144]. Concept drift makes the FL learning process more complicated because of the higher inconsistency between existing and upcoming data. Traditional concept drifts handling techniques (e.g., chunk-based and ensemble-learning-based) are unsuitable in the FL frameworks due to the heterogeneity of local devices [
145]. Similarly, handling some data types such as genome data in the FL environments poses various challenges [
146]. Considering these challenges, robust solutions are needed to address all of the above-mentioned challenges. In
Table 5, we propose technical recommendations to address these challenges by analyzing the existing open-source developments, as well as the detailed synthesis of published literature. The detailed guidelines presented in
Table 5 can contribute to enhancing the technical effectiveness of this recent paradigm.
7. Comparisons and Discussion
In this section, we compare our work with the existing state-of-the-art (SOTA) studies in multiple aspects. Although many studies have highlighted the potential of the FL paradigm in the medical field, only a few studies have focused on the applications of the FL paradigm in the COVID-19 era. To compare our work, we selected seven SOTA and recently published studies centering on the FL paradigm in the medical field. We have chosen various parameters for fair comparisons to prove the significance of our work in the body of knowledge.
Table 6 presents the in-depth analysis and comparison of our work with the existing SOTA studies.
As shown in
Table 6, our work has covered many more aspects of FL with regard to COVID-19 than the previous SOTA studies. In addition, this is the first work that has comprehensively covered FL’s role in the recent pandemic. The contents enclosed in this article can pave the way for understanding this leading technological role in the medical field, especially related to COVID-19. In addition, our work is the first that highlights the open-source developments of FL, which can assist in understanding the development status of this paradigm. In recent years, FL has been fused with multiple technologies (i.e., the industrial internet of things (IIoT), blockchain, edge computing, etc.) to address the privacy and security issues in real-life domains [
147]. Although FL can solve many cybersecurity-related issues, the FL paradigm is prone to multiple attacks due to its decentralized architecture. Therefore, more approaches are needed to address cybersecurity-related issues in FL-based systems.
The major contributions of this work compared to previous studies are: (i) higher coverage of FL applications in terms of numbers (i.e., 36) in the era of COVID-19; (ii) through discussion of challenges faced by FL paradigm which are either ignored or barely discussed by previous studies; (iii) systematic discussion of data types which were used to lower the spread of COVID-19; (iv) highlighted the open-source frameworks that have recently been developed along with their in-depth details; (v) a discussion and analysis of open source frameworks that were being developed specifically for the medical domain; (vi) it is the first study to provide recommendations to address the technical deficiencies of the FL paradigm; (vii) it is the first study that pinpoints and discusses the synergies of FL with other emerging technologies; (viii) the systematic coverage of issues that can emerge in FL deployment; (ix) a discussion about other COVID-19-fighting digital technologies; and (x) a detailed discussion of hot research area(s) targeting the FL paradigm. Furthermore, our study has covered many FL applications in the COVID-19 era that remained unexplored in previous works. Furthermore, this is the first study that discussed FL applications along with AI models and data details. This work can pave the way to providing the recent status of FL developments in the COVID-19 era.