Review of Federated Learning and Machine Learning-Based Methods for Medical Image Analysis

Abstract

Federated learning is an emerging technology that enables the decentralised training of machine learning-based methods for medical image analysis across multiple sites while ensuring privacy. This review paper thoroughly examines federated learning research applied to medical image analysis, outlining technical contributions. We followed the guidelines of Okali and Schabram, a review methodology, to produce a comprehensive summary and discussion of the literature in information systems. Searches were conducted at leading indexing platforms: PubMed, IEEE Xplore, Scopus, ACM, and Web of Science. We found a total of 433 papers and selected 118 of them for further examination. The findings highlighted research on applying federated learning to neural network methods in cardiology, dermatology, gastroenterology, neurology, oncology, respiratory medicine, and urology. The main challenges reported were the ability of machine learning models to adapt effectively to real-world datasets and privacy preservation. We outlined two strategies to address these challenges: non-independent and identically distributed data and privacy-enhancing methods. This review paper offers a reference overview for those already working in the field and an introduction to those new to the topic.

1. Introduction

Federated learning has emerged as a technology to enhance collaboration, sparking research interest in distributed techniques consisting of training models using computing resources spread across a network. To set the context of this review paper, we explain that distributed techniques involve dividing image datasets and machine learning-based image analysis models among different sites (clients and servers) to facilitate parallel processing and accelerate model training. Examples of these techniques include distributed learning and federated learning. In distributed learning [1], a centralised dataset partitions across multiple clients where the machine learning models train locally; the results are then combined and consolidated into a server model. In contrast, federated learning [2] keeps datasets within the clients, continuously training their models and periodically sharing model updates with a server. The server aggregates these updates to improve its model, which is then sent back to the clients. It enables all participating clients to benefit from collective knowledge without sharing their datasets, making federated learning a preferred choice for privacy-conscious applications (see Figure 1).

Examples of applications have been research in medical specialities including cardiology [3], dermatology [4,5,6,7,8,9,10,11,12,13,14,15,16], gastroenterology [17], neurology [18,19,20,21,22], oncology [23,24,25], respiratory medicine [26,27,28,29,30,31,32,33,34,35,36,37,38,39], and urology [40] (see

Table A2. Part 1/4 of the detailed literature corpus of the reviewed papers on medical image analysis research on federated learning.

Paper	Medical Data Speciality	Referenced Dataset	Referenced Algorithm	Research Strategy
[3]	Cardiology	M&M [93] Emidec [94]	3D U-Net [92]	Non-IID
[4]	Dermatology	HAM10000 [114]	PrivGAN [156]	Non-IID
[5]	Dermatology	ISIC [84]	DualGAN [65] KnEA [66]	Non-IID
[6]	Dermatology	ISIC [84]	EfficientNet [157]	Use Case
[7]	Dermatology	AtlasDerm [158] Dermnet [159]	VGG AlexNet FedAvg [160] FedML	Use Case
[8]	Dermatology	FMNIST [131]	Efficient-Net FedPerl	Non-IID
[9]	Dermatology	RSNA ICH [124] ISIC [84]	DenseNet [161] Client Matching	Non-IID
[10]	Dermatology	Proprietary Data	CNN	Privacy
[11]	Dermatology	TCGA [130]	DP-SGD [46]	Privacy
[14]	Dermatology	HAM10000 [114]	MobileNet [113]	Non-IID
[15]	Dermatology	TCGA [130]	DenseNet [161] MIL [162]	Privacy
[13]	Dermatology Neurology	RSNA ICH [124] HAM10000 [114]	FedAvg [160]	Non-IID
[12]	Dermatology Oncology Respiratory Medicine	MedMNIST [70] Camelyon17 [71]	ResNet	Non-IID
[16]	Dermatology Respiratory Medicine	Pcam [141] COVIDx [132]	MAE [163]	Privacy
[107]	Dermatology	TCGA [130] CRC-VAL-HE-7K [164] NCT-CRC-HE-100K [164]	CycleGAN	Non-IID
[165]	Dermatology	SkinLessions [166] Monkeypox [167]	MobileNet ResNet CycleGAN ViT [168]	Use Case
[169]	Dermatology	Proprietary data	ResNet	Use Case
[170]	Dermatology	ISIC [84]	ResNet	Use Case
[171]	Dermatology	ISIC [84]	CNN	Use Case
[172]	Dermatology	HAM10000 [114]	CNN	Use Case
[106]	Dermatology Miscellaneous (Anatomy Detection)	MNIST [173] HAM10000 [114] MedMNIST [70]	CNN	Non-IID
[103]	Dermatology Oncology Respiratory Medicine	MedMNIST [70] MNIST [173]	ResNet	Non-IID
[122]	Dermatology Oncology	CoNSeP [174] TCGA [130] GlaS [175] CryoNuSeg [176] Kumar [177] TNBC [178]	U-Net	Non-IID

and

Table A3. Part 2/4 of the detailed literature corpus of the reviewed papers on medical image analysis research on federated learning.

Paper	Medical Data Speciality	Referenced Dataset	Referenced Algorithm	Research Strategy
[17]	Gastroenterology	GLRC [97]	SCM [95] FCOS [96]	Non-IID
[41]	Miscellaneous (Anatomy Detection)	TCGA [130]	MobileNet [113]	Use Case
[123]	Miscellaneous (Disease Classification)	MedMNIST [70]	CNN FedAvg [160]	Non-IID
[43]	Miscellaneous (MRI Reconstruction)	fastMRI [115] HPKS [116] IXI [79] BraTS [80]	U-Net FedAvg [160]	Non-IID
[23]	Miscellaneous (Thyroid Cancer)	Proprietary Data	VGG ResNet	Use Case
[135]	Miscellaneous (Anatomy Detection)	ACDC [179]	U-Net	Privacy
[155]	Miscellaneous (Anatomy Detection)	Proprietary data	VGG	Use Case
[137]	Miscellaneous (Anatomy Detection)	MedMNIST [70] COVID-CT-dataset [180] PneumoniaMNIST [181]	ResNet	Privacy
[182]	Miscellaneous (Anatomy Detection)	Montgomery [183] India [184] Shenzhen [183] TBX11k [185] TB-Att [186]	ConvNeXt [187]	Use Case
[188]	Miscellaneous (Anatomy Detection)	X-RayKnee [189]	DenseNet	Use Case
[45]	Miscellaneous (Watermark Extraction)	Proprietary Data	Encoder–Decoders	Privacy
[18]	Neurology	ADNI [111] PPMI [190] MIRIAD [191] UK BioBank [192]	ENIGMA [193]	Non-IID
[19]	Neurology	ADNI [111] OASIS [82]	FedCM VGG 3D-CNN [110]	Non-IID
[20]	Neurology	BraTS [80]	FedAvg [160] Encoder–Decoders	Privacy
[21]	Neurology	BraTS [80]	U-Net	Use Case
[22]	Neurology	IXI [79] BraTS [80] MIDAS [81] OASIS [82]	PatchGAN [74]	Non-IID
[194]	Neurology	OASIS [82]	CNN	Use Case
[117]	Neurology	ADNI [111] AIBL [195] AI4AD [196]	ViT [197]	Non-IID
[85]	Neurology	ABIDE [198] ADNI [199]	Graph CNN [200]	Non-IID
[201]	Neurology	LUNA [202] Proprietary data	VGG	Use Case
[203]	Neurology	SARTAJ [204] Br35H [205]	VGG	Use Case
[206]	Neurology	Proprietary data	AlexNet	Use Case
[207]	Neurology	SARTAJ [204] Br35H [205]	DenseNet	Use Case
[121]	Neurology Miscellaneous (Anatomy Detection)	TCIA [208] Proprietary Data	Mean Teachers [209]	Non-IID
[210]	Neurology Respiratory Medicine	COVIDCT [211] COVID-CT-dataset [180] SARS-CoV-2 [212]	CapsuleNetwork [213]	Use Case

). The motivation for embracing federated learning in the medical field stems from two factors. Firstly, the significant cost associated with acquiring datasets serves as a driving force for increased collaborations. In certain cases, obtaining datasets, such as magnetic resonance imaging and computed tomography scans, necessitates substantial investments in specialised equipment and skilled personnel. With federated learning, clients can pool their resources and expertise, sharing the burden of acquisition costs while benefiting from a more diverse and comprehensive dataset. Secondly, the low prevalence of certain diseases also plays a role. Some medical conditions occur relatively infrequently, making it challenging to gather a sufficiently large dataset from a single client or geographic location. Federated learning enables healthcare providers and researchers to overcome this limitation.

Research in federated learning demonstrates the benefits of its implementation in several use cases [6,7,21,23,27,28,30,32,33,34,35,36,41]. Of the challenges documented, we found that the issues of addressing real-world datasets and privacy preservation appeared frequently. To set the context for this review paper, we define real-world datasets as those where medical image datasets do not randomly sample from a homogeneous population; instead, they sample from distributions that may be dissimilar. In other words, the images are not distributed identically and are not independent of each other. In real-world federated learning scenarios, we expect non-independent and identically distributed data (non-IID) with datasets located across multiple clients, and each client may have a particular dataset distribution. Privacy preservation refers to protecting sensitive or personal information from unauthorised access, disclosure, or misuse.

To address these challenges, researchers investigate two main strategies, outlined in this review paper as (1) non-IID methods [3,4,5,12,14,17,18,19,22,29,37,40,42,43], consisting of data augmentation [44] applied in scenarios of heterogeneity or imbalance datasets, and semi-supervised learning [9], which is well-suited for scenarios where labelled datasets are limited or unavailable; (2) privacy-enhancing methods [10,11,15,16,20,26,38,39,45,46]; these include differential privacy [11], which are methods that add random noise to datasets to prevent tracing back to specific users, homomorphic encryption [47] that allows computations on encrypted datasets, and differential privacy [15] that prevents learning unauthorised datasets.

Several researchers have recently published related reviews and survey papers. For example, Li et al. [48] and Yang [49] reviewed privacy-preserving computing based on homomorphic encryption, secure multi-party computing, and differential privacy. Yang et al. [50] published a federated learning survey describing methods in terms of dataset partitioning and architectures. Yin et al. [51] analysed privacy from the perspective of external attacks. Some focused on tabular datasets [52]. A few reviews and surveys limited their literature analysis to the application of federated learning in the medical field [53,54,55], lacking technical insight; others, due to the broad scope of the review and survey papers, dedicated less than 900 words to discuss machine learning-based methods for medical image analysis [56,57,58,59,60], which is the main focus of our review. Contrary to the above, our review paper emphasises the technical contributions of the reviewed literature. Our review aims to answer the research question “What machine learning-based methods research the analysis of medical images in federated learning?”. It adds to the existing literature by providing the following elements:

• A comprehensive review highlighting the shortcomings of current federated literature applied to machine learning-based medical image analysis.
• A taxonomy of federated learning papers on machine learning-based medical image analysis, including the medical applications, referenced datasets, and methods utilised.
• A summary of open-source frameworks for developing federated learning.

2. Methodology

This review paper follows Okali and Schabram’s methodology [61]. The selected papers focus on original machine learning-based medical image analysis contributions published in journals and conference proceedings written in English. There was no restriction on the publication year of the retrieved papers.

We used the following keywords: federated learning, image, medicine, healthcare, disease, well-being, machine learning, artificial intelligence, and expert systems. We retrieved the literature from four databases: IEEE Xplore, Scopus, ACM, and Web of Science. The last search update took place in July 2024.

As shown in Figure 2, a total of 433 papers were retrieved. We excluded 31 papers due to duplication across the databases. Following this, we excluded 132 papers because their titles and abstracts did not suggest the use of medical image analysis, machine learning algorithms, or federated learning research strategy. We read the remaining papers in full, from which we excluded 155 papers for not meeting five quality criteria: (1) the research objective of the study is clear; (2) the study focused on human medical images; (3) the use of machine learning algorithms and federated learning techniques is clear; (4) the study includes sufficient details in the methodology, experiment, and results; and (5) the study adds technical value to the existing literature or showcases the applicability of federated learning in the medical field. Finally, we added 3 papers as grey literature, i.e., papers unintentionally excluded (in previous steps) and found via other methods such as citation search and expert recommendations. The net result was 118 papers (52 journals, 51 conference proceedings, and 15 reviews/surveys) considered relevant and included in the review paper.

We then extracted relevant information from journals and conference proceedings in five categories to make it more accessible for examination and interpretation: application, dataset, referenced algorithm, key topic, and contribution. The application category denotes the medical specialities, including dermatology, neurology, and respiratory medicine. The dataset category pertains to the datasets employed. The referenced algorithm category denotes the particular algorithm or method that formed the basis for the technical implementations. The topic category organises the papers into four groups based on the paper’s claims: use case, for papers showcasing the benefits of federated learning in scenarios of distributed datasets; non-IID, for papers with a technical contribution to solving the problems pertaining to heterogeneity and imbalanced datasets; privacy, for papers with a technical contribution to addressing the challenges in data privacy preservation; and the research category, focused on the contribution of the reviewed papers. We grouped 49 of the papers in the use case topic category, 40 in the non-IID topic category, and 14 papers in the privacy topic category.

During the examination, we grouped the papers that introduced original methods to discuss them in detail. We assessed originality based on our expertise as authors of this review paper, considering papers that demonstrated new methods to address the challenges or significant improvements over existing approaches concerning medical image analysis. As a result, we selected 19 papers for detailed discussion in Section 3.

We also noted that several selected papers used open-source frameworks as tools. Hence, in

Table A1. Detailed list of open-source federated learning frameworks.

Name	Built-In Support	Aggregator	Security
FATE (1.8.0) [142]	PyTorch TensorFLow	FedAvg SecAgg SecMPC SecBoost	Public-key Cryptosystems
FedML (0.6.0) [143]	PyTorch	FedAvg FedOpt FedProx FedNova SplitNN Hierarchical FL	Differential Privacy Multi-party Computation
Flower (1.0.0) [144]	PyTorch TensorFlow JAX Hugging Face Scikit-learn MXNet PyTorch-Lightning TFLite	FedAvg FedAvgM QFedAvg FaultTolerantAvg FedOpt FedAdagrad FedAdam FedYogi	Differential Privacy
NVFlare (2.1.3) [145]	PyTorch TensorFLow	FedAvg FedOpt FedProx	Homomorphic Encryption Differential Privacy
OpenFL (1.3.0) [146]	PyTorch TensorFLow	FedAvg FedProx FedOpt FedCurv FedYogi FedAdam FedAdagrad	Mutual Transport Layer Security Secret-sharing Differential Privacy
PaddleFL + PaddlePaddle (1.2.0) [147]	PyTorch	FedAvg SecAgg	Public-key Cryptosystems Differentially Private Stochastic
PySyft + PyGrid (0.6.0) [148]	PyTorch TensorFLow	FedSGD	Differential Privacy Multi-Party Computation Homomorphic Encryption Public-key Cryptosystems
TensorFlow Federated (0.31.0) [149]	TensorFlow	FedAvg FedSGD FedProx FedOpt	Differential Privacy
PriMIA [46]	PyTorch	FedAvg SecAgg	Secure Aggregation Differential Privacy Multi-party Computation

, we provide a list of the relevant open-source frameworks to help readers quickly identify and adopt tools ready for use, offering practical utility and implementation guidance. Furthermore, by promoting open-source frameworks, we trust that the review paper will foster collaboration and innovation, encourage collective improvement, and advance the field.

This review paper is structured as follows. In Section 3, we report the newly proposed methods found in the selected papers, including a briefing of their method, information about the datasets used during evaluation, results, and a critique of their advantages and disadvantages. The summary of open-source frameworks is available in Section 4, followed by discussion and final remarks in Section 5 and Section 6, respectively.

3. Strategies in Federated Learning for Machine Learning-Based Image Analysis

The findings highlight challenges, including working with real-world datasets and preserving privacy. Two strategies are (1) non-IID methods, including data augmentation, semi-supervised learning, data distribution adjustment, and parameter adaptation, and (2) privacy-enhancing methods, including differential privacy, model aggregation, and homomorphic encryption.

3.1. Non-Independent and Identically Distributed Data Methods

In a medical setting, the most common sources of non-IID data are caused by confounding factors, referring to variables that can affect the input datasets, including differences in image acquisition, image quality, and variation in image appearance. In the context of confounding factors, non-IID refers to situations where the images are not independent and identically distributed. Confounding factors can lead to dependencies between images, resulting in non-identical distribution across different datasets; this can be problematic in federated learning, as models trained on non-IID data may not generalise well to new datasets, leading to poor performance [62]. Research suggests four strategies to address this issue: data augmentation, data distribution, parameter adaptation, and semi-supervised methods (Figure 3).

3.1.1. Data Augmentation

Data augmentation is a technique used to artificially increase the training dataset size. Traditional methods include minor image modifications such as rotation, scaling, flipping, and applying various filters [63]. While these techniques help preserve the original data distribution, they do not necessarily enhance model generalisation, as they may overlook differences in dataset distributions. In contrast, newer methods like generative adversarial networks (GANs) can generate new data that maintain the original distribution, thereby improving model performance across diverse datasets [12,40,64]. The reviewed literature explores two such methods: conditional GANs [22] and dual GANs [65], as well as evolutionary algorithms [5,66,67].

GAN methods have included techniques to address non-IID data in different ways. For instance, regularisation methods like virtual adversarial training (VAT) add small amounts of Gaussian noise to input images, which are then added to the training dataset to improve the model’s ability to classify unseen images [68]. Zhu and Luo [12] proposed federated learning with virtual sample synthesis (FedVSS). FedVSS uses ResNet-18 [69] as a backbone network and applies VAT to the clients’ models, enhancing the generalisation ability of the server’s model. It generates synthetic training datasets and aligns the clients’ models with the server’s model by synthesising high-confidence samples from the server model’s dataset distribution. Both synthesised and original datasets update the client’s model, enabling FedVSS to achieve more generalised and consistent performance. Challenges may lie in the complexity of synthesising high-quality images due to the computational overhead associated with aligning clients’ models to the server’s model, requiring substantial resources and coordination among multiple clients. FedVSS evaluated its performance on the MedMNIST [70] and Camelyon17 [71] datasets, achieving an F1-score of $81.27$ and an accuracy of $75.32$ in the method’s effectiveness in synthesising images and aligning clients’ models to the server’s model.

Another type of GAN is the conditional generative adversarial network (cGAN), characterised by providing high-frequency textural information relevant to medical images. cGAN performs adversarial learning via a pair of networks: a generator and a discriminator. The generator predicts a synthetic target-contrast image given an acquired source-contrast image as input, while the discriminator tries to distinguish between actual and synthetic target-contrast images. To learn image translation, cGAN trains to minimise a loss function composed of adversarial and pixel-wise terms [72,73].

To address some of the challenges in federated learning, Dalmaz et al. [22] proposed specificity-preserving federated learning (SPFL-Trans) based on a cGAN. “Specificity” refers to information or characteristics specific to a particular client’s dataset, such as computational resources, quality and size of datasets, and disease prevalence. SPFL-Trans, informed by PatchGAN [74], consists of an adversarial model that adaptively normalises the feature maps across the generator based on the client’s dataset-specific latent variables (variables that are not directly observed but inferred from the model). SPFL-Trans consists of nine residual blocks and a latent parameter space with six dense layers to produce latent variables. SPFL-Trans takes an image and the client-specific latent variables as input to generate scale and bias vectors, two learnable parameters in the normalisation layer that adjust the mean and standard deviation. The outcome is then modulated to the first- and second-order statistic measures of distribution [75].

SPFL-Trans shows competitive performance compared to a centralised baseline model while outperforming competing methods (FedGAN [76], FedMRI [77], and FedMedGAN [78]) both visually and quantitatively. However, challenges may arise when the training data are insufficiently large or diverse. In such cases, the latent space might not effectively capture the full variability of the data. This challenge stems from the complexity introduced by using latent parameter spaces with dense layers in combination with residual blocks. SPFL-Trans evaluated its performance on the IXI [79], BraTS [80], MIDAS [81], and OASIS [82] datasets. The experiments achieved an average of $25.7$ dB for peak signal-to-noise ratio, $88.6\%$ for structural similarity index, and $20.1$ points for Fretchet inception distance.

Findings also included the adoption of evolutionary algorithms and dual generative adversarial networks (DualGANs) [65]. Evolutionary algorithms use iterative processes to simulate biological mechanisms to find the optimal solution to a problem. The basic idea is to generate a set of candidate solutions and then use it to produce new candidate solutions. This process continues until either finding a satisfactory solution or meeting a predetermined stopping criterion. An example is the knee point-driven evolutionary algorithm (KnEA), which aims to identify the “knee point” of a trade-off curve, representing the optimal balance between different objectives [83].

To put DualGAN in context, traditionally, the generator learns to generate synthetic images from a random initialisation in a GAN architecture. In contrast, the discriminator learns to distinguish between real and synthetic images. In DualGAN, the generator translates images from one domain space to another. The discriminator then evaluates the translated images and provides feedback to the generator, helping it generate more realistic translations.

Cai et al. [5] proposed the skin cancer detection model based on federated learning integrated with DualGANs (FDSCDM). This framework integrates KnEA and DualGAN to address the problem of insufficient datasets. To enhance the number of images generated through DualGAN, FDSCDM synchronously optimises four metrics using KnEA: the sharpness of images (the degree of clarity and detail in an image), Frechet inception distance, image diversity, and loss. Results suggest that using evolutionary algorithms with DualGAN can help improve performance and efficiency, reduce the need for manual tuning, increase scalability, and enhance the diversity of generated images by automatically exploring the parameter space. However, generating offspring is yet to be further researched, as performing non-dominated sorting and environmental selection involves significant computational resources. Additionally, as the number of objectives and the size of the population increase, the scalability of evolutionary algorithms may become impractical for very large-scale problems or real-time applications where rapid solutions are needed. FDSCDM evaluated its performance on the ISIC [84] dataset, achieving an accuracy of $91\%$ and an area under the curve of $88\%$ for a seven-class classification task.

The reviewed literature also revealed a small number of papers suggesting sharing synthetic datasets as a strategy to address the problems of non-IID data [4,37,40,85]. Although sharing synthetic datasets may not fulfil the definition of federated learning adopted in this review, such a strategy can be beneficial in several ways. First, it can improve the performance of the server model by providing additional training datasets that are representative of the underlying distribution. Second, it can preserve privacy by reducing the need for clients to share their real datasets, which may contain sensitive information. Third, it can enable clients to collaborate more effectively by providing common datasets that they can use to train their models. However, it is important to note that synthetic datasets may not always be a perfect substitute for real data [86], as their quality may depend on the quality of the model used to generate them, and leakage of identifiable information and biases may arise [87].

3.1.2. Dataset Distribution and Client Selection

Selecting clients for collaboration is particularly relevant in medical imaging, where we expect dataset imbalance and heterogeneity across clients. Findings suggest two alternatives: adopting distillation [3,88] and performance deterioration recognition methods [17,89,90,91].

Distillation methods involve teaching a smaller and simpler machine learning model (client’s model) to learn from a larger and more complex model (server’s model) by mimicking its behaviour [88]. Qi et al. [3] proposed the cross-centre cross-sequence medical image segmentation FL framework (FedCRLD). This framework uses 3D U-Net [92] as the basis for the encoder and decoder and comprises two main components: contrastive re-location (CRL) and momentum distillation (MD). The aim is to correct representation bias and continually optimise the client’s model. CRL helps transfer only locally correlated representations from the server model. At the same time, MD builds self-training by distilling the client model’s history momentum version as additional optimisation guidance on a dynamically updated momentum bank. The momentum bank is a method used to accelerate convergence during the training process. It stores a moving average of the gradients of the neural network parameters used to update them during the optimisation process [25].

The CRL module corrects representation bias using a contrastive difference metric of mutual information, improving representation for heterogeneous datasets. However, the MD component requires maintaining a momentum bank and performing additional computations to update and distil historical momentums, which may add significant computational overhead compared to traditional federated learning methods. FedCRLD evaluated its performance on the M&M [93] and Emidec [94] datasets, achieving an average Dice score of $85.96\%$ for a segmentation task on cardiac magnetic resonance images.

Performance deterioration recognition detects and corrects errors in machine learning models before they cause major problems; this requires monitoring the model’s performance over time and detecting any decline in accuracy or other metrics that indicate a decrease in performance. Using noise datasets involves intentionally adding random variations to the datasets to test the robustness of the machine learning model [17]. Liu et al. [17] proposed the intervention and interaction FL framework (FedInI). FedInI adopts a structural causal model (SCM) [95] and a fully convolutional one-stage object detector (FCOS) [96] to address dataset selection across clients by identifying noisy datasets that lead to performance deterioration. SCM is employed for feature extraction and representation, leveraging its capacity to handle sparse datasets efficiently. Meanwhile, FCOS is utilised for object detection tasks within the framework, providing a robust and efficient means to localise and identify anomalies directly in the signals without requiring anchor boxes, thus simplifying the detection process and improving accuracy. FedInI enhances the training of the server model by shuffling and mixing features extracted from different client models to suppress noise gradually. They propose an interaction strategy to tackle the challenge of the server model being unaware of local training. This strategy considers training synchronisation and the noise heterogeneity between datasets and adaptively generates manifold mixup weights. Performance deterioration recognition is crucial for detecting and correcting errors before they become severe. However, this can be challenging due to the distributed and heterogeneous nature of the datasets and models involved. These methods require further examination to develop specialised techniques to address them effectively. This method evaluated its performance on the GLRC [97] dataset, achieving an average mAP of $89.91\%$ and an IOU of $75\%$ for an object detection task.

3.1.3. Parameter Adaptation

Parameter adaptation is a method that involves adjusting the parameters of a model to improve its performance on a specific task. This method is particularly useful in dynamic environments where data tend to be non-IID. It typically involves monitoring the model’s real-time performance and adjusting its parameters accordingly to enhance machine learning models’ accuracy, efficiency, and robustness [98]. Findings suggest various approaches, including graph neural networks (GNNs) [42,99], model distillation [19,88,100,101], adaptive clustering [14,102,103,104,105,106], using learned intermediate latent features [43], and multivariate analysis [18,107].

GNNs are a type of machine learning model designed to represent and analyse the relationships between datasets. They use this structure to identify patterns, making them well-suited for interconnected datasets [99]. Chakravarty et al. [42] trained a server model in conjunction with a GNN [108] on clients to capture specific variations in dataset distributions. While the server model weights are learned and shared across clients, a separate GNN is constructed and fine-tuned for each client to leverage the dataset’s client-specific prevalence and comorbidity statistics, which refer to the frequency and likelihood of medical conditions. Results demonstrated the effectiveness of GNNs; however, further research has yet to prove how this method addresses imbalanced class distributions. This approach evaluated its performance on the CheXpert [109] dataset, achieving an average AUC of $0.79$ for a 14-class disease classification task.

Model distillation in federated learning can be conceptualised as a data-private collaborative method where participating models leverage the available data by distilling knowledge through the average prediction scores. Huang et al. [19] proposed the federated conditional mutual learning (FedCM) framework, which aims to personalise models to client-specific datasets through distillation. FedCM uses VGG and 3D-CNN [110] as backbone networks. FedCM allows a subset of each client’s datasets, referred to as public datasets, to be transmitted across the network of clients in a federated setting; this enables the server model to benefit from the collective knowledge of the other datasets.

FedCM incorporates a mutual knowledge distillation framework and a condition monitoring mechanism that assesses performance and probability distribution similarity. The workflow of the FedCM framework involves three main steps: First, each client periodically uploads its predicted results and cross-entropy (CE) loss, calculated based on its private dataset, to the server model. The CE loss provides valuable information for refining the server model. Second, the server model aggregates all clients’ parameters and CE losses, excluding the one receiving the update. This step ensures that the receiving client benefits from the knowledge contributed by the others. Finally, each client uses the received server model parameters to fine-tune its model with its private data, adapting the model to the specific features of its dataset. Although FedCM addresses the challenge of heterogeneous datasets by utilising public datasets for model training, ensuring privacy while sharing distillation outputs poses challenges that require further research. Careful handling is needed to prevent data leakage. FedCM evaluated its performance on the ADNI [111] and OASIS [82] datasets, achieving accuracy rates of $74.5\%$, $76.0\%$, and $76.0\%$ for a three-class classification task.

Adaptive hierarchical clustering is a method that organises datasets into a hierarchy of clusters based on their similarity, with clustering dynamically adjusting to dataset characteristics [14]. Research has also explored the benefits of combining this clustering method with meta-learning (learning how to learn rather than just learning a specific task), which involves adapting and generalising to clustered datasets. This approach includes learning higher-level abstractions and strategies that can be applied across tasks [112]. For example, Yeganeh et al. [14] proposed federated adaptive personalisation (FedAP).

This adaptive hierarchical clustering method produces intermediate semi-federated models by forming clusters of datasets using meta-learning. The FedAP framework uses MobileNet [113] as its backbone network and introduces an adaptive personalisation mechanism that leverages the information contained within clients. This mechanism allows the server to selectively incorporate knowledge from specific models most relevant to a dataset.

The adaptive personalisation process identifies the most relevant models for each dataset based on their data characteristics. During the dataset selection step, the meta-model evaluates the relevance of each model to a specific data distribution. By leveraging learned meta-knowledge, FedAP can determine which models will likely provide the most valuable insights. This selective incorporation personalises the server’s model better to suit the characteristics of the client’s dataset. Despite its significant performance, training FedAP for too many rounds can lead to decreased performance, indicating sensitivity to overfitting; addressing this issue is a key area for future research. FedAP evaluated its performance on the HAM10000 [114] dataset, achieving an accuracy of $86.9\%$ for a seven-class classification task.

Learned intermediate latent features refer to representations acquired by deep neural networks at layers that are neither the input nor the output layers. These latent features capture high-level information about the input dataset [43]. Guo et al. [43] introduced the federated learning-based magnetic resonance reconstruction with cross-client modelling (FL-MRCM), which employs a U-Net style encoder–decoder architecture for reconstruction networks. FL-MRCM aligns the learned intermediate latent features from datasets with the distribution of these features.

The key components of FL-MRCM involve leveraging the encoder part of the reconstruction networks to project the input dataset onto the latent space of the server’s model. FL-MRCM incorporates an adversarial domain identifier for each client–server pair to align the latent space distribution with models trained in an adversarial manner. The FL-MRCM process includes two optimisation steps. The first step trains the client reconstruction networks on their respective datasets. The second step aligns the latent space distributions between the client and server domains. FL-MRCM’s generalisability and computational overhead in scenarios involving many clients have yet to be fully explored. FL-MRCM evaluated its performance on the fastMRI [115], HPKS [116], IXI [79], and BraTS [80] datasets, achieving a structural similarity index measure of $92.32\%$ and a peak signal-to-noise ratio of $32.44$ dB.

3.1.4. Semi-Supervised Learning

Semi-supervised learning is an approach that combines a small amount of labelled data with a large amount of unlabelled data during training. In federated semi-supervised learning (FSSL), most datasets are unlabelled [24]. A straightforward solution might be to apply centralised semi-supervised methods to federated learning. However, traditional semi-supervised learning (SSL) methods typically assume a centralised setting where labelled data are readily accessible to assist in learning from unlabelled data.

For instance, in consistency-based methods, regularising perturbation-invariant model predictions require synchronous supervision from labelled datasets to provide the necessary task knowledge for reliable predictions on unlabelled data. In FSSL, where the clients’ datasets might be entirely unlabelled, this close supervision from labelled data is absent, causing the client’s model to lose critical task information during consistency-based training and failing to leverage knowledge from unlabelled datasets. Thus, the main challenge in FSSL compared to traditional SSL is effectively building interaction between learning from labelled and unlabelled datasets. The reviewed literature explores solutions such as dynamic banks [13], consistency regularisation [8,31,117,118,119,120,121,122], and distillation [88,123] (see Figure 4).

A dynamic bank refers to a mechanism used to store and update the momentum values of each client model during the training process of a federated learning model. The momentum values represent the direction and magnitude of the gradient descent updates of the client model parameters at each client. The dynamic bank is updated periodically by aggregating the momentum values from the clients participating in the training process. This mechanism aims to benefit from the knowledge accumulated by other clients, thus improving the performance and convergence speed of the federated learning model [13].

Jiang et al. [13] proposed a method consisting of two parts. First, the dynamic bank construction extracts class proportion information within each sub-bank classification to enforce the client model to learn different class proportions. The dynamic bank iteratively collects highly confident samples during training to estimate the dataset’s class distribution and splits samples into sub-banks with different pseudo-label proportions. Second, a prior transition function transforms the original classification task into a sub-bank classification task, using different class proportions to train the client model. This label-proportion-aware supervision enhances clients’ training by learning different distributions of imbalanced classes, thus avoiding dominance by the local majority class. The effectiveness of the method is demonstrated on two large-scale real-world medical datasets. Future research should explore dynamic bank construction further by incorporating information from other datasets to address potential limitations in handling severe class imbalances. This method evaluated its performance on the RSNA ICH [124] and HAM10000 [114] datasets, achieving an average accuracy of $88.94\%$ and an F1-score of $33.79\%$ for a seven-class classification task.

Yang et al. [31] showcased the benefits of federated learning in a semi-supervised setting. They presented work of a centralised semi-supervised strategy for federated learning using pseudo-labelling and consistency regularisation. Pseudo-labelling is a self-training process that assigns synthetic labels to unlabelled data samples based on the predicted class with a softmax probability exceeding a pre-specified threshold; this is followed by training the model on the labelled and pseudo-labelled samples in a purely supervised manner. Consistency regularisation is a co-training method that enforces the condition that augmented versions of the same data sample should yield the same prediction. The challenge with this method lies in ensuring that these constraints remain effective and reliable across diverse, heterogeneous, and sometimes noisy data sources. Further work may consider handling domain shifts, variability in annotation quality, and integrating unlabelled data to complement supervised learning. This method evaluated its performance on the LIDC [125] dataset, achieving a dice score of $0.651$ for a segmentation task in 3D computed tomography scans.

Tariq Bdair et al. [8] proposed the peer learning and ensemble averaging for peer anonymisation method (FedPerl). Peer learning involves using similar peers (client and server models) to assist with pseudo-labelling. FedPerl combines the learned knowledge from different models through ensemble averaging before sharing it with other peers, thereby preserving anonymity. In summary, an anonymised peer aggregates the learned knowledge from similar peers and shares it with the client to assist in the pseudo-labelling process. This approach has an advantage over other methods [31] by allowing clients to gain additional knowledge through collaboration and leveraging unlabelled data for pseudo-labelling. FedPerl ensembles the results of multiple models, encouraging them to learn from each other. The peer anonymisation policy, which hides the client’s identities, helps avoid model inversion and de-anonymisation, thereby preserving privacy. FedPerl is simple yet effective for anonymising peers, making it less prone to model inversion or de-anonymisation. Nevertheless, researchers have not thoroughly investigated the privacy guarantees for aggregated models, leaving it an open issue. FedPerl evaluated its performance on the LIDC [125] dataset, achieving an average accuracy of $82.75\%$ for an eight-class classification task.

Saha et al. [123] proposed an isolated federated learning method (IsoFed) that aims to integrate labelled and unlabelled datasets using both federated learning and transfer learning. IsoFed first isolates the aggregation of labelled and unlabelled datasets and then performs self-supervised pretraining of the server models. Specifically, IsoFed employs a dynamically weighted averaging scheme to separately aggregate the model parameters for labelled and unlabelled datasets. After this aggregation, IsoFed conducts self-supervised pretraining on each client’s dataset by optimising an information maximisation loss. This approach ensures that the server’s model provides individually reliable predictions but is collectively diverse. Further research is encouraged to demonstrate how IsoFed would address issues such as weight divergence and domain shift (the difference between the data distribution in the training set and the data distribution in the real world), as the client’s model may forget the original task as training progresses. IsoFed evaluated its performance on the MedMNIST [70] dataset, achieving an average accuracy of $87.10\%$ for a two-class classification task.

3.2. Privacy-Enhancing Methods

Privacy-enhancing methods are employed to protect the privacy of the data used in the federated learning process. These methods enable machine learning algorithms to learn from distributed data while ensuring that sensitive information is not exposed. The strategies discussed in the reviewed literature include differential privacy, model aggregation, and homomorphic encryption, highlighting methods such as selective content-aware differential privacy, parameter aggregation, multi-party computation, and blockchain, respectively, (see Figure 5).

3.2.1. Differential Privacy

Differential privacy methods protect the privacy of sensitive data by adding random noise to a dataset while still allowing key information to be derived, thus mitigating confidentiality and privacy issues associated with medical datasets [11,45]. Differential privacy has been utilised with other methods, such as model aggregation and homomorphic encryption. Additionally, research has explored the combination of differential privacy with invertible neural networks (INNs), a type of neural network architecture that can perform both forward and inverse computations, allowing them to revert their outputs to the original inputs. The ability of INNs to perform reversible computations is due to the use of invertible functions in their architecture, such as coupling layers, which enable the objective transformation of data [38].

Tölle et al. [38] presented a method to achieve differentially private images based on INNs [126], namely content-aware differential privacy (CADP). They applied this method to images of patients diagnosed with a disease, ensuring that their pathology was not changed by conditioning the INN on the class labels. Their experiments on diverse datasets demonstrated that classifiers trained with CADP-generated data outperformed conventional approaches significantly. CADP privately alters the content of the input image to preserve as much information as possible while only modifying dimensions unrelated to identification, which is crucial for data privacy. However, the extent to which CADP can modify images while maintaining their informative value and ensuring privacy has not yet been thoroughly explored. CADP evaluated its performance on X-ray datasets [127], achieving an average accuracy of $92.94\%$ for a classification task.

3.2.2. Model Aggregation

Model aggregation consists of methods that involve the iterative process of constructing models incrementally over several iterations, where clients share selective information from their models with the server. Three methods found in the reviewed literature are selective parameter updates [20], secure multi-party computation [15], and partial networks [26].

Selective parameter updates reduce the amount of information shared between clients while maintaining high accuracy. The client usually updates all machine learning model parameters during each training iteration in federated learning. However, with selective parameter updates, only a subset of the parameters is shared. Updating only a subset reduces the information transferred between devices, leading to faster training times and strong protection against indirect data leakage [128]. Li et al. [20] researched the benefits of combining selective parameter updates with the sparse vector technique (SVT) [129], which is fundamental for achieving differential privacy. Their selective parameter-sharing method limits the information a client shares by clipping the client’s model gradients to a fixed range. The selective parameters are then submitted to a Laplacian-based function implementing SVT as a differential privacy technique. This method strikes a balance between ensuring privacy protection and maintaining model performance. While it offers robust differential privacy protection, further research is needed to evaluate its performance impact at scale. This method evaluated its performance on the BraTS dataset [80], achieving an accuracy of $85\%$ for the brain tumour segmentation task.

Secure multi-party computation (SMC) is a cryptographic method that enables multiple clients to train their models jointly as a cluster. In SMC, each client encrypts their data and sends them to a server. The server’s model parameters are then returned to the client, which can decrypt them to update its model [15]. Hosseini et al. [15] used SMC to develop a framework for cluster training with privacy protection. In their proposed framework, the clients’ models are grouped into clusters using geographical locations as a strategy. After training, each client shares its model weights with others in the same cluster. The clusters of clients sum up the received weights and send the results to the server. The server aggregates the results, retrieving the average of the models’ weights. Results showed that, compared to differential privacy, the framework achieves higher accuracy with no privacy leakage risk, albeit with more communication overhead. The experiment consisted of six clients grouped into two clusters based on their geographical locations. However, further research is needed to explore the benefits and drawbacks of adopting a more sophisticated strategy for clustering clients, such as using data domains. This method evaluated its performance on the TCGA dataset [130], achieving an F1-score of $79.84\%$ for a two-class classification task.

Using partial networks involves training smaller versions of a full model on subsets of the dataset and then aggregating the partial networks to form the full model. Yang et al. [26] proposed a federated learning framework for medical datasets using partial networks (FLOP). The partial networks are smaller versions of the entire model trained on subsets of the dataset, aggregated to form the full model and trained on the combined dataset. This approach allows for better data distribution management and class imbalance while preserving privacy. The FLOP approach also includes knowledge distillation and training of the partial networks to mimic the behaviour of the full model; this enables the partial networks to capture essential data features and contribute to training the entire model, even with limited data access. However, research has yet to ensure that the design of partial networks remains accurate and unbiased. FLOP evaluated its performance on the FMNIST [131], COVIDx [132], and Kvasir [133] datasets, achieving an accuracy of $97.44\%$ for a 10-class classification task.

3.2.3. Homomorphic Encryption

Homomorphic encryption can be used in federated learning to increase security between client iterations. With homomorphic encryption, each client encrypts its models before sharing them. The server then uses the encrypted models on the dataset, generating encrypted results that the client can decrypt after the computation. Two methods discussed in the literature are privacy-preserving [46,134,135,136,137] and blockchain [16].

Kaissis et al. [46] presented an end-to-end privacy-preserving method called privacy-preserving medical image analysis (PriMIA), which is an extension of the PySyft/PyGrid ecosystem available at https://github.com/OpenMined/PySyft (accessed on 8 August 2024). PriMIA uses encrypted aggregation of model updates and encrypted inference. They use augmentation techniques, including MixUp—a method that interpolates pairs of existing examples and their corresponding labels to generate synthetic datasets in a weighted manner, which has been shown to enhance privacy attributes [138]. Additionally, they use a tree-structured Parzen estimator algorithm to efficiently explore the hyperparameter space and find the optimal set of hyperparameters for a given model [139]. PriMIA enables homomorphic encryption, allowing computations to occur on encrypted data without decryption. The encrypted gradients are securely transmitted to the client, aggregated, and used for model updates. Experiments have shown that PriMIA can protect against gradient-based model inversion attacks, in which an attacker tries to infer private information about an individual by using the gradients of a machine learning model trained on that individual’s data. PriMIA evaluated its performance on the MedNIST [70] and X-ray [127] datasets, achieving an accuracy of up to $90\%$ for a three-class classification task, which is $25\%$ higher than the performance of a client training only with its dataset.

Blockchain is a distributed ledger technology that enables secure, transparent, and tamper-proof transactions without intermediaries. While blockchain is commonly used in serial computing, the benefits of decentralised dataset interaction in blockchain are desirable in federated learning to preserve dataset privacy during model training. Aggarwal et al. [16] proposed a privacy-preserving decentralised medical image analysis framework powered by blockchain technology (DeMed). DeMed comprises two essential components, each serving a distinct purpose. The first component is a self-supervised learning module running on the client, obtaining low-dimensional dataset representations. The second component is the smart contract module, which facilitates the secure transfer and retrieval of machine learning model results. Smart contracts, self-executing agreements with predefined conditions encoded on the blockchain, ensure the integrity and immutability of the datasets and results exchanged within the framework. By leveraging the transparency and security features of the blockchain, DeMed establishes a trustworthy environment for sharing and accessing the outputs of machine learning models trained on medical images. However, these methods have yet to demonstrate their computational cost, which might impact practical scenarios. For instance, Ethereum, the most commonly used blockchain, has a significantly high transaction cost, making transmitting models with many parameters impractical [140]. DeMed evaluated its performance on the Pcam [141] and COVIDx [132] datasets, achieving an accuracy of $87.3\%$ for a two-class classification task.

4. Open-Source Framework Implementations

Federated learning has gained significant attention as a promising approach to developing machine learning-based image analysis models while preserving user privacy. Open-source frameworks have played a crucial role in developing and adopting federated learning by providing accessible tools to build and test federated learning models. These frameworks offer a range of features and capabilities to develop and deploy robust and scalable federated learning solutions. Examples include FATE [142], FedML [143], Flower [144], NVFlare [145], OpenFL [146], PaddleFL + PaddlePaddle [147], PySyft + PyGrid [148], TensorFlow Federated [149], and PriMIA [46]. See Table A1 for details.

FATE integrates homomorphic encryption and multi-party computation. It includes a scalable serving system for modelling, an end-to-end pipeline platform, a multi-party communication network, and a managed workload using cloud-native technologies. A current limitation of FATE (v1.8.0) is the lack of a core API, requiring developers to modify the source code to implement their algorithms. FATE does not currently support a decentralised architecture, which may limit its use in certain applications. The source code is available at https://github.com/FederatedAI/FATE (accessed on 8 August 2024).

FedML encompasses a range of capabilities, including model acceleration, computer resource management, and GPU/CPU compatibility. It supports natural language processing, computer vision, graph neural networks, and the Internet of Things. The source code is available at https://github.com/FedML-AI/FedML (accessed on 8 August 2024).

Flower is an agnostic framework that allows users to seamlessly leverage their existing pipelines. Its ability to handle large numbers of clients makes it well-suited for real-world applications. However, Flower (v1.0.0) requires allocating a fixed amount of memory before the process begins, which remains allocated until the process exits. The source code is available at https://github.com/adap/flower (accessed on 8 August 2024).

NVIDIA FLARE offers a high degree of flexibility and customisation. FLARE (v2.1.3) includes extensible management tools that provide secure provisioning, orchestration, and monitoring capabilities for federated learning experiments. The rich programmable APIs allow users to experiment with new workflows and privacy-preserving algorithms. The source code is available at https://github.com/NVIDIA/NVFlare (accessed on 8 August 2024).

Intel’s Open Federated Learning (OpenFL) provides users with a secure and semi-automated process. While OpenFL (v1.3.0) officially supports Linux servers, many workloads are also unofficially supported on Mac and Windows. The source code is available at https://github.com/intel/openfl (accessed on 8 August 2024).

PaddleFL provides a flexible and programmable approach to architecting neural networks, supporting declarative and imperative programming. PaddleFL (v1.2.0) has specific hardware requirements, including a minimum of 6GB RAM and 100GB of storage space, which might limit its usage in some scenarios. The source code is available at https://github.com/PaddlePaddle/PaddleFL (accessed on 8 August 2024).

PySyft and PyGrid enable the implementation of complex privacy-preserving methods, such as secure multi-party computation and differential privacy. Their deep learning API offers an accessible and user-friendly interface. At the same time, their ability to operate at a lower abstraction level provides advanced users greater flexibility and control. The source code is available at https://github.com/OpenMined/PySyft (accessed on 8 August 2024).

TensorFlow Federated (TFF) enables the local simulation of distributed computing. TFF (v0.31.0) is only compatible with the TensorFlow framework. Additionally, the decentralised architecture for building the system is not supported, which may limit its usefulness for specific applications. Nonetheless, TFF remains a valuable tool for users exploring the potential of federated learning and distributed computing. The source code is available at https://github.com/tensorflow/federated (accessed on 8 August 2024).

Privacy-preserving medical image analysis (PriMIA) enables differentially private methods, secure data aggregation, and encrypted inference for imaging datasets. The framework integrates cutting-edge privacy preservation techniques from PySyft and enhances them with features customised for medical imaging. However, deploying PriMIA demands significant computational resources, and encrypted inference’s latency remains considerably higher than unencrypted inference. The source code is available at https://github.com/gkaissis/PriMIA (accessed on 8 August 2024).

5. Discussion

The widespread adoption of federated learning technology depends on several factors, including the availability of suitable infrastructure, the development of robust algorithms, and the establishment of model-sharing policies and protocols. As these factors continue to evolve and improve, the adoption of federated learning is expected to increase. Additionally, the increasing awareness of data privacy and security concerns will likely drive the adoption of federated learning. Furthermore, developing open-source tools and platforms for federated learning will likely accelerate its adoption. These tools and platforms can enable users to experiment with federated learning and develop custom solutions that meet their specific requirements.

A common assumption across the reviewed papers was the availability of well-curated datasets and reliable communication and computational resources, which is unlikely in real-world scenarios. Papers evaluated their method on different datasets (see Table A2, Table A3,

Table A4. Part 3/4 of the detailed literature corpus of the reviewed papers on medical image analysis research on federated learning.

Paper	Medical Data Speciality	Referenced Dataset	Referenced Algorithm	Research Strategy
[214]	Neurology Oncology	SRI24 [215] BraTS [80]	U-Net	Use Case
[216]	Neurology Oncology	QUASAR [217] YCR BCI [218] BraTS [80]	U-Net	Use Case
[219]	Oncology	INbreast [220] VinDr-Mammo [221] CMMD [222]	CNN	Use Case
[223]	Oncology	DDSM [224]	MobileNet DenseNet	Use Case
[134]	Oncology	BreakHis [225]	E-EIE [226]	Privacy
[118]	Oncology	RETOUCH [227]	U-Net	Non-IID
[102]	Oncology	DDSM [224]	ACO [228]	Non-IID
[229]	Oncology	BreakHis [225]	ResNet	Use Case
[67]	Oncology	LC25000 [230]	Fuzzy Rough Sets [231]	Non-IID
[90]	Oncology	MultiChole2022 [232]	ResNet	Non-IID
[104]	Oncology	Kvasir [233]	VGG	Non-IID
[100]	Oncology	ChestX-ray8 [234] IQ-OTH/NCCD [235]	ResNet	Non-IID
[91]	Oncology	LC25000 [230]	Encoder–Decoders	Non-IID
[236]	Oncology	BHI [237]	ResNet GaborNet [238]	Use Case
[239]	Oncology	Microcal [240]	EfficientNet [241]	Use Case
[242]	Oncology	Proprietary data	CNN	Use Case
[243]	Oncology	Baheya [244] BUS-Set [245]	U-Net	Use Case
[246]	Oncology	LC25000 [230]	Inception	Use Case
[247]	Oncology	DDSM [224] VinDr-Mammo [221]	ResNet	Use Case
[248]	Oncology	MSD [249]	U-Net	Use Case
[250]	Oncology	Thyroid [251] Thyroid2 [252]	Swin Transformer [253]	Use Case
[89]	Oncology Miscellaneous (Anatomy Detection)	PBC [254] HyperKvasir [255] LiTS [256]	ResNet	Non-IID
[24]	Oncology	MSD [249] KITS19 [257]	FedAvg [160]	Non-IID
[64]	Respiratory Medicine	QaTa-COV19-v2 [258]	Encoder–Decoders	Non-IID
[105]	Respiratory Medicine	PneumoniaMNIST [181] RSNA ICH [124]	ViT [197]	Non-IID
[259]	Respiratory Medicine	SARS-CoV-2 [212]	MobileNet	Use Case
[260]	Respiratory Medicine Oncology	VinDr-CXR [261] UKA-CXR [262]	ResNet	Use Case
[101]	Respiratory Medicine Oncology	RSNA ICH [124] CheXpert [109] ChestX-ray8 [234]	ResNet	Non-IID
[263]	Respiratory Medicine	COVID X-Ray [264] POCUS [265]	VGG	Use Case
[266]	Respiratory Medicine	CXR [267]	Xception	Use Case
[268]	Respiratory Medicine	SIRM [269] TCIA [208] Radiopaedia [270] PneumoniaMNIST [181] GitHub [271]	DenseNet	Use Case
[136]	Respiratory Medicine	X-RayTransition [272]	VGG	Privacy

and

Table A5. Part 4/4 of the detailed literature corpus of the reviewed papers on medical image analysis research on federated learning.

Paper	Medical Data Speciality	Referenced Dataset	Referenced Algorithm	Research Strategy
[27]	Respiratory Medicine	X-Ray [127]	CNN ResNet VGG AlexNet	Use Case
[28]	Respiratory Medicine	X-Ray [127] COVID X-Ray [264] COVID-19 Radio [273]	CNN	Use Case
[42]	Respiratory Medicine	CheXpert [109]	Graph NN	Non-IID
[29]	Respiratory Medicine	X-Ray [127] COVID X-Ray [264] COVID-19 Radio [273]	FedAvg [160]	Non-IID
[30]	Respiratory Medicine	Not Disclosed	SqueezeNet Glowworm Swarm CovidNet	Use Case
[31]	Respiratory Medicine	LIDC [125]	3D U-Net [92] FedAvg [160]	Non-IID
[32]	Respiratory Medicine	COVID X-ray [264]	ResNet Inception	Use Case
[33]	Respiratory Medicine	Not Disclosed	MobileNet [113] ResNet COVID-Net	Use Case
[34]	Respiratory Medicine	Proprietary Data	RetinaNet	Use Case
[35]	Respiratory Medicine	Not Disclosed	CNN	Use Case
[36]	Respiratory Medicine	Proprietary Data	ResNeXt SVM CNN RNN	Use Case
[26]	Respiratory Medicine	FMNIST [131] COVIDx [132] Kvasir [133]	FedAvg [274]	Privacy
[37]	Respiratory Medicine	Montgomery [275] Shenzhen [276]	StyleGAN [277]	Non-IID
[38]	Respiratory Medicine	X-Ray [127]	INN [126]	Privacy
[39]	Respiratory Medicine	PPPD [278]	ResNet	Privacy
[40]	Urology	PROSTATEx [279]	WGAN-GP CycleGAN FedAvg [160]	Non-IID
[120]	Urology Miscellaneous (Anatomy Detection)	CVC-ClinicDB [280] CVC-ColonDB [281] ETIS [282] Kvasir [233] NCI-ISBI 2013 [208] I2CVB [283] PROMISE12 [284]	U-Net	Non-IID
[119]	Urology Miscellaneous (Anatomy Detection)	RIM-ONE-r3 [285] Drishti-GS [286] REFUGE-challenge [287] NCI-ISBI-2013 [208] I2CVB [283] PROMISE12 [284]	FegAvg MobileNet DeepLabv3+	Non-IID
[288]	Urology	FUrology [289]	ResNet	Use Case

) and used different metrics, which made direct comparison challenging. Datasets, for example, varied in imaging modalities (e.g., ultrasound, X-rays, MRI), conditions of data collection (e.g., controlled vs. real-world), and domain distributions (e.g., inter- and intra-participant, and inter- and intra-medical conditions), and image resolution. What follows is a summary of the challenges:

• Heterogeneous datasets: Medical image datasets come from different settings (medical equipment and data management software) where the prevalence of medical conditions and acquisition protocols may vary. Neglecting these variations when designing machine learning models can lead to performance issues and reduced generalisability of the models.
• Imbalanced datasets: Medical image datasets can often be imbalanced, with a small number of pathological cases and mostly healthy cases; this can lead to model generalisation and performance issues, particularly in scenarios where some rare diseases or conditions require accurate detection.
• Data privacy and security: Maintaining dataset privacy is paramount, requiring strict privacy and security measures. Federated implementations must protect patient data during the model training process.
• Communication: Client communication may be limited due to the high computational cost of transmitting large models. The client may have limited computational power, making it challenging to scale and requiring the development of scalable and efficient machine-learning models that can address large amounts of data. Strategies include adopting lightweight protocol, semi-synchronisation, and model distribution. It should be noted that this review omitted this topic because it falls outside the scope of medical image analysis. However, further details appear in [150].

In alignment with the diversity of medical image modalities, papers addressed the progress and unique challenges in federated learning for different imaging modalities and parameters. This is especially critical as medical images can be acquired using various modalities (e.g., X-rays, MRI, CT, and ultrasound) and customised parameters (e.g., multiband factors in echo-planar imaging acquisition) even within the same modality.

The primary challenge in federated learning for X-ray images is managing the variability in image quality, resolution, and anatomical focus across different datasets. Techniques such as GANs have been instrumental in creating synthetic X-ray images that help balance the training data across different clients. Conditional GANs (cGANs) [72,73] have been used to generate high-quality synthetic images that preserve the original data distribution, improving the model’s generalisation ability across diverse datasets. Virtual adversarial training (VAT) methods [12] have shown promise in regularising models by introducing slight perturbations to the input images, which helps in dealing with the non-IID nature of X-ray datasets in FL settings. However, differences in X-ray machine types and settings across institutions can lead to significant variability in image characteristics, adversely affecting model performance if not properly managed. This can be addressed by incorporating domain adaptation methods in federated learning settings [25,151,152].

The complexity of MRI data, including parameter variations like echo times, repetition times, and multiband factors, presents significant challenges for federated learning. Techniques such as SPFL-Trans [22] leverage client-specific latent variables to adaptively normalise feature maps, thereby preserving important dataset-specific information during federated learning. Adversarial methods like FedVSS [12] use virtual sample synthesis to align the clients’ models with the server’s model, enhancing the generalisation capability by generating synthetic datasets that help bridge the gap between different MRI data distributions. The variation in MRI acquisition parameters, such as different multiband factors, necessitates sophisticated models, such as gradient alignment across clients that can adapt to these variations without losing performance [153].

The main challenges in applying federated learning to CT scans include handling large image sizes and managing differences in scanning protocols. Techniques such as DualGAN [5] combined with evolutionary algorithms have effectively generated diverse and high-quality synthetic CT images, which help mitigate the effects of non-IID data. Methods like FedCRLD [3] use contrastive re-location and momentum distillation to correct representation bias and continually optimise client models, which is particularly useful for handling the large and complex datasets typical of CT scans. A recent work by Ding et al. [154] mitigates the distribution heterogeneity in CT image-based FL across clients. It suppresses the inter-client heterogeneity component by proposing a local drift smoothing (LDS) module that converts the input from feature space to frequency space, thereby improving model generalisability.

Research on federated learning for ultrasound imaging is still in its early stages. Although domain gaps due to biases from different imaging devices, frequencies, and variations in grey distribution and contrast are common in ultrasound datasets from various medical centres; current studies do not explicitly address these issues. For instance, Lee et al. [23] found that the performance of federated learning with decentralised data was comparable to traditional deep learning with pooled data for cancer classification. Similarly, Qi et al. [155] implemented four data partitioning strategies and evaluated four federated learning algorithms to investigate the impact of data distribution on model performance in detecting stenosis using B-mode ultrasound images. However, they focused on class distribution mismatch rather than addressing domain gaps.

6. Final Remarks

We conducted a comprehensive review discussing machine learning-based methods for medical imaging analysis. We provided a taxonomy of selected papers, including medical applications, referenced datasets, technical methods utilised, and a summary of open-source frameworks for developing federated learning. The reviewed literature highlighted two primary challenges: difficulties accessing real-world datasets and preserving privacy. The strategies discussed included non-IID data handling and privacy-enhancing methods.

Federated learning is still a relatively new technology. Its performance may vary depending on several factors, such as the quality and quantity of available datasets, the complexity of the learning task, the number and computational capabilities, the availability and quality of the communication network, the level of privacy and security required, and the efficiency and effectiveness of the federated learning algorithms. Findings in the reviewed literature suggest that federated learning can accelerate the development of machine learning models, leading to practical medical applications if appropriately implemented.