2.1. Federated Learning
Federated learning is a decentralized machine learning architecture where numerous clients work together to build a common global model while keeping their data private [
22]. The usual training procedure contains three major steps: (1) Server Initialization: The central server sets up the initial local models for the clients. (2) Local Training: Each client trains its model with local data and broadcasts the changed model back to the server. (3) Server Aggregation: The server aggregates the local models to build an updated global model, which is subsequently broadcast to the clients. (4) Client Model Update: Clients utilize the revised global model to revise their local models, setting the scene for the upcoming training session.
Currently, many studies are exploring ways to improve federated learning’s handling of non-IID (non-independent and identically distributed) data [
23,
24,
25]. These studies can be categorized into two main approaches: improving local training on clients and enhancing server aggregation.
In terms of improving local training on clients, FedProx [
8] introduces a proximal term based on the distance between the global model and the local models. This term controls the training of local models during the federated learning process. On the other hand, SCAFFOLD [
26] corrects local model updates by introducing control variates during local training and synchronizing these variates across all clients. The gradient in the local training process is controlled by the difference between local and global control variates. However, SCAFFOLD has only been tested on a single dataset, lacking generalization tests on more diverse datasets.
For improving server aggregation, FedMA [
27] employs a Bayesian non-parametric method for layer-wise matching and weighted averaging. FedAvgM [
28] uses momentum to update the global model on the server. Most of these methods primarily allocate weights based on the size of the datasets. In contrast, FedDisco [
21] identifies the difference between local and global class distributions as a more effective criterion for determining aggregation weights.
Most of the existing methods focus on classical federated learning scenarios. However, the complexity of non-IID data significantly increases the difficulty of adversarial defense. The BCMCFAT approach proposed in this paper addresses the robustness loss in federated learning caused by non-IID data. By accounting for both intra-client class distribution differences and inter-client data distribution disparities, BCMCFAT effectively mitigates challenges such as client drift and slow convergence in federated adversarial training. However, similar to traditional adversarial training methods, our approach relies on generating adversarial samples to enhance network robustness, which inevitably increases computational complexity and the resource consumption of clients in the federated learning process.
2.2. Self-Supervised Learning
Self-supervised learning has gained significant interest due to its capability to learn useful representations without relying on costly labeled data [
29,
30,
31]. Specifically, contrastive learning has become a well-liked method in the visual domain. The goal of this approach is to maximize the distance between representations of distinct pictures (negative pairs) and minimize the similarity between representations of enhanced perspectives of the same image (positive pairs) within the feature space.
The most classic contrastive learning framework is SimCLR [
32]. In SimCLR, each image from the original dataset generates two augmented samples,
and
. Each augmented sample is first transformed into a feature representation through a feature extraction network
and then projected into a feature space by a projection head
. The NT-Xent contrastive loss is applied to the projection vectors
. Specifically, given
augmented samples, the contrastive loss can be defined as follows:
where
denotes the cosine similarity function,
represents the feature of the augmented image,
is the temperature parameter, and
is an indicator function that equals 1 when
.
To solve the difficulty of effectively storing and exploiting a high number of negative samples in large-scale datasets, MoCo [
33] proposes momentum-updated encoders and a dynamic negative sample queue. This approach allows for contrastive learning in a larger context, enabling the model to learn richer representations. BYOL [
34] further resolves the dependency on negative samples in contrastive learning by using an exponential moving average technique for updating the target net based on the online network, allowing the self-supervised learning process to track useful signals.
Recent research has explored combining contrastive learning with federated learning. van Berlo et al. [
35] first introduced the concept of federated unsupervised representation learning based on autoencoders, but their study did not consider the non-IID nature of federated data. Zhang et al. [
36] advocated employing a common dictionary module to overcome the non-IID problems in federated learning. Zhuang et al. [
37] combined the classic self-supervised framework BYOL with federated learning and introduced adaptive updates for local predictors to further improve performance. Zhuang et al. [
38] further studied the essential components of federated self-supervised learning frameworks and solved the non-IID issue using the Exponential Moving Average for adaptive updates to local models. Li et al. [
39] aggregated online networks from client nodes using FedAVG and then improved accuracy by applying an incremental moving average update approach to the target network. This study covers supervised learning settings and presents a model comparison to learn distinct data distributions among clients with the goal of achieving optimal aggregation weights, while previous research has concentrated on unsupervised learning environments.
2.3. Adversarial Training
Adversarial examples are samples constructed by introducing tiny changes to original samples to deceive machine learning models [
40]. Malicious users, hackers, or other nefarious actors may create adversarial examples to mislead models and achieve illicit objectives, which significantly affects the practical applications of deep learning networks.
To defend against the potential threats posed by adversarial examples, adversarial defense has increasingly attracted the attention of researchers. One of the best strategies for adversarial defense is adversarial training, which works on the basis of introducing adversarial cases into the model to increase its robustness. Common methods for generating adversarial examples include FGSM [
41], BIM [
42], and PGD [
43].
For instance, FGSM generates adversarial examples as follows:
where
is the magnitude of the perturbation, sign is the sign function that returns the sign of the gradient,
L is the loss function,
denotes the model parameters,
x is the clean input image, and
y is the input label. By applying a single-step perturbation proportionate to the gradient of the loss function with respect to the input image, FGSM creates adversarial instances. This method is computationally efficient but may not always produce the most effective adversarial examples. BIM extends FGSM by applying iterative updates to generate adversarial examples:
where
is the perturbation constraint,
is the magnitude of the pixel update per iteration, sign is the sign function that returns the sign of the gradient,
L is the loss function,
denotes the model parameters,
is the image at iteration
t, and
y is the input label. BIM applies FGSM’s perturbation multiple times, with each step clipping the result to ensure that the adversarial example remains within the allowed perturbation range. This iterative approach often produces more robust adversarial examples compared to a single-step FGSM attack.
PGD further generalizes the iterative approach:
where
represents the perturbation constraint,
is the step size for each iteration, sign is the sign function that returns the sign of the gradient,
L is the loss function,
denotes the model parameters,
is the image at iteration
t, and
y is the input label. PGD applies a fixed step size perturbation iteratively and projects the result to ensure that it remains within the allowed perturbation range. This approach is notable for its ability in producing strong adversarial cases and is often used for testing the resilience of models.
The model learns to recognize and defend against tiny adjustments that aim to mislead its assessments by adding these adversarial events to the usual training dataset and training the model on the augmented dataset. The main training objective of adversarial training can be summarized by the following formula:
where
w represents the model parameters,
denotes the perturbation,
is the perturbation bound,
x is the clean input image, and
y is the input label.
Research on enhancing the adversarial robustness of federated learning is currently quite limited. Directly incorporating adversarial training into federated learning is a straightforward solution. Zizzo et al. [
5] were the first to apply adversarial training methods in federated learning, developing MixFAT to improve the robustness of the global model. MixFAT applies adversarial training to only a portion of local data to enhance the stability and convergence of federated learning. Chen et al. [
20] discovered that there is a significant accuracy gap between MixFAT and centralized adversarial training, especially when data heterogeneity is strong. They proposed CalFAT, which addresses data heterogeneity by increasing the weight of minority classes in client data. Hong et al. [
6] introduced an effective robustness propagation method using batch normalization to address the imbalance in computational resources among participants. Our BCMCFAT is designed for scenarios where participants have average computational resources but face extreme imbalances in data distribution between clients. Compared to existing federated adversarial training methods, BCMCFAT directly focuses on the distribution of non-IID data that causes issues such as client drift, by calibrating the biases introduced by non-IID data at both the client and server levels.