MDIFL: Robust Federated Learning Based on Malicious Detection and Incentives

Abstract

Federated Learning (FL) is an emerging distributed framework that enables clients to conduct distributed learning and globally share models without requiring data to leave the local. In the FL process, participants are required to contribute data resources and computing resources for model training. However, the traditional FL lacks security guarantees and is vulnerable to attacks and damages by malicious adversaries. In addition, the existing incentive methods lack fairness to participants. Therefore, accurately identifying and preventing malicious nodes from doing evil, while effectively selecting and incentivizing participants, plays a vital role in improving the security and performance of FL. In this paper, we propose a Robust Federated Learning Based on Malicious Detection and Incentives (MDIFL). Specifically, MDIFL first uses a gradient similarity to calculate reputation, thereby maintaining the reputation of participants and identifying malicious opponents, and then designs an effective incentive mechanism based on contract theory to achieve collaborative fairness. Extensive experimental results demonstrate that the proposed MDIFL can not only preferentially select and effectively motivate high-quality participants, but also correctly identify malicious adversaries, achieve fairness, and improve model performance.

1. Introduction

The Internet of Things (IoT) has grown rapidly in recent years. Among them, the data generated by emerging IoT applications such as autonomous driving, surveillance and smart home is growing exponentially [1]. Therefore, we need to use advanced machine learning techniques to process these large amounts of data. However, in traditional machine learning, all data are uploaded to the cloud data center for centralized training, which not only requires a lot of time and communication costs for data transmission, but also causes serious data leakage and privacy issues. In addition, it is impractical to upload all data to the cloud, and applications with high real-time requirements should be quickly calculated and stored at edge nodes. In order to solve these problems, federated learning (FL) [2,3,4] was proposed, which uses distributed and edge deployment to realize the original data from not going out of the local area, so as to achieve the purpose of protecting privacy and reducing unnecessary communication resources. Similarly, edge nodes such as sensors, small cells, and servers in Mobile Edge Computing (MEC) can work with remote clouds to complete large-scale distributed tasks.

As shown in Figure 1, participants in federated learning obtain local gradients through training on local datasets. Different gradients will be uploaded to the cloud at the same time, and the aggregator will aggregate and update the model, and then return the updated gradients to the participants. This process continues iteratively until convergence. Although the distributed nature of federated learning provides a certain degree of privacy protection, due to the different geographical and equipment conditions of each user device, there may be malicious nodes launching poisoning attacks [5]. For example, using polluted data for model training will lead to a decrease in the performance of the global model or even training failure, or data leakage without authorization. In addition, due to differences in the computing power and equipment, some participants can only train low-quality gradients, and the long-term aggregation of low-quality gradients will also lead to a decrease in the performance of the global model. Therefore, the selection of participants is very important. It is not only necessary to select high-quality participants from many participants to join the model training, for example, participants with high-quality data [6,7], but also to accurately identify and remove malicious nodes to ensure that the model training is minimally impacted. In addition, in real federated learning, rational participants will not actively participate in model training and actively contribute their own resources. Therefore, it is equally important to design an incentive mechanism to motivate participants [8,9].

However, the existing methods mainly focus on eliminating the single point of the failure problem, lack of effective malicious node detection, and cannot prevent the adversary from destroying the model training in time [10,11]. In addition, some judgment methods of the malicious node are likely to cause misjudgment, resulting in the final training of a low-performance model [12,13]. It is necessary to design an efficient node selection method.

In the MEC [14], participants will first consider whether it is beneficial to themselves, and then choose to contribute their own resources accordingly, and will not make choices that are not beneficial to themselves. Without fair incentives, free-riders attacks are likely to occur [15]. These schemes require effective incentive mechanisms to achieve collaborative fairness.

The contribution of this research is to propose a Robust Federated Learning Based on Malicious Detection and Incentives (MDIFL). First, we propose a node selection method, which uses reputation to select participants, and judges malicious nodes through gradient similarity. Once a malicious node triggers a large-scale pollution model or is suspected more than three times, the mechanism will remove the malicious node. While preferentially selecting participants with high-quality data to join the model training, it is also possible to correctly identify and exclude malicious participants. Secondly, we also propose an incentive mechanism based on contract theory to increase the willingness of participants to join, realize the mutual benefit of participants, and encourage participants to contribute their own high-quality data and computing resources as much as possible with a fair and reasonable incentive method. Furthermore, we evaluate the performance of the proposed MDIFL scheme through extensive experiments. Extensive experiments on benchmark datasets demonstrate that MDIFL can achieve high security and cooperative fairness, and improve the performance of the algorithm.

The structure of this paper is as follows. We review related work in Section 2, describe the MDIFL scheme in detail in Section 3, and present and discuss experimental results in Section 4. Finally, this work is concluded in Section 5.

2. Literature Review

In this section, we review the relevant literature on node selection and participant incentives in order to connect our study with existing research.

2.1. Node Selection Issue

Due to the large number of heterogeneous clients in the federated learning environment, the adverse effect of data heterogeneity will be exacerbated if participants are randomly selected. Therefore, it is crucial to preferentially select high-quality participants, which can quickly improve the model performance. Lai et al. [16] propose an actor selection framework (Oort) that can identify and select those actors for training and testing who have both data that provide the greatest utility in improving model accuracy and the ability to run training quickly. Chai et al. [17] proposed a layer-based federated learning system (TiFL), which divides participants into layers based on their training performance and selects clients from the same layer in each round of training. Wang et al. [18] propose an experience-driven federated learning framework (Favor) based on reinforcement learning, which intelligently selects client devices to participate in each round of federated learning to counteract the bias introduced by the non-IID data and accelerate convergence. Kang et al. [19] took reputation as a fair measure, and used a multi-weight subjective logic model to efficiently calculate the reputation of participants to select reliable participants for federated learning to defend against unreliable model updates in mobile networks.

In addition, although the data set does not leave the local area during the federated learning process, a certain degree of privacy protection is achieved, but the federated learning may also be subject to poisoning attacks [20]. For model poisoning, Bhagoji et al. [21] propose two key stealth concepts to detect malicious updates, use an alternating minimization strategy to improve the stealth of the attack, and demonstrate that effective and covert model poisoning attacks are possible. In addition, since the participants participating in the model training task are all rational, if malicious participants appear, they may destroy the training of the model by changing the gradient parameters of the model uploaded to the aggregator. Fang et al. [22] conducted a systematic study on the local model poisoning attack of federated learning, and the results demonstrated that the Byzantine-robust federated learning method can not defend against poisoning attacks in all cases; thus, new defense measures need to be proposed to resist a poison attack. Zhang et al. [23] proposed an unsupervised approach, where the server predicts the model updates of participants in each iteration based on historical model updates, identifies whether participants are malicious, and removes most of the malicious participants. Xu et al. [12] proposed a precision-based malicious node detection mechanism; through the results of each training, each participant is marked with a reputation score, and nodes with a reputation score below zero will lose the opportunity to join any training process. Xu et al. [13] proposed a novel robust and fair federated learning (RFFL) framework to maintain the reputation of each participant by uploading gradients to check each participant’s contribution, thereby identifying non-contributing or malicious participation to be removed.

However, the scheme proposed by Xu et al. [12] did not take into account that malicious nodes may accumulate reputation scores to prepare for future evil and continue to participate in training. During the training process, when a malicious node has a great impact on the model training, the mechanism will still choose to deduct the reputation score. As long as the reputation score is non-negative, the malicious node will continue to participate in the model training. Therefore, when the reputation score is large enough, malicious nodes may perform large-scale destruction behaviors many times, which will greatly reduce the performance of the global model, resulting in model training failure. Similarly, RFFL in [13] only sets a threshold to identify malicious nodes, and it is easy to exclude those participants whose model performance is low due to external factors in a certain training. In this regard, the judgment method we propose considers each round of the independent judgment of malicious nodes and double threshold settings, which is more comprehensive than existing methods.

2.2. Incentive Mechanism

Incentives are an important research area in federated learning because the participants are rational. Xu et al. [12] proposed a contribution-based incentive mechanism, using the improvement of model accuracy and model training time to quantify the contribution of participants, and according to the ranking of the participants’ contribution, each participant’s different degrees of substitution Coin rewards. Kang et al. [19] combined reputation with contract theory to incentivize high-reputation mobile devices with high-quality data to participate in model learning. Lim et al. [24] proposed a hierarchical incentive framework, using contract theory and alliance game theory to reward workers and model owners, respectively. Kang et al. [25] adopted contract theory to design an effective incentive mechanism to simulate mobile devices with high-quality data to participate in federated learning. Li et al. [26] propose a novel optimization objective (q-FFL) inspired by a fair resource allocation in wireless networks, which encourages a fairer distribution of accuracy across devices in a joint network.

The above method only considers that the participants’ own choices will affect the final reward, which may cause the participants to only consider their own interests. However, in MDIFL, the interests of participants are also related to the decision-making of other participants. When the participants motivate each other and contribute considerable resources, there will be additional benefits, so as to achieve higher benefits together.

3. Materials and Methods

In this section, we will select high-quality participants to join the model training to improve the performance of the model, and identify malicious nodes to avoid damage to the model training task. In addition, we use the contract theory to implement a reasonable incentive mechanism, increase the willingness of participants to join the training task of the federated learning model, and contribute their own data resources and computing resources.

3.1. Node Selection

To improve model performance, cosine similarity has been used to determine the quality of gradients [27,28]. In terms of participant selection, we use cosine similarity to determine the gradient quality of participants’ training epochs, resulting in gradient similarity. The reputation is calculated through the gradient similarity, and participants with a high reputation will be selected to join the model training in each round of training. In other words, the higher the reputation, the higher the quality of the data the participant has, and the better the performance of the final trained model. The reputation of the participants is calculated by the aggregator. Here, we consider that there are N participants, $P=\{P_1,P_2,\cdots ,P_i,\cdots ,P_N\}$ is shown as the set of all participants. Initially, at the beginning of model training, the reputation of each participant will be initialized to zero, and $R_i$ is used to represent the reputation of participant $P_i$, which can be specifically expressed as Equation (1).

$$\begin{array}{c}\hspace{1em}{R}_i=\beta R_i^{t-1}+(1-\beta )(1-e^{R_i^t})\\ s.t.R_i^t=cos(\Delta {\omega }_g^t,\Delta {\omega }_i^t)=\frac{\Delta {\omega }_g^t·\Delta {\omega }_i^t}{\Vert \Delta {\omega }_g^t\Vert ·\Vert \Delta {\omega }_i^t\Vert }\end{array}$$

(1)

Among them, $\beta$ is a weight factor. We use $\beta$, which can be specified by the model aggregator, to control the degree of importance between the early and late reputation of the participants. If the model aggregator pays attention to the early reputation of the participants, then the value of $\beta$ is set close to 1. Oppositely, the model aggregator pays attention to the late reputation of the participants, then sets the value of $\beta$ to be small. $R_i^t$ is the reputation obtained by participant $P_i$ in the t-th round of training, and the similarity is evaluated by calculating the cosine value of the angle between the global gradient parameter $\Delta {\omega }_g^t$ and the local gradient parameter $\Delta {\omega }_i^t$ of participant $P_i$ in the t-th round. The smaller the angle, the higher the similarity, and the better the performance of the trained model.

During the first round of model training, the model aggregator randomly selects participants to join the training and calculates the reputation of each participant. In the following training rounds, the model aggregator will sort all participants according to their reputation values, and at the beginning of each round of training, they will give priority to participants with higher reputations to join. This is conducted to improve the quality of the participants who join the training. For ease of reference, we refer readers to

Table 1. Symbol table.

Symbol	Description
P	Collection of participants
$\beta$	Reputation weighting factor
$m{P}_i$	Malicious participant
$s{P}_i$	Suspected as malicious participant
$g{P}_i$	Good participant
$\Delta {\omega }_g^t$	Global gradient of round t
$\Delta {\omega }_i^t$	Local gradient of participant $P_i$ in round t
$R_i^t$	Cosine similarity between $\Delta {\omega }_g^t$ and $\Delta {\omega }_i^t$
${\lambda }_1$	Threshold for judging malicious nodes 1
${\lambda }_2$	Threshold for judging malicious nodes 2

for commonly used notations.

In terms of malicious node detection, gradient similarity is also used to judge whether a participant is malicious. If a participant behaves maliciously, resulting in a decrease in the accuracy of the training model or a failure in training, the malicious node will be punished accordingly. Here, we simulate three kinds of nodes, namely malicious nodes (M_P), general nodes (S_P) and high-quality nodes (G_P). Among them, the general node refers to the node with average data quality, we call it S_P; this node does not intentionally do evil, but because of other reasons such as low data quality, the performance of the model trained each time is low. If it is kept low for a long time, the performance of the model participates in global aggregation, which may eventually degrade the performance of the entire model. Equation (2) shows the judgment method of malicious nodes.

$$P_i=\left\{\begin{array}{cc}\hfill m{P}_i,& \hfill if{R}_i^t<{\lambda }_2\hfill \\ \hfill s{P}_i,& \hfill \hspace{1em}\hspace{1em}if{\lambda }_2\le R_i^t<{\lambda }_1\hfill \\ \hfill g{P}_i,& \hfill if{R}_i^t\ge {\lambda }_1\hfill \end{array}\right.$$

(2)

$m{P}_i$, $s{P}_i$ and $g{P}_i$ refer to identified malicious nodes, suspected nodes and high-quality nodes, respectively. Here, we need to set two thresholds as ${\lambda }_1$ and ${\lambda }_2$ as a criterion for judging malicious nodes. This can be divided into three situations:

Case1: When $R_i^t$ is less than ${\lambda }_2$, the participant $P_i$ is set as the malicious node $m{P}_i$, and the node is prohibited from continuing to participate in this federated learning model training task.

Case2: When $R_i^t$ is greater than ${\lambda }_2$ and less than ${\lambda }_1$, the participant $P_i$ is set as the suspected node $s{P}_i$. At this time, the participant may be malicious, or may have low cosine similarity due to low-quality local data or low computing power, but it has little impact on model training. Such actors cannot appear more than once, otherwise it will degrade the performance of the whole model. Therefore, when the number of suspected nodes $s{P}_i$ is greater than 3, it is also forbidden to continue to participate in this federated learning model training task.

Case3: When $R_i^t$ is greater than ${\lambda }_1$, the participant $P_i$ is set as a high-quality node $g{P}_i$, and is allowed to continue to participate in this federated learning model training task.

In addition, you need to pay attention to the setting of the threshold $\lambda$. If the threshold $\lambda$ is set to be small, malicious nodes cannot be accurately identified, but if the threshold $\lambda$ is set large, it is easy to judge high-quality nodes as malicious nodes. Here, we have obtained a relatively suitable value range of the threshold $\lambda$ based on a large number of test experiments and experience. At the same time, the threshold $\lambda$ can also be set to an appropriate threshold $\lambda$ according to the security expectations of different schemes and different data. Similarly, the allowed number of occurrences of $s{P}_i$ can also be set according to the safety expectation. When the safety expectation is low, multiple occurrences of $s{P}_i$ can be allowed, and on the contrary, the number of occurrences of $s{P}_i$ will be reduced.

The malicious node detection mechanism we proposed effectively avoids the situation where participants do not do evil in the early stage, accumulate a large number of reputation scores, and then destroy the global model to a large extent. It also allows participants to appear as case2 no more than three times, so as to avoid misjudgment. The specific detailed implementation is given in Algorithm 1.

Algorithm 1: MDIFL Algorithm for Server

3.2. Incentive Mechanism

With the help of contract theory in economics, this paper proposes an incentive mechanism based on contract theory to encourage participants to contribute more computing resources and data resources to model training. In traditional contract theory [25], different types of participants can maximize their own interests only when they choose contracts designed corresponding to their types. However, in the incentive mechanism we propose, the benefits of participants are not only related to the type of contract they choose, but are also related to the decisions of other participants. Each participant needs to choose a contract designed for its type, and there will be additional benefits when motivating each other and contributing considerable resources, so as to achieve higher benefits together.

3.2.1. Problem Formulation

The resources consumed by participants can be divided into computing resources and communication resources [25]. The time it takes for participant $P_n$ to iterate once in local model training is

$$T_n^{cmp}=\frac{c_n{s}_n}{f_n}$$

(3)

$c_n$ is the number of CPU cycles for single data training, $s_n$ is the size of the local data sample of participant $P_n$, and $f_n$ is the computing resource contributed by each participant $P_n$ for local model training, that is, the CPU cycle frequency. The transfer rate of participant $P_n$ can be expressed as

$$r_n=Bln(1+\frac{{\rho }_n{h}_n}{N_0})$$

(4)

B is the transmission bandwidth, ${\rho }_n$ is the transmission power of participant $P_n$, $N_0$ is the background noise, $h_n$ is the channel gain of the peer-to-peer link between participant $P_n$ and the aggregator. Considering the data size $\sigma$ of the local model update as a constant, the transmission time of the local model update can be obtained as

$$T_n^{com}=\frac{\sigma }{Bln(1+\frac{{\rho }_n{h}_n}{N_0})}$$

(5)

In this way, the total time for a complete global iteration of participant $P_n$ can be expressed as

$$T_n^t=\tau T_n^{cmp}+T_n^{com}$$

(6)

$\tau$ is the number of local model update iterations. In addition, the CPU energy consumption of participant $P_n$ is

$$E_n^{cmp}\left(f_n\right)=\zeta c_n{s}_n{f}_n^2$$

(7)

$\zeta$ denotes the effective capacitance parameter of the computing chipset of participant $P_n$. The energy consumed by participant $P_n$ to transmit local model updates in global iterations is

$$E_n^{com}=T_n^{com}·{\rho }_n=\frac{\sigma {\rho }_n}{Bln(1+\frac{{\rho }_n{h}_n}{N_0})}$$

(8)

The total energy consumption of a complete global iteration of participant $P_n$ can be obtained as

$$E_n^t=\tau E_n^{cmp}+E_n^{com}$$

(9)

The types of participants can be divided into M categories, ${\theta }_1<{\theta }_2<{\theta }_3<\cdots <{\theta }_m<\cdots <{\theta }_M,n\in \{1\cdots M\}$. Among them, ${\theta }_m$ represents the quality of the data that the participants have. The larger ${\theta }_m$ means the higher the quality of the data the participants have, the higher the willingness to participate in the training. Similarly, participants can be divided into M categories from high to low according to the amount of data they have, $s_1<s_2<s_3<\cdots <s_m<\cdots <s_M,n\in \{1\cdots M\}$. The larger $s_m$ means the more data it has. That is to say, the participants of type M are the best participants, whereas the participants of type 1 are relatively the worst. If participants provide more large and high-quality data, this will make the model performance higher. Model performance can be expressed as

$$M({\theta }_n,s_n)=1-e^{-\phi \left({\theta }_n{s}_n\right)}$$

(10)

$\phi$ is the weighting factor. Assuming that the number of participants in the model training task S is $\left|S\right|$, the total profit function of the entire training task can be expressed as

$$\begin{array}{c}p\left(s\right)={\displaystyle \sum _{n=1}^{\left|S\right|}}[(1-e^{-\phi \left({\theta }_n{s}_n\right)})-G_n]\\ \hspace{1em}s.t.G_n=\gamma E_n^t=\gamma (\tau \zeta c_n{s}_n{f}_n^2+\frac{\sigma {\rho }_n}{Bln(1+\frac{{\rho }_n{h}_n}{N_0})})\end{array}$$

(11)

$G_n$ is the cost consumed by the participants, and $\gamma$ is the predefined weight parameter for energy consumption.

Due to information asymmetry, for different types of participants with different data quality, the aggregator should design corresponding different types of contracts to maximize benefits. Therefore, we can obtain the maximized utility function of participant $P_n$, as follows

$$\begin{array}{l}max{U}_n=R_n-G_n+log(\omega \frac{p(s\backslash \{n\left\}\right)}{p\left(s\right)}+1)\\ s.t.A_n=log(\omega \frac{p(s\backslash \{n\left\}\right)}{p\left(s\right)}+1)\end{array}$$

(12)

$R_n$ and $A_n$ are the corresponding contract rewards and extra income of type n participants, respectively. $p(s\backslash n)$ is the total profit function of the remaining participants in the training task, except the participant $P_n$ in the training task S. $\omega$ is the weight parameter. If a participant behaves maliciously, the aggregator will prohibit the participant from continuing to participate in the model training and stop reward distribution.

In this way, we can obtain the maximum utility function of the aggregator as

$$\begin{array}{l}max{U}_a=p\left(s\right)-{\displaystyle \sum _{n=1}^{\left|S\right|}}{A}_n-\left|S\right|C\\ \hspace{1em}\hspace{1em} ={\displaystyle \sum _{n=1}^{\left|S\right|}}[(1-e^{-\phi \left({\theta }_n{s}_n\right)})-G_n-A_n-\left|S\right|C\\ s.t.C=\mu T_n^t=\mu (\tau \frac{c_n{s}_n}{f_n}+\frac{\sigma }{Bln(1+\frac{{\rho }_n{h}_n}{N_0})})\end{array}$$

(13)

C is the communication cost and $\mu$ is a predefined weight parameter for time.

3.2.2. Optimal Contract Design

The contract theory uses the self-disclosure mechanism under information asymmetry. In order to make the contract feasible, each contract must meet the following constraints to ensure that each participant receives a fair and appropriate incentive reward.

• Individual Rationality (IR): Participants will only participate in the task of federated learning model training if their maximum utility is not less than zero, i.e.,

  $$U_n=R_n-G_n+A_n\ge 0$$

  (14)
• Incentive Compatibility (IC): Each participant of type m can maximize utility only if he chooses the contract $R_m$ designed for his type, i.e.,

  $$R_n-G_n+A_n\ge R_m-G_n+A_n,\forall n,m\in \{1,\cdots ,N\},n\ne m$$

  (15)

According to the above definition, we have the following lemma.

Lemma 1.

Monotonicity.

For any feasible contract, when ${\theta }_n\ge {\theta }_m$, we obtain $R_n\ge R_m,\forall n,m\in \{1,\cdots ,N\},n\ne m$. Therefore, it should be satisfied

$$\left\{\begin{array}{l}0<R_1<R_2<\cdots <R_m<\cdots <R_n\\ 0<s_1<s_2<\cdots <s_m<\cdots <s_n\end{array}\right.$$

(16)

Lemma 2.

The other individual rationality constraints also hold if the individual rationality constraints of type 1 participants are satisfied, where type 1 refers to the type of participants with the worst data quality.

Lemma 3.

Local downward Incentive constraint.

According to the monotonicity of Lemma 1, the incentive compatibility condition can be simplified as a local downward Incentive constraint (LDIC), which can be expressed as

$$R_n-\tau E_n^{cmp}+A_n\ge R_{n-1}-\tau E_{n-1}^{cmp}+A_{n-1},\forall n\in \{2,\cdots ,N\}$$

(17)

Under the constraints of individual rationality and incentive compatibility, we can obtain rewards [25], which can be expressed as

$$R_n=\tau E_1^{cmp}+E_n^{com}+\sum _{k=1}^{\left|S\right|}{\Delta }_k$$

(18)

${\Delta }_k=\tau E_k^{cmp}-\tau E_{k-1}^{cmp}$, and ${\Delta }_1=0$. We can rewrite the utility function to obtain

$$\begin{array}{c}max{U}_n=\tau E_1^{cmp}+E_n^{com}+{\displaystyle \sum _{k=1}^{\left|S\right|}}{\Delta }_k\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}-\gamma (\tau \zeta c_n{s}_n{f}_n^2+\frac{\sigma {\rho }_n}{Bln(1+\frac{{\rho }_n{h}_n}{N_0})})+A_n\\ s.t.R_1-G_1+A_1>0\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{R}_n-G_n+A_n\ge R_m-G_m+A_m\\ {\displaystyle \sum _{n=1}^{\left|S\right|}}{R}_n\le R_{max}\end{array}$$

(19)

$R_{max}$ is the total reward budget.

4. Experiment Results and Discussion

In this section, we provide the experimental setup, then perform numerical experiments and analyze the experimental results to evaluate our proposed MDIFL.

4.1. Experimental Setup

4.1.1. Datasets

We conduct experiments on three datasets, namely MNIST [29], CIFAR-10 [30] and text classification datasets: Movie Review (MR) [31]. MNIST is a large database of handwritten digits collected by the National Institute of Standards and Technology. The training set contains 60,000 images and labels, and the test set contains 10,000 images and labels. CIFAR-10 is a small dataset for universal object recognition. A total of 10 categories of the RGB color images are included, and there are 50,000 training images and 10,000 test images in the dataset. MR contains both positive and negative comments. We use a two-layer convolutional neural network (CNN) for MNIST and a three-layer CNN for CIFAR10, and a text embedding CNN for MR. In addition, we vary the dataset size for heterogeneous data in addition to considering the standard independent and identically distributed (I.I.D) data distribution (UNI). We randomly partition 3000, 6000, 12,000 MNIST examples among 5, 10, 20 participants, respectively, following power-law partitioning (POW) [32]. Obviously, each participant has a different number of examples, with the first participant receiving the fewest examples and the last participant receiving the most examples.

4.1.2. Baselines

To verify the effectiveness of our proposed method, we examine the performance by two metrics, predictive performance and robustness. The classic FedAvg [32], q-FFL for fair resource allocation in [26] and the state-of-the-art RFFL proposed in [13] are compared with our method MDIFL.

4.1.3. Attack Settings

We consider three types of adversaries, namely free-riders [33], label flipping attack in directed poisoning [34] and non-directed poisoning attack [35].

• Free-riders: Free-riders is a passive attack, which means that participants only use the global model to update their own local model and refuse to provide valuable local information to the global model. They usually upload random gradients.
• Targeted poisoning attack: This is such as label-flipping, which refers to the fact that the labels of honest training examples of one class are flipped to another class, while the characteristics of the data remain unchanged. For example, the adversary is trained using ‘1’ as the label for the actual image ‘7’, resulting in distorted gradients.
• Non-targeted poisoning attack: Randomize the sign, rescale the gradient, and invert the value, respectively.

In addition, we also consider a non-malicious actor (S_P), whose trained model performs poorly due to having low-quality data, etc.

4.1.4. Hyper-Parameters

We set the reputation weight as $\beta$ = 0.9, and the thresholds for judging malicious nodes in MNIST and CIFAR10 as ${\lambda }_1=0.6$ and ${\lambda }_2=0.5$, and in MR ${\lambda }_1=0.4$ and ${\lambda }_2=0.35$.

4.2. Reputation Setting and Malicious Node Identification Analysis of MDIFL

In this section, we analyze the proposed MDIFL’s reputation setting method and malicious node identification method. We set up three types of participants: (1) malicious nodes (M_P), such as: free-riders and label-flipping, etc.; (2) participants who do not intend to do evil, but have low-quality data or other factors that lead to low performance of the trained model (S_P); (3) high-quality nodes (G_P).

As shown in Figure 2, M_P is directly assigned a reputation of 0 after being identified in the first round, and will not be able to continue to participate in federated learning training tasks in the future. After the S_P class participants are identified three times, they also lose the opportunity to continue to participate in the training and assign the reputation value to 0. At the beginning of the next training, we will provide priority to selecting participants with a high reputation from the remaining G_P to continue to participate in model training.

The method of MDIFL judging malicious nodes is described in detail in Figure 3, and we select some representative nodes among all nodes as a display. Among them, it can be clearly observed that M_P cannot participate in training after the malicious node is identified in the first round. S_P was identified three times; thus, the gradient similarity is in the range of $({\lambda }_1,{\lambda }_2)$, and it was also excluded.

4.3. Comparison of MDIFL with Other Methods under Different Datasets

In terms of a predictive performance,

Table 2. Maximum accuracy (%) including S_P in different situations.

	MNIST		CIFAR10		MR
N	10		10		5
Data Split	UNI	POW	UNI	POW	POW
FedAvg	94	94	39	37	52
q-FFL	85	27	41	36	12
RFFL	94	93	45	42	53
MDIFL	96	96	51	53	56

illustrates the maximum accuracy of the model performance in different cases. We cooperate with the incentive mechanism and node selection, and it can be clearly observed that MDIFL is superior to other methods as a whole, especially for heterogeneous data, where POW is used to split data.

In terms of robustness, we compare FedAvg and RFFL with our method MDIFL in different data sets with 20% and 110% malicious nodes, respectively, and use accuracy as the evaluation index.

As shown in Figure 4 and Figure 5, the accuracy comparison of different schemes is shown in MNIST and CIFAR10, and the number of S_P is set to 20% of the number of honest participants. It can be observed that the addition of a small number of S_P participants has a little effect on MNIST, but the final convergence accuracy of FedAvg and RFFL is still lower than that of MDIFL. This is due to the fact that a small amount of S_P does not have a large impact on the overall model performance. However, under CIFAR10, FedAvg and RFFL are relatively affected. It can be observed that the accuracy of the early model fluctuates greatly, and finally converges to a lower position than MDIFL.

In addition, as shown in Figure 6 and Figure 7, in MNIST and CIFAR10, the accuracy comparison of different schemes is shown, respectively, when the number of S_P is set to 110% of the number of honest participants. It can be clearly observed that after adding a large number of S_P class participants, the accuracy of FedAvg and RFFL under the two data sets both dropped significantly in the early stage and fluctuated greatly. This is because neither FedAvg nor RFFL can recognize S_P. When a large number of models trained by S_P are aggregated, the overall model accuracy will be reduced and the performance will be poor. Therefore, FedAvg and RFFL cannot maintain a stable accuracy under the influence of such participants. On the contrary, MDIFL can identify such participants and exclude them in time, effectively avoiding the impact on the overall model. It can be found that the accuracy of our proposed MDIFL has been rising steadily and reached a better accuracy.

In addition, we also analyzed five attack methods, and the results are shown in

Table 3. Maximum accuracy (%) of five attack modes for MNIST under UNI with 10 G_P and additional 20% M_P.

	Free-Riders	Label- Flipping	Sign- Randomizing	Re-Scales	Value- Inverting
FedAvg	96	89	10	10	9
RFFL	96	96	96	96	96
MFSFL	96	96	96	96	96

and

Table 4. Maximum accuracy (%) of five attack modes for MNIST under UNI with 10 G_P and additional 110% M_P.

	Free-Riders	Label- Flipping	Sign- Randomizing	Re-Scales	Value- Inverting
FedAvg	93	88	94	10	9
RFFL	96	96	96	96	96
MFSFL	96	96	96	96	96

. It can be observed that FedAvg is not robust to the three attack methods of non-directional poisoning under the MNIST of UNI data segmentation, no matter whether the number of malicious adversaries is 20% or 110%. RFFL has certain robustness to these five attack methods, but it cannot accurately identify S_P, resulting in a low performance of the overall model.

Table 5. Maximum accuracy (%) and Attack success rate (%) for MNIST under UNI with 10 G_P and additional 110% S_P.

	Maximum Accuracy	Attack Success Rate
FedAvg	94.5	0.7
RFFL	94.0	0.5
MFSFL	95.8	0

shows that although RFFL can correctly identify free-rider attacks, directed poisoning attacks and non-directed poisoning attacks, it cannot recognize S_P. FedAvg also does not recognize S_P. If the number of S_P is large enough, the long-term aggregation of gradients uploaded by such participants will affect the overall model accuracy, resulting in the final training of a low-performance model. Since MDIFL has added the method of identifying and judging S_P, the accuracy has been improved. These results all show that MDIFL is the most robust overall.

Moreover, we also put MDIFL under different amount of free-riders for comparison. The results show that our proposed MDIFL can handle any number of malicious nodes. As shown in Figure 8, the number of malicious nodes is set to 2, 5 and 11, respectively. In the first round of training, as the number of malicious nodes increases, the accuracy of the model decreases. After the malicious nodes are accurately identified and removed in time, it can be clearly observed that the accuracy increases rapidly until it becomes stable, and finally the model can reach convergence. However, when the number of malicious nodes reaches a certain level, there will be fewer relatively honest nodes. The accuracy of the model trained in this way may not be high, and the model convergence speed will slow down. Therefore, when most of the participants are malicious, stop and end the model training.

5. Conclusions

In this study, we propose a Robust Federated Learning Based on Malicious Detection and Incentives (MDIFL). Based on the gradient similarity, we preferentially select high-quality nodes to participate in training, and identify malicious nodes, so as to prevent damage to model training from being destroyed. The incentive mechanism of collaborative fairness is realized by introducing contract theory. A large number of experiments were carried out on three data sets, and two data distribution methods of UNI and POW were also considered. The experimental results show that, compared with the existing methods, MDIFL can not only correctly identify various types of adversaries, but also eliminate malicious nodes in time, ensuring that the performance of the model will not be degraded, while achieving high accuracy, and also implement fair rewards for different types of participants. We conclude that the used method is effective in detecting malicious nodes and incentivizing participants.

However, the study was not without its flaws. Although FL does not require data to leave the local area, the gradient parameters of the model may still be attacked, resulting in privacy leaks. Therefore, in future work, we will consider privacy protection for the gradient parameters of the model, and techniques such as differential privacy or homomorphic encryption can be used to find a more secure training strategy. In addition, it is also considered to optimize the number of iterations in the training process, so that model training can quickly achieve a higher accuracy with fewer rounds.