Federated Learning Incentive Mechanism Design via Shapley Value and Pareto Optimality

Abstract

Federated learning (FL) is a distributed machine learning framework that can effectively help multiple players to use data to train federated models while complying with their privacy, data security, and government regulations. Due to federated model training, an accurate model should be trained, and all federated players should actively participate. Therefore, it is crucial to design an incentive mechanism; however, there is a conflict between fairness and Pareto efficiency in the incentive mechanism. In this paper, we propose an incentive mechanism via the combination of the Shapley value and Pareto efficiency optimization, in which a third party is introduced to supervise the federated payoff allocation. If the payoff can reach Pareto optimality, the federated payoff is allocated by the Shapley value method; otherwise, the relevant federated players are punished. Numerical and simulation experiments show that the mechanism can achieve fair payoff allocation and Pareto optimality payoff allocation. The Nash equilibrium of this mechanism is formed when Pareto optimality payoff allocation is achieved.

1. Introduction

As artificial intelligence (AI) continues to develop at a rapid pace, massive and diverse data are rapidly being generated. Data from all walks of life have high utility value and privacy information. In particular, there are data islands and data privacy problems. To solve the problems of data islands and data privacy, McMahan et al. [1,2,3] proposed a distributed machine learning method called federated learning (FL) in 2016. FL is used to train the model at each node, and their own sensitive data are not leaked, which not only makes full use of the data to train models but also protects sensitive data privacy [4].

However, in the FL process, to make each data owner willing to contribute their data to train the model and improve the accuracy of the model, designing an incentive mechanism in the FL system is a valuable research topic. If there is a phenomenon of free riding among federated players, such as the lack of real-time training data, the accuracy of federated model training will be seriously affected. Therefore, establishing an attractive incentive mechanism is valuable research work in the FL system.

Nowadays, the FL incentive mechanism has attracted much attention from academics [5,6]. They have conducted a lot of research in [7,8,9], and although they considered fairness [10,11], they did not consider the Pareto optimality efficiency. We aim to establish an incentive model in a federated system to achieve fair payoff allocation and Pareto optimality. For this purpose, we propose a method via the Shapley value and Pareto optimality. Since the Shapley value is consistent with the principle of budget balance, however, according to Holmstrom’s team production theory [12], budget balance and Pareto optimality cannot be reached simultaneously. Therefore, although the Shapley value method can satisfy fair payoff allocation after the completion of FL, it cannot achieve the optimality incentive input of each federated player before FL, i.e., it cannot achieve Pareto efficiency optimization before FL. Therefore, we design this mechanism for the payoff allocation of FL to make the payoff allocation of FL fair and efficient. We introduce supervisor and set penalty conditions. If the federated payoffs reach Pareto efficiency optimality, the payoffs are allocated by Shapley’s value formula; otherwise, the relevant federated players are penalized. Finally, numerical experiments are performed to confirm our theoretical analysis. The main contributions of the paper can be summarized as follows:

(1)

Discussing the conditions satisfied by the fines paid to the regulator by the limited-liability federated agent if Pareto optimality is achieved;

(2)

Demonstrating that the federated players’ inputs constitute the mechanism’s Nash equilibrium when Pareto optimality is satisfied;

(3)

Numerical examples are performed to verify the rationality of designing the mechanism for the both equal and unequal statuses of the federated players.

The remaining work is organized as follows: The related work is introduced in Section 2. The preliminaries are introduced in Section 3. The FL incentive mechanism is established in Section 4. The rationality of this incentive mechanism is verified by numerical and simulation experiments in Section 5. The conclusion and future work are drawn in Section 6. The discussion is drawn in Section 7. The proof of the theorem is given in Appendix A.

2. Related Work

The main theoretical approaches currently used for the FL incentive mechanism include the Stackelberg game [13], contract theory [14], auction mechanism [15] and Shapley value [16]. In this paper, we review the research works related to the FL incentive mechanism design by the Shapley value. In [17], Song et al. proposed a new Shapley value based on the contribution metric to evaluate the contribution of each player who owns the data for the training of the FL model. In [18], the authors proposed a new expression for the Shapley value for FL, which can be computed without consuming additional communication costs, can play a role in the value of the FL data, and can have an incentive effect on the players. Wang et al. [19] used the Shapley value to fairly calculate each federated player’s contribution. To properly incentivize data owners to contribute their data to train the federated model, in [20], the authors proposed a blockchain-based peer-to-peer payment system for FL to achieve a feasible fair payoff allocation mechanism based on the Shapley value. In the literature [21], a fair incentive mechanism based on the Shapley value was proposed. This can motivate more players to be willing to share their data and receive a certain fee. In [22], the authors proposed the bootstrap truncated gradient Shapley approach for the fair valuation of the FL players’ contributions. This approach mainly reconstructs the FL model from gradient updates for Shapley value calculation. Nagalapattiet et al. [23] proposed a cooperative game, where players share gradients and compute players’ Shapley values to filter those with relevant data. To address the fact that there are still other inequities in calculating the FL Shapley value, Fan et al. [24] proposed a new complete federated Shapley value mechanism to improve the fairness of the federated Shapley value. In addition, to address the fact that the calculation of the Shapley value in FL requires a certain communication cost, in [25], the authors proposed a Shapley value based on a contribution evaluation metric called the vertical federated Shapley value (VerFedSV) and verified the fairness of VerFedSV through experiments. In [26], the authors considered several factors affecting FL and proposed an FL incentive mechanism according to the enhanced Shapley value method, and numerical experiments verified that the payoffs allocated among all participants can be fairer when using the enhanced Shapley value method.

3. Preliminaries

3.1. Federated Learning Framework

Federated learning (FL) is a distributed machine learning technique or machine learning framework. The goal of FL is to technically break down data silos and enable AI collaboration, where participants’ data do not leave the local area during the model training process, to achieve common modeling based on ensuring data privacy, and security and legal compliance [1].

Let $N=\{1,2,\dots ,n\}$ be defined as n data players and participate in the training model M; their local dataset is $D=\{D_1,D_2,\dots ,D_n\}$. $M_{FED}$ denotes the shared model that FL requires players to train together, and $M_{SUM}$ denotes the traditional machine learning model, which puts all data together to train the model. $V_{FED}$ and $V_{SUM}$ are the model accuracy of $M_{FED}$ and $M_{SUM}$, respectively, if there exists a positive number $\delta \ge 0$ which satisfies

$$|V_{FED}-V_{SUM}|<\delta ,$$

(1)

we say that the FL algorithm has $\delta$-accuracy loss [4].

The framework diagram of FL is shown in Figure 1, and the training steps in the FL model are as follows:

Step 1: Local federated players download the initialized global model from the aggregation server;

Step 2: Each federated player trains the local model with the initializing global model;

Step 3: After training the local model, the updated model and parameters are uploaded to the aggregation server;

Step 4: The aggregation server aggregates the models and parameters uploaded by each federated player for the next update round.

The commonly used aggregation method is the federated averaging (FedAvg) algorithm [27], Steps 2 and 3 are repeated until the local model converges.

3.2. Cooperative Games

Let $G(N,v)$ be a defined cooperative game, satisfying the following conditions [28]:

$$v\left(S_1\right)+v\left(S_2\right)\le v(S_1\cup S_2),$$

(2)

$$S_1\cap S_2=⌀,v(⌀)=0.$$

(3)

where N is a finite set of players, $S_1,S_2\in 2^N,v:2^N$$\to \mathbb{R}$ is a game-characteristic function, and $2^N$ is the set of all the subsets of N. Let $v\left(S\right)$ be the players’ payoff function, $v\left(N\right)$ indicate the coalition payoff, and ${\phi }_i\left(v\right)$ be the payoff of player i in $v\left(N\right)$, which satisfy two constraints:

$$v\left(N\right)=\sum {\phi }_i\left(v\right)\mathrm{and}v\left(i\right)\le {\phi }_i\left(v\right),\forall i\in N,i=1,2,\dots ,n,$$

(4)

$$v\left(S\right)\le \sum _{i\in S}{\phi }_i\left(v\right),\forall S\subseteq N,S\ne ⌀.$$

(5)

Formulas (4) and (5) are called individual rationality and coalition rationality, respectively.

3.3. Shapley Value

The Shapley value was proposed in the cooperative game theory [28], which can effectively solve the problem of cooperative payoff allocation and is defined as

$${\phi }_i\left(v\right)=\sum _{i\in S,S\subseteq N}w\left(\right|S\left|\right)\left[v\left(S\right)-v(S\setminus i)\right],$$

(6)

$$w\left(\right|S\left|\right)=\frac{(n-|S\left|\right)!\left(\right|S|-1)!}{n!}.$$

(7)

where $S\subseteq N$, $i\in S$, $i=1,2,\dots ,n$, $\left|S\right|$ is the number of players in subset S, $w\left(\right|S\left|\right)$ is the weight coefficient, $v\left(S\right)$ is the profit of subset $\left|S\right|$ and satisfies the conditions (2) and (3), the expression $v\left(S\right)-v(S\setminus i)$ assesses the marginal contribution of i to the coalition S, and $v(S\setminus i)$ indicates the payoff of the other players in the subset $\left|S\right|$ other than i.

3.4. Pareto Optimality

Let $\pi =\left({\pi }_1,{\pi }_2,\dots ,{\pi }_n\right):x\to {\mathbb{R}}^n$ be the vector of the players’ payoff function, and x be a feasible action space for two actions $x_1$ and $x_2\in x$ [29]:

(1)

If $\forall i\in \left[n\right]:{\pi }_i\left(x_1\right)\ge {\pi }_i\left(x_2\right)$, then $x_1$ weakly dominates $x_2$ and is marked by $x_1⪰x_2$;

(2)

If $x_1⪰x_2$ and $\exists i\in \left[n\right]:{\pi }_i\left(x_1\right)>{\pi }_i\left(x_2\right)$, then $x_1$ dominates $x_2$ and is marked by $x_1\succ x_2$.

If no other action in x dominates it and the collection of the players’ action vectors of all Pareto optimality actions is the Pareto front, then an action is called the Pareto optimality [29]. In other words, an allocation is considered to be Pareto optimality if no alternative allocation could make someone better off without making someone else worse off [30].

3.5. Nash Equilibrium

We consider a game with n players, and the set of players is denoted as $N=\{1,2,\dots ,n\}$. The player i’s payoff function is ${\pi }_i\left(x\right)$, where $x={\left[x_1,x_2,\dots ,x_n\right]}^T\in$${\mathbb{R}}^n$ is the vector of the player’s actions, and $x_i\in \mathbb{R}$ is the action of player i. If player j is not a neighbor of player i, then player i has no direct access to player j’s action.

Nash equilibrium is an action profile on which no player can gain more payoff by unilaterally changing its action, i.e., an action profile $x^{*}=\left(x_i^{*},x_{-i}^{*}\right)$ is the Nash equilibrium [31,32] if

$${\pi }_i\left(x_i^{*},x_{-i}^{*}\right)\ge {\pi }_i\left(x_i,x_{-i}^{*}\right),\forall i\in N$$

where $x_{-i}={\left[x_1,x_2,\dots ,x_{i-1},x_{i+1},\dots ,x_n\right]}^T$. Note that ${\pi }_i\left(x\right)$ and $\pi x$ might alternatively be written as ${\pi }_i\left(x_i,x_{-i}\right)$ and $\left(x_i,x_{-i}\right)$, respectively, in this paper.

4. Federated Learning Incentive Mechanism

4.1. The FL Incentive Model

We make the following assumptions before setting up the FL incentive model:

(1)

All players can pay for FL, and in the payoff distribution process, they adopt the best payoff distribution scheme.

(2)

All players are satisfied with the final distribution of payoffs, as all players were willing to join the coalition.

(3)

All players are entirely trustworthy and have no cheating in the FL.

(4)

To ensure the smooth implementation of the strategy, the FL should adopt a multi-party agreement to accept the payoff distribution plan.

According to the idea of FL, we establish the FL incentive model in Figure 2, and the main steps of the FL incentive mechanism are as follows:

Step 1: Assume that there are n players for federated model training, and each player has its local dataset $D_i$.

Step 2: Each player downloads the initialized model from the aggregation server, trains the model using its local dataset $D_i$, and uploads the trained model $m_i$ to the federated aggregation server.

Step 3: The federated aggregation server collects the model parameters $m_i$ uploaded by all players and uses the federated aggregation algorithm (FedAvg) to aggregate these parameters to obtain a new global model.

Step 4: The contribution of each player to the global model is calculated using Shapley values or other methods. These contribution values are used to determine the distribution of the player’s payoffs.

Step 5: The supervising organization determines whether each player’s payoff is Pareto optimal, and if it is, the federated payoff is distributed using the Shapley value formula; otherwise, the player receives a penalty from the supervising organization.

Step 6: According to the gain allocation formula, rewards are issued to each player who achieves Pareto optimality.

4.2. The Conflict between Fairness and Pareto Optimality

All players are allowed to actively contribute to the FL to guarantee that each player is happy with the federated payoff allocation method and to make the allocation process motivating. The Shapley value considers each player’s contribution to be $1/n$ and ignores the conflict between the fairness of Shapley’s value and Pareto optimality.

Assume there are n players, and the coalition input of player i is $x_i$ and satisfies $x_i\in (0,\infty ),$$i=1,2,\dots ,n$. The coalition inputs of all players form an n-dimensional vector $x=(x_1,x_2,\dots ,x_n)$. The coalition cost input of player i is $c_i\left(x_i\right)$ and is a differentiable convex function that is strictly monotonically increasing, which satisfies $\frac{\partial c_i}{x_i}>0$, $\frac{{\partial }^2{c}_i}{\partial x_i^2}>0$, and $c_i\left(0\right)=0$. The federated payoff $v(x_1,x_2,\dots ,x_n)$ determined by the federated input of n players is a strictly monotonically increasing differentiable concave function, which satisfies $\frac{\partial v_i}{x_i}>0$, $\frac{{\partial }^2{v}_i}{\partial x_i^2}<0$, and $v(0,0,\dots ,0)=0$. The federated payoffs of n players are distributed according to Formulas (6) and (7).

Although the Shapley value method is fair for the federated payoff allocation, the phenomenon of the free-riding of federated players cannot be avoided. That is, the Pareto optimization before payoff allocation is not satisfied. Therefore, we obtain the following theorem:

Theorem 1.

The Shapley value method satisfies the fairness payoff allocation after FL but not the optimality incentive of federated players’ inputs before FL, i.e., it does not achieve the Pareto efficiency optimization before FL.

Proof. 

The proof is given in Appendix A.1 of the Appendix A. □

4.3. FL Incentive Mechanism via Introducing Supervisory Organization

4.3.1. The Establishment of Supervisory Organization Mechanism

In [33], Alchian and Demsetz argued that introducing supervisory organizations would address free riding in the FL process. To encourage supervisor initiative, federated members must pay a certain fee to the supervisor. In [12], Holmstrom pointed out that the phenomenon of free riding can be addressed by using an incentive mechanism. The supervisor’s main task is to break the equilibrium and create incentives.

If the supervisor knows that the federated payoff is greater than or equal to the Pareto optimality payoff, and the supervisor distributes this payoff to the players according to Formulas (6) and (7), then if the federated payoff is less than the Pareto optimality value, the federated player must pay a fee $k_i$ as in the following:

$$r_i\left(x\right)=\left\{\begin{array}{c}{\phi }_i\left(v\right),ifv\ge v\left(x^{*}\right)\\ {\phi }_i\left(v\right)-k_i,ifv<v\left(x^{*}\right)\end{array}\right.$$

(8)

where $x^{*}=(x_1^{*},x_2^{*},\dots ,x_n^{*})$ is the federated input vector satisfying Formula (A4).

4.3.2. Penalty Conditions

Theorem 2.

When the mechanism of federated input $x^{*}=(x_1^{*},x_2^{*},\dots ,x_n^{*})$ that satisfies Pareto optimality is a Nash equilibrium, the penalty $k_i$ must satisfy the two conditions as follows:

(1)

If the independent input $x_i$ of the player i is less than the Pareto optimality federated input $x_i^{*}$, i.e., e.g., $x_i<x_i^{*}$, and $v\left(x\right)$ is monotonically increasing, i.e., $v(x_i,x_{n-i}^{*})<v(x_i^{*},x_{n-i}^{*})$, then the player i is fined, and the payoff remaining after the fine is $r_i\left[v(x_i,x_{n-i}^{*})\right]={\phi }_i\left(v\right)-k_i$, and finally the profit of the player i is

$${\pi }_i\left[v(x_i,x_{n-i}^{*})\right]={\phi }_i\left(v\right)-k_i-c_i\left(x_i\right),i=1,2,\dots ,n.$$

(9)

(2)

If the independent input $x_i$ of the player i is equal to the Pareto optimality federated input $x_i^{*}$, i.e.,$x_i=x_i^{*}$, then $v=v\left(x^{*}\right)$, and the payoff remaining after the penalty is $r_i\left[v(x_i^{*},x_{n-i}^{*})\right]={\phi }_i\left(v^{*}\right)$, and finally the profit of player i is

$${\pi }_i\left[v(x_i^{*},x_{n-i}^{*})\right]={\phi }_i\left(v^{*}\right)-c_i\left(x_i^{*}\right),i=1,2,\dots ,n.$$

(10)

where $x_{n-i}^{*}=(x_1^{*},\dots x_{i-1}^{*},x_{i+1}^{*},\dots ,x_n^{*})$ represents the Pareto optimality federated input vector composed of $n-i$ players.

Proof. 

The proof is given in Appendix A.2 of the Appendix A. □

5. Numerical Examples and Simulation Experiments

In this section, we will use two examples to verify the rationality of the above discussion. In example 1, we consider that the status of the federated players is equal, and in example 2, we consider that the status of the federated players is unequal.

5.1. Numerical Example 1: Equal Status of Federated Players

Assuming there are three players in the FL system, and the federated payoff function is $v(x_1,x_2,x_3)=x_1+x_2+x_3+x_1{x}_2+x_1{x}_3+x_2{x}_3$, and the return function $v(x_1,x_2,x_3)$ is a strictly monotonically increasing concave function, the cost functions of the three players are $c_1\left(x_1\right)=\frac{3}{2}{x}_1^2$, $c_2\left(x_2\right)=\frac{3}{2}{x}_2^2$, and $c_3\left(x_3\right)=\frac{3}{2}{x}_3^2$. It is easy to know that the cost function $c_i\left(x_i\right)$ is a strictly increasing convex function. When the federated input is $x^{*}=(x_1^{*},x_2^{*},x_3^{*})$, the federated profit

$$maxR=v(x_1,x_2,x_3)-\sum _{i=1}^3{c}_i\left(x_i\right)$$

maximizes and satisfies Pareto optimality, and its first-order condition is

$$\left\{\begin{array}{c}1+x_2^{*}+x_3^{*}-3x_1^{*}=0\hfill \\ 1+x_1^{*}+x_3^{*}-3x_2^{*}=0\hfill \\ 1+x_1^{*}+x_2^{*}-3x_3^{*}=0\hfill \end{array}\right.$$

We determine that the federated inputs satisfying the Pareto optimality conditions are $x_1^{*}=1$, $x_2^{*}=1$, and $x_3^{*}=1$, the federated profit is $v\left(x^{*}\right)=6$, and maximum profit is $max{R}^{*}=1.5$. Because of the equal status of the three players, according to the anonymity of Formulas (6) and (7), the effectiveness of Formulas (6) and (7) can be obtained as

$$\begin{array}{c}\hfill {\phi }_1\left(v\left(x\right)\right)={\phi }_2\left(v\left(x\right)\right)={\phi }_3\left(v\left(x\right)\right)\\ \hfill {\phi }_1\left(v\left(x\right)\right)+{\phi }_2\left(v\left(x\right)\right)+{\phi }_3\left(v\left(x\right)\right)=v\left(x\right)\\ \hfill {\phi }_1\left(v\left(x\right)\right)={\phi }_2\left(v\left(x\right)\right)={\phi }_3\left(v\left(x\right)\right)=\frac{1}{3}v\left(x\right).\end{array}$$

Therefore, the profit functions of the three players are

$$\begin{array}{cc}& {\pi }_1(\frac{1}{3}v,x_1)=\frac{1}{3}v\left(x\right)-\frac{3}{2}{x}_1^2\hfill \\ & {\pi }_2(\frac{1}{3}v,x_2)=\frac{1}{3}v\left(x\right)-\frac{3}{2}{x}_2^2\hfill \\ & {\pi }_3(\frac{1}{3}v,x_3)=\frac{1}{3}v\left(x\right)-\frac{3}{2}{x}_3^2.\hfill \end{array}$$

The Nash equilibrium requires other players to decide their investment in the FL, and each player has the right to decide their investment to maximize their profits. Therefore, the first-order condition that satisfies the Nash equilibrium is

$$\left\{\begin{array}{c}1+x_2+x_3-9x_1=0\hfill \\ 1+x_1+x_3-9x_2=0\hfill \\ 1+x_1+x_2-9x_3=0\hfill \end{array}\right.$$

We determine that the federated input satisfying the Nash equilibrium conditions are ${\tilde{x}}_1=0.14$, ${\tilde{x}}_2=0.14$, and ${\tilde{x}}_3=0.14$, the federated payoff is $v\left(\tilde{x}\right)=0.49$ and the maximum profit is $max\tilde{R}=0.4$.

By comparing the Pareto optimality solution and Nash equilibrium solution in

Table 1. Example 1: Federated input and profit comparison.

Input and Profit Comparison	Input ${\mathit{x}}_1$	Input ${\mathit{x}}_2$	Input ${\mathit{x}}_3$	Federated Profit	Maximum Profit
Pareto optimality	1	1	1	6	1.5
Nash equilibrium	0.14	0.14	0.14	0.49	0.4

, we know that using the Shapley value method can achieve fairness post-FL; the optimality incentive is not reached before FL. Under the Nash equilibrium, the input of each player is lower than the Pareto optimality level, and the federated profit cannot reach the maximum.

Next, we introduce the supervisory authority. If the supervisory authority knows that the federated payoff is greater than or equal to Pareto optimality payoff 6, the payoff is allocated to the federated players by Formulas (6) and (7). If it knows that the federated payoff is lower than Pareto optimality payoff 6, the payoff of the federated players is 0. The specific expression is as follows:

$${\phi }_i\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{1}{3}v\left(x\right),& ifv\left(x\right)\ge 6,\\ 0,& ifv\left(x\right)<6.\end{array}\right.$$

Here, we prove that the Nash equilibrium constituting the supervision mechanism satisfies the Pareto optimality condition $x^{*}=(x_1^{*},x_2^{*},x_3^{*})=(1,1,1)$. If the Pareto optimality values of the players 2 and 3 are $x_1^{*}=1$ and $x_2^{*}=1$, respectively, the federated payoff is $v\left(x\right)=3+3x_3$. If the input of player 3 is $x_3<1$, there is $v\left(x\right)<6$, at this time,

$$\begin{array}{c}\hfill {\phi }_3\left(v\left(x\right)\right)=0,\\ \hfill {\pi }_3(0,x_3)=0-\frac{3}{2}{x}_3^2<0,\end{array}$$

so rational player 3 will not input $x_3<1$. If the federated input of player 3 is $x_3>1$, then $v\left(x\right)>6$ and

$$\begin{array}{c}\hfill {\phi }_3\left(v\left(x\right)\right)=\frac{1}{3}v\left(x\right)=\frac{4}{3}+\frac{2}{3}{x}_3,\\ \hfill {\pi }_3\left(x_3\right)=\frac{4}{3}+\frac{2}{3}{x}_3-\frac{3}{2}{x}_3^2.\end{array}$$

when $x_3\ge 1$, then $\frac{\partial {\pi }_3\left(x_3\right)}{\partial x_3}=\frac{2}{3}-3x_3<0$, obviously, ${\pi }_3({\phi }_3,x_3)$ is a monotonic decreasing function of $x_3$ on interval $[1,\infty )$, and the profit of player 3 reaches the maximum at $x_3=1$. Therefore, the Nash equilibrium constituting the supervision mechanism satisfies the Pareto optimality condition $x^{*}=(x_1^{*},x_2^{*},x_3^{*})=(1,1,1)$.

Next, we consider the minimum value of penalty mechanism $k_i={\phi }_i\left(x\right)-{\pi }_i\left(x^{*}\right)$. If the supervisory authority knows that the federated payoff is greater than or equal to Pareto optimality payoff 6, the payoff is allocated to the federated players by Formulas (6) and (7). If it knows that the federated payoff is lower than Pareto optimality payoff 6, the payoff of the federated players is 0.5. The specific expression is as follows:

$${\phi }_i\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{1}{3}v\left(x\right),& ifv\left(x\right)\ge 6,\\ 0.5,& ifv\left(x\right)<6.\end{array}\right.$$

Here, we prove that the Nash equilibrium constituting the supervision mechanism satisfies the Pareto optimality condition $x^{*}=(x_1^{*},x_2^{*},x_3^{*})=(1,1,1)$. If the Pareto optimality values of players 2 and 3 are $x_1^{*}=1$ and $x_2^{*}=1$, respectively, the federated payoff is $v\left(x\right)=3+3x_3$. If the input of player 3 is $x_3<1$, there is $v\left(x\right)<6$. At this time,

$$\begin{array}{c}\hfill {\phi }_3\left(v\left(x\right)\right)=0.5,\\ \hfill {\pi }_3(0,x_3)=0.5-\frac{3}{2}{x}_3^2\le 0.5,\end{array}$$

so rational player 3 will not input $x_3<1$. If the federated input of player 3 is $x_3\ge 1$, then $v\left(x\right)\ge 6$ and

$$\begin{array}{c}\hfill {\phi }_3\left(v\left(x\right)\right)=\frac{1}{3}v\left(x\right)=\frac{4}{3}+\frac{2}{3}{x}_3,\\ \hfill {\pi }_3\left(x_3\right)=\frac{4}{3}+\frac{2}{3}{x}_3-\frac{3}{2}{x}_3^2.\end{array}$$

In the last part, we proved that ${\pi }_3({\phi }_3,x_3)$ is a monotonic decreasing function of $x_3$ on interval $[1,\infty )$, and the profit of player 3 reaches the maximum at $x_3=1$.

According to the above, when $x_1^{*}=1$, $x_2^{*}=1$, and the input of player 3 is $x_3^{*}=1$, it just reaches the Pareto optimality value. At this time, ${\pi }_3\left(x_3^{*}\right)=0.5$. Therefore, the condition to reach the Nash equilibrium of the mechanism is that the Pareto optimality input is $x^{*}=(x_1^{*},x_2^{*},x_3^{*})=(1,1,1)$.

5.2. Numerical Example 2: Unequal Status of Federated Players

Assuming that there are three federated players that form a coalition, and the federated payoff function is

$$v(x_1,x_2,x_3)=2x_1+4x_2+6x_3+x_1{x}_2.$$

The payoff function $v(x_1,x_2,x_3)$ is a strictly increasing linear function, and the cost functions $c\left(x_1\right)=\frac{1}{2}{x}_1^2$, $c\left(x_2\right)=x_2^2$ and $c\left(x_3\right)=\frac{1}{4}{x}_3^2$ of these three players are strictly monotonically increasing convex functions. When the federated input $x^{*}=(x_1^{*},x_2^{*},x_3^{*})$, the federated profit

$$maxR=v(x_1,x_2,x_3)-\sum _{i=1}^3{c}_i\left(x_i\right)=2x_1+4x_2+6x_3+x_1{x}_2-\frac{1}{2}{x}_1^2-x_2^2-\frac{1}{4}{x}_3^2$$

maximizes and satisfies Pareto optimality, its first-order condition is

$$\left\{\begin{array}{c}2-x_1^{*}+x_2^{*}=0\hfill \\ 4+x_1^{*}-2x_2^{*}=0\hfill \\ 6-\frac{1}{2}{x}_3^{*}=0.\hfill \end{array}\right.$$

we determine that the federated inputs satisfying the Pareto optimality conditions are $x_1^{*}=8$, $x_2^{*}=6$, and $x_3^{*}=12$, the federated payoff is $v\left(x^{*}\right)=160$, and the maximum profit is $max{R}^{*}=56$. According to Formulas (6) and (7), because $v\left(0\right)=0$, $v\left(x_1\right)=2x_1$, $v\left(x_2\right)=4x_2$, $v\left(x_3\right)=6x_3$, $v(x_1,x_2)=x_1{x}_2$, $v(x_1,x_3)=v(x_2,x_3)=0$, and $v(x_1,x_2,x_3)=2x_1+4x_2+6x_3+x_1{x}_2$, then

$$\begin{array}{l}{\phi }_1\left(v\left(x\right)\right)=\frac{4}{3}{x}_1+\frac{2}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2\hfill \\ {\phi }_2\left(v\left(x\right)\right)=\frac{1}{3}{x}_1+\frac{8}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2\hfill \\ {\phi }_3\left(v\left(x\right)\right)=\frac{1}{3}{x}_1+\frac{2}{3}{x}_2+4x_3.\hfill \end{array}$$

Therefore, the profit functions of the three players are

$$\begin{array}{cc}& {\pi }_1(v,x_1)=\frac{4}{3}{x}_1+\frac{2}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2-\frac{1}{2}{x}_1^2\hfill \\ & {\pi }_2(v,x_2)=\frac{1}{3}{x}_1+\frac{8}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2-x_2^2\hfill \\ & {\pi }_3(v,x_3)=\frac{1}{3}{x}_1+\frac{2}{3}{x}_2+4x_3-\frac{1}{4}{x}_3^2.\hfill \end{array}$$

Nash equilibrium requires other players to decide their investment in the FL, and each player has the right to decide their investment, to maximize their profits. Therefore, the first-order condition satisfying Nash equilibrium is

$$\left\{\begin{array}{c}\frac{4}{3}-x_1+\frac{1}{2}{x}_2=0\hfill \\ \frac{8}{3}+\frac{1}{2}{x}_1-2x_2=0\hfill \\ 4-\frac{1}{2}{x}_3=0.\hfill \end{array}\right.$$

we determine that the federated inputs satisfying the Pareto optimality conditions are $x_1=2.29$, $x_2=1.90$, and $x_3=8$, the federated profit is $v\left(x\right)=64.53$, and the maximum profit is $maxR=42.30$.

Comparing Pareto optimality and the Nash equilibrium solution in

Table 2. Example 2: Federated input and profit comparison.

Input and Profit Comparison	Input ${\mathit{x}}_1$	Input ${\mathit{x}}_2$	Input ${\mathit{x}}_3$	Federated Profit	Maximum Profit
Pareto optimality	8	6	12	160	56
Nash equilibrium	2.19	1.90	8	64.53	42.30

, we know that using the Shapley value method can achieve post fairness, but there is no prior optimality incentive. Under the Nash equilibrium, the input of each player is lower than the Pareto optimality level, and the federated profit does not reach the maximum.

Next, we introduce the supervisory authority. If the supervisory authority knows that the federated payoff is greater than or equal to the Pareto optimality payoff 160, the payoff is allocated to the federated players by Formulas (6) and (7). If it knows that the federated payoff is lower than Pareto optimality payoff 160, the federated player payoff is ${\tau }_i$. When the input of players reaches the Pareto optimality value, i.e., $x_1^{*}=8$, $x_2^{*}=6$, and $x_3^{*}=12$, then

$$\begin{array}{ccc}\hfill {\phi }_1(v\left(x^{*}\right)& =& \frac{4}{3}{x}_1^{*}+\frac{2}{3}{x}_2^{*}+x_3^{*}+\frac{1}{2}{x}_1^{*}{x}_2^{*}=50.67\hfill \\ \hfill {\phi }_2(v\left(x^{*}\right)& =& \frac{1}{3}{x}_1^{*}+\frac{8}{3}{x}_2^{*}+x_3^{*}+\frac{1}{2}{x}_1^{*}{x}_2^{*}=54.67\hfill \\ \hfill {\phi }_3(v\left(x^{*}\right)& =& \frac{1}{3}{x}_1^{*}+\frac{2}{3}{x}_2^{*}+4x_3^{*}=54.67\hfill \\ \hfill {\pi }_1\left(x^{*}\right)& =& 18.67\hfill \\ \hfill {\pi }_2\left(x^{*}\right)& =& 18.67\hfill \\ \hfill {\pi }_3\left(x^{*}\right)& =& 18.67.\hfill \end{array}$$

Therefore, the value ranges of ${\tau }_1$, ${\tau }_2$ and ${\tau }_3$ are $0\le {\tau }_1\le 18.67$, $0\le {\tau }_2\le 18.67$ and $0\le {\tau }_3\le 18.67$, respectively. The payoffs ${\phi }_1\left(v\left(x\right)\right)$, ${\phi }_2\left(v\left(x\right)\right)$ and ${\phi }_3\left(v\left(x\right)\right)$ of players 1, 2 and 3 are as follows:

$${\phi }_1\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{4}{3}{x}_1+\frac{2}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2,& ifv\left(x\right)\ge 160\\ {\tau }_1,& ifv\left(x\right)<160\end{array}\right.$$

$${\phi }_2\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{1}{3}{x}_1+\frac{8}{3}{x}_2+x_3+\frac{1}{2}{x}_1{x}_2,& ifv\left(x\right)\ge 160\\ {\tau }_2,& ifv\left(x\right)<160\end{array}\right.$$

$${\phi }_3\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{1}{3}{x}_1+\frac{2}{3}{x}_2+4x_3,& ifv\left(x\right)\ge 160\\ {\tau }_3,& ifv\left(x\right)<160.\end{array}\right.$$

In further work, we will prove that the Nash equilibrium constituting the supervision mechanism satisfies the Pareto optimality condition $x^{*}=(x_1^{*},x_2^{*},x_3^{*})=(8,6,12)$.

(1) We consider the value of player 1. When $x_2^{*}=6$ and $x_3^{*}=12$, then

$${\phi }_1\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{13}{3}{x}_1+16,& ifv\left(x\right)\ge 8\\ {\tau }_1,& ifv\left(x\right)<8.\end{array}\right.$$

where $0\le {\tau }_1\le 18.67$, and if the input of player 1 is $x_1\ge 8$, there is $v_1\left(x\right)=\frac{13}{3}{x}_1+16$, ${\pi }_1\left(x\right)=\frac{13}{3}{x}_1+16-\frac{1}{2}{x}_1^2$, then $\frac{\partial {\pi }_1\left(x_1\right)}{\partial x_1}=\frac{13}{3}-x_1<0$. It indicates that the profit function ${\pi }_1\left(x\right)$ of player 1 is monotonically decreasing on $[8,\infty )$; therefore, when player 1 invests $x_1=8$, it can obtain the maximum profit ${\pi }_1\left(x\right)=18.67$. If player 1 invests $x_1<8$, the player payoff is ${\tau }_1$ and profit is ${\pi }_1\left(x\right)={\tau }_1-\frac{1}{2}{x}_1^2$, then ${\pi }_1\left(x\right)={\tau }_1-\frac{1}{2}{x}_1^2\le 18.67$. Thus, when $x_2^{*}=6$ and $x_3^{*}=12$, player 1 can obtain the maximum profit ${\pi }_1\left(x\right)=18.67$ by investing $x_1^{*}=8$.

(2) We consider the value of player 2. When $x_1^{*}=8$ and $x_3^{*}=12$, then

$${\phi }_2\left(v\left(x\right)\right)=\left\{\begin{array}{cc}\frac{20}{3}{x}_2+\frac{44}{3},& ifv\left(x\right)\ge 6\\ {\tau }_2,& ifv\left(x\right)<6.\end{array}\right.$$

where $0\le {\tau }_2\le 18.67$, and if the input of player 2 is $x_2\ge 6$, there is $v_2\left(x\right)=\frac{20}{3}{x}_2+\frac{44}{3}$ and ${\pi }_2\left(x\right)=\frac{20}{3}{x}_2+\frac{44}{3}-x_2^2$, then $\frac{\partial {\pi }_2\left(x_2\right)}{\partial x_2}=\frac{20}{3}-2x_2<0$. It indicates that the profit function ${\pi }_2\left(x\right)$ of player 2 is monotonically decreasing on $[6,\infty )$; therefore, when player 2 invests $x_2=6$, it can obtain the maximum profit ${\pi }_2\left(x\right)=18.67$. If player 2 invests $x_1<6$, the player payoff is ${\tau }_2$ and profit is ${\pi }_2\left(x\right)={\tau }_2-x_2^2$, then ${\pi }_2\left(x\right)={\tau }_2-x_2^2\le 18.67$. Thus, when $x_1^{*}=8$ and $x_3^{*}=12$, player 2 can obtain the maximum profit ${\pi }_2\left(x\right)=18.67$ by investing $x_2=6$.

(3) We consider the value of player 3. When $x_1^{*}=8$ and $x_2^{*}=6$, then

$${\phi }_3\left(v\left(x\right)\right)=\left\{\begin{array}{cc}4x_3+\frac{20}{3},& ifv\left(x\right)\ge 12\\ {\tau }_3,& ifv\left(x\right)<12.\end{array}\right.$$

where $0\le {\tau }_3\le 18.67$, and if the input of player 3 is $x_3\ge 12$, there is $v_3\left(x\right)=4x_3+\frac{20}{3}$, ${\pi }_3\left(x\right)=4x_3+\frac{20}{3}-\frac{1}{4}{x}_3^2$, then $\frac{\partial {\pi }_3\left(x_3\right)}{\partial x_3}=4-\frac{1}{2}{x}_3<0$. It indicates that the profit function ${\pi }_3\left(x\right)$ of player 3 is monotonically decreasing on $[12,\infty )$; therefore, when player 3 invests $x_3=12$, it can obtain the maximum profit ${\pi }_3\left(x\right)=18.67$. If player 1 invests $x_1<12$, player payoff is ${\tau }_3$ and profit is ${\pi }_3\left(x\right)={\tau }_3-\frac{1}{4}{x}_3^2$, then ${\pi }_3\left(x\right)={\tau }_3-\frac{1}{4}{x}_3^2\le 12$. Thus, when $x_1^{*}=8$ and $x_2^{*}=6$, player 3 can obtain the maximum profit ${\pi }_3\left(x\right)=18.67$ by investing $x_3=12$.

5.3. Numerical Simulation Experiments

Figure 3 shows the simulations of numerical experiments 1 and 2, respectively. Subplot (a) and subplot (c) in Figure 3 show that the optimality inputs of the federated players have reached the Pareto optimality, and satisfying the Nash equilibrium’s inputs does not reach the Pareto optimality. Subplots (b) and (d) show that the payoffs of satisfying the Pareto optimality inputs are more than the federal payoffs of satisfying the Nash equilibrium inputs.

As in the literature [17,19], although the Shapley value method can satisfy the fair distribution of payoffs after FL, it cannot achieve Pareto efficiency optimality before FL, i.e., it cannot reach the optimality incentives for federated player inputs before FL, and neither individual nor collective maximum payoffs are achieved.

Therefore, according to the supervisory organization introduced in this paper, all federated players are observed to have lower federated payoffs than those satisfying the Pareto optimality, and the penalty goes to the supervisory organization. In this way, the supervisory organization supervises and disciplines all federated players.

By analyzing and proving that the inputs that satisfy the Pareto efficiency optimality constitute a Nash equilibrium for FL through the introduced supervisory organization, it is possible to solve the conflict that the payoff allocation of FL is fair and efficient.

6. Conclusions and Future Work

In the model training of FL, to obtain an accurate federated model, this paper designs an incentive mechanism to encourage all federated players to contribute to their data-training model. Under the condition of payoff determination, combined with the Shapley value method, a federated payoff allocation mechanism with third-party supervision is introduced. Under this mechanism, the federated payoff can reach Pareto optimality, and finally, the federated payoff is allocated by the Shapley value method. This mechanism solves the conflict between the fairness and efficiency of the payoff allocation in the FL system. Through the verification of numerical and simulation experiments, when the optimality payoff allocation of Pareto optimality is achieved, the Nash equilibrium of the mechanism is formed. Therefore, the use of an incentive mechanism will play a better role for federated players.

In future research, we apply the incentive mechanism solution proposed in this paper to solve the problem of payoff allocation among players in the training scenario of the FL model. In particular, the incentive mechanism proposed in this paper can be applied to banks, hospitals, insurance companies, etc., providing important theoretical assistance for them to train accurate federated models and improve economic efficiency in practice.

7. Discussion

The Shapley value method is a way to measure how much each player contributes to FL, and it is a fair rule for allocating resources. Pareto efficiency is a resource allocation’s ideal state, in which all resources are in full use and there is no waste. Although the Shapley value method can achieve fair payoff allocation after the completion of FL, it cannot guarantee that the inputs of each federated player achieve optimality before FL.

Therefore, the combination of the Shapley value and Pareto optimality provides a solution for federated payoff distribution that is both fair and efficient, and the solution can help ensure the stability and dynamic equilibrium of the payoff distribution.

However, this combination may have some shortcomings. For example, calculating the Shapley values in federated model training can be very complicated in the case of a large number of players. Furthermore, to determine the distribution of Pareto efficiency, the utility functions of all federated payers must be provided, which may be difficult to implement in practical applications.