Towards Reliable Federated Learning Using Blockchain-Based Reverse Auctions and Reputation Incentives

Abstract

In recent years, the explosion of big data has presented unparalleled opportunities for the advancement of machine learning (ML). However, the vast size and sensitive nature of these datasets present significant challenges in terms of privacy and security. Federated Learning has emerged as a promising solution that enables a group of participants to train ML models without compromising the confidentiality of their raw data. Despite its potential, traditional federated learning faces challenges such as the absence of participant incentives and audit mechanisms. Furthermore, these challenges become more significant when dealing with the scale and diversity of big data, making efficient and reliable federated learning a complex task. These limitations may compromise model quality due to potential malicious nodes. To address the above issues, this paper proposes a BlockChain-based Decentralized Federated Learning (BCD-FL) model. In BCD-FL, we design a smart contract approach based on the reverse auction-based incentive mechanism and a reputation mechanism to promote the participation of reliable and high-quality data owners. Theoretical analysis shows that the BCD-FL model satisfies several desirable properties, such as individual rationality, computational efficiency, budget balance, and truthfulness. In addition, experimental results also show that the proposed model enables more efficient federated learning and provides some level of protection against malicious nodes. Therefore, the BCD-FL model presents a potential solution to the challenges in federated learning and opens up new possibilities for achieving efficient large-scale machine learning.

1. Introduction

Nowadays, artificial neural network-based machine learning techniques [1] have become a driving force across various industries, yielding significant advancements in areas such as face recognition [2], autonomous driving [3], natural language processing (NLP) [4], and medical analysis [5]. Despite these advancements, the methodologies used in these studies often rely heavily on the accumulation of vast data volumes on a centralized server to build high-quality models. This approach not only presents significant computational challenges but also raises serious privacy concerns. Recent research involving big datasets [6,7] has highlighted critical issues associated with this traditional approach. The requirement for extensive data centralization can lead to privacy breaches, and the sensitive nature of the data often makes users hesitant to share it. This reluctance contributes to the creation of data silos [8], which in turn hampers the development of optimal machine learning models. To address these privacy-related obstacles inherent in traditional machine learning methods, Federated Learning (FL) has emerged as an innovative and promising paradigm [9].

When compared to more traditional machine learning models, FL does not need its users to transmit all the raw data to a centralized server for model creation. Instead, users only submit model parameters that have been trained on data from their own region. This strategy has the potential to successfully minimize breaches in privacy caused by data transfer, reduce expenses associated with transmission, and assure the model’s efficiency all at the same time. As a direct consequence, FL has received substantial attention and recognition from both professional and academic worlds [10]. Although FL has shown great potential, there are still some challenges that remain open. One challenge in FL is incentivizing participants to contribute their data while ensuring the long-term stability of the system. In traditional FL, participants are expected to provide their data with honesty and without any conditions. However, in practice, participants may be self-interested and unwilling to provide their data for free [11]. In addition, some of the participants may be malevolent and may transmit poisoned updates to the central server. This may result in the global model taking an excessive amount of time to converge or may cause the learning process to be suddenly terminated. In an effort to find a solution to this problem, a number of different pieces of research [12,13,14] have concentrated on the development of incentive mechanisms for FL. In [12], the concept of contract theory has been introduced into the FL environment and utilized as a stimulation mechanism. Other research initiatives, such as those detailed in [13,14], have used Stackelberg game theory and Deep Reinforcement Learning (DRL) to encourage participant engagement. This success can be attributed to the fact that reinforcement learning [15,16] has been proven to be a highly effective method in various fields of artificial intelligence. On the other hand, these solutions do not take into account the influence that the quality of model updates might have on the effectiveness of the learning process.

Although FL is effective in preventing raw data leakage, the central aggregator has the potential to be a single point of failure. This vulnerability might have an adverse effect on the resilience of the system [17]. The blockchain technology [18,19,20] could offer a solution by replacing the central aggregator with a distributed ledger system, which owns tamper-proof, non-repudiation, and traceability features [21]. Additionally, smart contracts [22] embedded in the blockchain can provide automated contract execution in which program codes and predefined rules will be executed automatically when specific conditions are met, further enhancing the security and trustworthiness of the system.

To improve the model quality and system robustness in FL, we propose a novel BlockChain-based Decentralized Federated Learning (BCD-FL) model, which takes advantage of the proposed reverse auction and reputation incentive mechanisms for transparent and fair execution of FL tasks. Local models that are offered by distributed data owners are able to be evaluated by the public inside BCD-FL. The use of smart contracts combined with reverse auctions enables reputable data owners with low bids to be picked in an open and transparent manner so that they may take part in FL assignments. At the same time, the reputation mechanism enables the evaluation of the contributed local models and indirectly enhances the reliability and data quality of the data owner. Our extensive analysis shows that the proposed approaches feature individual rationality, computational efficiency, budget balance, and truthfulness, which can provide benefits in terms of cost, accuracy, transparency, and fairness in FL. Our contributions are as follows:

• We propose a BlockChain-based Decentralized Federation Learning (BCD-FL) model, which can mitigate the single point of failure issue in traditional FL and create a secure and trustworthy system by using blockchain to ensure data privacy and fairness. In addition, we introduce the Interplanetary File System (IPFS) [23] as the storage medium for training models in BCD-FL to effectively reduce the storage cost.
• We design a smart contract incentive mechanism using the reverse auction technique to incentivize data owners to join and contribute high-quality models with a limited payment budget.
• To enhance the model quality in BCD-FL, we design a reputation mechanism based on the proposed marginal contribution model quality detection method, which can effectively translate the training model quality into the reputation of the data owner.
• Finally, we conduct extensive experiments on a public dataset. The experimental results show that the proposed approaches can effectively enhance the quality of FL model updates while maintaining promising individual rationality, computational efficiency, budget balance, and truthfulness.

The rest of the paper is organized as follows. Section 2 presents the related work. Section 3 discusses the preliminary work related to our work. Section 4 presents the proposed model. Section 5 introduces the smart contract incentive mechanism based on reverse auction and the reputation mechanism based on model quality evaluation. Section 6 presents the theoretical analysis. Section 7 provides the simulations of the proposed model and the experimental results. Finally, Section 8 concludes the paper.

2. Related Work

In this section, we provide the related work on Federated Learning, incentive mechanisms, and blockchain, as well as an overview of current trends and challenges in the field.

2.1. Federated Learning

Google first introduced the concept of FL in 2016, specifically for its application in input prediction [24] on mobile devices. Subsequently, both the academic and industrial sectors [25] started to extensively explore FL and its associated applications across various domains. In healthcare, for instance, Akter et al. [26] introduced a privacy-preserving framework based on FL to safeguard healthcare systems against privacy infringements at the edge. Similarly, Xing et al. [27] built a federated machine learning platform to enable sharing of medical data while protecting data privacy. For autonomous driving, Fu et al. [28] developed a federated reinforcement learning approach specifically designed to alleviate the issues of communication load and privacy concerns inherent in the transmission of raw data. Rjoub et al. [29] constructed a deep reinforcement learning framework leveraging FL to enhance reliability in edge computing. Within the realm of the Internet of Things (IoT), Yang et al. [30] proposed a hierarchical trust interaction structure that is federated and designed to facilitate cross-domain industrial IoT applications. This structure aims to ensure security and promote trustworthiness in multi-domain collaboration. Similarly, Peng et al. [31] introduced an FL framework customized for IoT applications in the textile industry that addresses non-independent and identically distributed data challenges. Although there efforts have been made in FL, these initiatives rely on the participation of volunteers. They lack effective incentives to ensure the completion of high-quality tasks. In the absence of such incentives, completing FL tasks to a high standard can prove challenging.

2.2. Incentive Mechanism

In FL networks, devising strategies that encourage user participation in tasks and foster high-quality data contribution presents a significant challenge. Numerous researchers have introduced incentive mechanisms for FL. The Shapley Value-based approach has been a major research hotspot for FL participant contribution evaluation in recent years. Introduced in 1953 as a solution concept in cooperative game theory, Shapley Value can fairly calculate the contribution of a participant in a coalition [32]. For example, Song et al. [33] developed a contribution index based on the Shapley value to evaluate the contributions of FL participants. Wang [34] utilized the Shapley value to determine the importance of data features. Concurrently, Agarwal et al. [35] and Jia et al. [36] applied an approximate Shapley value to data pricing and fair payoff distribution. Yang et al. [37] proposed a method based on Shapley value to decide the payment for each user by combining quality estimates with monetary incentives. However, among these methods, the computational complexity of the Shapley value increases exponentially with the number of participants.

An incentive mechanism based on the Stackelberg game was proposed by Pandey et al. [38] to enhance the communication efficiency of the global model. Similarly, Khan et al. [39] proposed the integration of Stackelberg games in the design of FL incentives for edge networks. Feng et al. [40] proposed a Stackelberg game model to analyze the interaction between servers and mobile devices in collaborative relay communication networks. In this model, mobile devices determine data prices to maximize profits. At the same time, servers choose to optimize profits by training data size. The goal of the model is to achieve a Nash equilibrium between the servers and mobile devices. In addition, contract theory has received significant attention due to its unique self-disclosure mechanism. Kang et al. [41] used contract theory to design incentive mechanisms that simulate mobile device participation in FL and minimize computational resources to increase utility. The mechanism gives higher rewards to data owners with more accurate local data. Lim et al. [42] used dynamic contract to design a two-stage incentive mechanism that satisfies cross-period incentive compatibility and is suitable for healthcare applications where global model training requires continuous participation of data owners. Ding et al. [43] designed an incentive mechanism based on multidimensional contract theory to maximize server revenue. The mechanism summarizes users’ multidimensional private information into a one-dimensional criterion, which allows users to perform complete classification. Despite these efforts, in such methodologies, the central aggregator might remain unaware of the participants’ costs, and the question of how to encourage distributed participants to contribute high-quality data/models continues to pose an unresolved challenge.

2.3. Blockchain-Based Federated Learning

Blockchain technology holds the potential for managing tasks within the realm of FL. Majeed and Hong [44] have presented an innovative blockchain-based FL approach known as FLchain. This scheme supports multiple global models for diverse tasks by utilizing channels within a decentralized ledger system. However, this scheme carries several limitations, including the absence of a validation mechanism for global models computed by miners and an insufficient incentive system. Similarly, Kim et al. [45] proposed BlockFL, a concept that integrates blockchain into FL to facilitate a decentralized training process. Notably, in BlockFL, the payment amount is proportional to the size of local data used by the user, which may be inflated by malicious nodes. Li et al. [46] also contributed to the discourse by presenting BFLC, a methodology aimed at the selection of credible participants in FL. The authors carefully detail the process of model storage on the blockchain, the training process, and an innovative committee consensus approach. However, a significant limitation of this method is its requirement for a distinct blockchain for every FL task.

In this work, we conduct a comprehensive analysis of the quality of contributed data, model aggregation, and reward distribution in FL. We proposed a novel model called BlockChain-based Decentralized Federated Learning (BCD-FL). This model leverages blockchain technologies to mitigate the single point of failure associated with the central aggregation server. Simultaneously, it employs off-chain storage to decrease storage costs and enhance block retrieval efficiency. Our proposed BCD-FL model is complemented by a smart contract and incentive strategies designed to significantly improve the quality of FL model updates. This structured approach serves to promote reliable and efficient decentralized FL, addressing the challenges and limitations inherent in the current models. Finally,

Table 1. Summary of related work on Federated Learning, incentive mechanisms, and blockchain.

Schemes	Main Contribution
Google [24]	Introduced FL for input prediction on mobile devices.
Akter et al. [26]	Proposed a privacy-preserving framework for healthcare systems using FL.
Xing et al. [27]	Developed an FL platform for medical data sharing with privacy protection.
Fu et al. [28]	Created a federated reinforcement learning approach for autonomous driving.
Rjoub et al. [29]	Constructed a deep reinforcement learning framework with FL for edge computing reliability.
Yang et al. [30]	Introduced a hierarchical trust structure for federated IoT applications.
Peng et al. [31]	Introduced an FL framework for IoT in the textile industry.
Song et al. [33]	Developed a contribution index based on Shapley value for FL participants.
Wang [34]	Utilized Shapley value for data feature importance determination.
Kang et al. [41]	Designed incentive mechanisms using contract theory for FL participation.
Majeed and Hong [44]	Presented FLchain, a blockchain-based FL approach.
Kim et al. [45]	Proposed BlockFL for decentralized FL training.
Li et al. [46]	Introduced BFLC for credible participant selection in FL.
Our	Proposed BCD-FL model enhances data quality, optimizes model aggregation, and improves reward distribution in FL.

presents a comprehensive summary of the literature review.

3. Preliminaries

This section provides an overview of the concepts related to BCD-FL, which include Federated Learning, blockchain, and smart contracts.

3.1. Federated Learning

We assume there exist a central server and N clients, denoted by $\mathcal{N}=\{1,2,\dots ,N\}$, each with its own local dataset $D_i$, $i=1,2,3.....N$. The number of samples in $D_i$ is denoted by $n_i$$=|D_i|$. The total number of samples from all datasets is denoted by $n={\sum }_i{n}_i$. The parameters to be trained in the FL model are denoted by $\mathit{w}$. In FL, the N clients collaboratively train the global model $\mathit{w}$ without uploading their local dataset to the central server, while minimizing the global loss function [47], as shown in Equation (1).

$$\underset{\mathit{w}}{min}F\left(\mathit{w}\right)=\sum _{i=1}^N\frac{n_i}{n}{F}_i\left(\mathit{w}\right),$$

(1)

where

$$F_i\left(\mathit{w}\right)=\frac{1}{n_i}\sum _{i=1}^{n_i}f\left(\mathit{w}\right).$$

(2)

Here, the local loss of a client i is denoted by $F_i$. The $f\left(\mathit{w}\right)$ in Equation (2) represents the multiclass cross-entropy loss on a data point $\left(x,y\right)$ for one-hot-encoded labels and is defined as shown in Equation (3).

$$f\left(\mathit{w}\right)=-\sum _{q=1}^C{y}_{q}log{p}_q(x,\mathit{w}),$$

(3)

where $p_q(x,\mathit{w})$ gives the probability that $x\in X$ belongs to class q, and C represents the total number of classes. It is important to note that our model is designed for a multi-class classification task. Therefore, we employ a multi-class cross-entropy loss function to quantify the difference between predicted and true class labels.

Stochastic Gradient Descent (SGD) is a commonly used optimization algorithm in machine learning to minimize the model’s loss function. The SGD algorithm involves iteratively computing the gradient of the loss function and adjusting the model’s parameters in the opposite direction of the gradient. This iterative process helps to gradually minimize the loss function, leading to improved accuracy and efficiency of the model. In FL, multiple clients have their own local datasets; we can use SGD to optimize the global model (i.e., $\mathit{w}$) in a decentralized manner. In the tth round (i.e., ${\mathit{w}}^t$), client i computes the gradient $g_i=\nabla F_i\left({\mathit{w}}^t\right)$ with the local dataset and updates ${\mathit{w}}^t$ at a learning rate $\eta$, as Equation (4).

$${\mathit{w}}_i^{t+1}\leftarrow {\mathit{w}}^t-\eta g_i.$$

(4)

At the $t+1$ iteration, the global model (i.e., $\mathit{w}$) is updated in Equation (5).

$${\mathit{w}}^{t+1}\leftarrow \sum _{i=1}^N\frac{n_i}{n}{\mathit{w}}_i^{t+1}.$$

(5)

In the training process, a threshold denoted as $\theta$ can be established to regulate convergence and model quality. FL allows the training of models on decentralized datasets by iteratively updating the global model. This process is crucial as it avoids transferring sensitive data to a central server, thereby enhancing data privacy and security.

3.2. Blockchain

Blockchain technology, a decentralized and peer-to-peer system, leverages cryptographic attributes to guarantee tamper-resistance and immutability. It stores transactional information in blocks with timestamps and references to preceding blocks, thus forming a chain-like structure. This structure exhibits symmetry characteristics, as every participating node in the network maintains a mirrored copy of the entire blockchain. Consensus algorithms ensure the consistency of this ledger. There are two main types of blockchains: public chains and federated chains. Public chains, such as Bitcoin [48] and Ethereum [49], are open to anyone who wants to participate, and the number of participants is not fixed. On the other hand, federated chains, such as IBM’s Hyperledger fabric [50], are restricted to only authorized participants, and the set of participants is usually pre-defined.

The tamper-proof nature of the blockchain can be leveraged to provide audit trails for FL. In other words, the transaction histories stored on the blockchain cannot be modified or deleted, providing on-chain accountability and a trustworthy record of the operations.

3.3. Smart Contract

Smart contracts, self-executing programs that operate on the blockchain, facilitate automated execution, verification, and enforcement of predetermined agreements between parties [51]. Smart contracts can be used to manage digital assets, enable electronic voting, execute financial transactions, and more. They are designed to trigger automatically and update the blockchain when pre-defined conditions are met. Unlike traditional contracts, the result of a smart contract execution cannot be changed or reversed, which provides assurance of the trustworthiness of decentralized applications. Smart contracts work similarly to If-Then [52] statements in other computer programs but with higher security and decentralization features.

3.4. InterPlanetary File System

The Interplanetary File System represents a decentralized mechanism dedicated to file storage and sharing. Employing a peer-to-peer methodology for data exchange, IPFS provides a resilient and distributed alternative to traditional file storage systems, promoting data permanence and reducing reliance on centralized data repositories. This approach ensures data are tamper-evident and persistent by using hashing technology. At the same time, IPFS supports version control, which allows users to access and recover previous file versions at any time, ensuring the security and stability of data. To improve network efficiency, IPFS uses Distributed Hash Tables (DHT) to store and find files, making it a fast and secure decentralized data storage and sharing platform. When combined with Ethernet smart contracts, IPFS enables high-capacity decentralized application data storage, which greatly promotes the development of distributed storage and applications.

4. System Model

In this section, we propose our BCD-FL architecture and the related processes.

4.1. Proposed System Architecture

As shown in Figure 1, the proposed BCD-FL system architecture includes data owners (participants or candidates), task publishers, blockchain networks, learning models, and multi-smart contracts. The numbers in the figure represent the corresponding steps.

• Data Owner: A data owner is a node that holds a local dataset that is not publicly available in FL. Data owners can be candidates or participants. Candidates are the nodes that are eligible to participate in the FL task and can bid by calling a smart contract. The selection process will be carried out by BCD-FL, and only successful bidders will be able to participate in the task. Once selected, the candidate will become a participant and can call the corresponding smart contract for calculation. Only the selected nodes that are designated as participants can receive rewards through smart contracts. This ensures that only the authorized nodes are part of the FL task and are incentivized appropriately for their contributions.
• Task Publisher: The task publisher is the entity that initiates the training task. It describes the type of data needed and the model structure and provides the payment budget for each round of the FL process. The above information is then published to the BCD-FL platform. Data owners who hold local datasets and have computational resources can participate in the learning process by bidding on the training task. The data owners develop their bidding strategy based on multiple factors, including their computational power, data volume, quality, etc., and invoke the corresponding smart contract to bid on the task publisher’s offer.
• Blockchain Network: We utilize blockchain in the BCD-FL as a medium for storing the IPFS hash of model weights ($\mathit{w}$). Each data owner initiates a transaction and invokes a smart contract to participate in FL. After validation, the transaction is encapsulated into a new block, which is then added to the blockchain network following consensus authentication.
• Learning Model: The learning model, often referred to as the training model, uses a dataset for training purposes. In the BCD-FL model, we distinguish between two types of learning models: the local model and the global model. The local model is trained individually by each participant using its own dataset. The global model is obtained through the cooperation of all winning nodes, with the model quality being evaluated and aggregated by the task publisher.
• Multi-smart Contracts: To enhance the trustworthiness and reliability of BCD-FL, we use smart contracts during the execution of FL tasks. Our model includes several types of smart contracts that are specifically designed to facilitate different stages of the process. These include Task Initialization Contract, Auction Contract, FL Contract, Reputation Contract, and Reward Contract. The FL process within the BCD-FL relies on several contracts to ensure its robustness, security, and fairness. Each of these contracts plays a critical role in achieving these goals.

4.2. Multi-Smart Contracts in BCD-FL

To better illustrate the multi-smart contracts used to undertake FL tasks, we briefly describe these contracts and their roles.

(1)

Task Initialization Contract: When the task publisher initiates a new FL task, the BCD-FL guarantees a systematic and organized start through the use of this contract, which is defined as follows:

• InitializeTask(): This function can be used by the task publisher to write the model parameter type, model structure, and payment budget information for an FL task to BCD-FL.

(2)

Auction Contract: After a task is published, interested candidates can bid through this type of contract during the auction time. After the auction ends, the task publisher combines the reputation and bids of the candidates to select the nodes. The main functions are as follows:

• StartAuction(): This function is called by the task publisher in order to initiate the auction process. The purpose of this function is to allow the task publisher to set the auction time and bid range, as well as verify their identity before proceeding with the execution of the auction. It is crucial to highlight that only the task publisher possesses the authority to activate this function, ensuring accountability and controlled management of FL tasks within the BCD-FL.
• BidPrice(): This function is designed to facilitate bidding by candidates, allowing them to specify the size of data used. If a candidate fails to bid during the auction period, it is considered to have abandoned the auction.

(3)

Federated Learning Contract: This contract serves as a means of communication between the task publisher and the data owner. The data owner uses this contract to retrieve the hash of the global model and access it from IPFS. On the other hand, the task publisher uses this contract to obtain the hash of the local model and upload the evaluation result and IPFS hash of the aggregated global model. The main functions are as follows:

• DownloadGlobalModel(): After the task is published, the winning node can retrieve the latest global model hash by invoking this function. Subsequently, the node obtains the most recent global model from the IPFS.
• UploadLocalModel(): Once the winning node obtains the global model, it proceeds to train this model and subsequently uploads the trained local model to the IPFS. The node then invokes this function to upload the IPFS hash of the local model.
• DownloadLocalModel(): The task publisher invokes this function to download the IPFS hash of all local model parameters uploaded by the winning nodes. Subsequently, the publisher retrieves the local models from the IPFS file storage system using these hashes. Importantly, this function also requires identity verification of the caller, thereby ensuring the integrity of the FL process and safeguarding the data from unauthorized access.
• UploadTestResult(): When the task publisher finishes downloading all local models, the task publisher performs model quality evaluation tests on all local models. Then, this function uploads the evaluation result and calls the reputation contract to update the reputation of the node.
• UploadGlobalModel(): After obtaining the global model and uploading it to IPFS through the model aggregation algorithm, the task publisher can use this function to update the IPFS hash of the global model.

(4)

Reputation Contract: Reputation contract is designed to enhance trust by incentivizing good behavior and penalizing poor behavior among data owners. Reputation contract will add a reputation score to those nodes with good behavior and deduct the reputation of nodes with poor quality of uploaded models. The main functions are discussed as follows:

• UpdateReputation(): This function updates the reputation of a node by testing the input of the results and combining the historical reputation values. The reputation indirectly reflects the quality of the local models so that the task publisher can select high-quality models for aggregation.
• QueryReputation(): The data owner can call this function to view the current reputation as well as the historical reputation of the record.

(5)

Reward Contract: This contract pays the reward for the winning node. The main function is described as follows:

• Payment(): The task publisher considers the reputation of the data owner and the bid price, and then makes a payment to the winning node based on the reward payment algorithm.

4.3. Federated Learning Processes in BCD-FL

The proposed BCD-FL model utilizes blockchain technology to share global models and reputation scores. This innovative approach addresses the common issue of single points of failure in traditional models and improves the credibility and reliability of FL systems by maintaining a transparent and unchangeable record of transactions. In the BCD-FL, even individuals with limited budgets can act as task publishers, initiating tasks and recruiting participants to create a more inclusive and diverse learning environment. This model takes into account the constraints of computation, costs, and potential rewards for potential participants, allowing them to make informed decisions about participating in FL based on their own resources and potential rewards. To facilitate this process, interested individuals can submit their bid prices to the task publisher before the auction time expires, if necessary. The task publisher then collects all bids and selects the winning node based not only on the bid amount but also on the combined reputation of the nodes. This approach ensures a fair and balanced selection process that rewards both competitive bids and good reputations. Once selected, the winning node receives the corresponding reward, providing motivation for active and high-quality participation in the FL process. The following steps provide a more detailed breakdown of the working process.

• Step 1: After sending the initialized model to IPFS, the task publisher invokes the task initialization contract function with the information of the payment budget, model parameters, and computational resources.
• Step 2: The task publisher uses reverse auction to call StartAuction() to initiate the auction. Interested candidates can develop a strategy based on their computational power, data volume, and computational cost, and then call BidPrice() function to bid. The task publisher combines the bids and reputation of the nodes and selects the winning node.
• Step 3: The winning node uses the DownloadGlobalModel() function to obtain the global model hash and retrieve the actual global model from the IPFS.
• Step 4: The winning node sends the trained local model to IPFS, and then calls UploadLocalModel() to upload the model IPFS hash.
• Step 5: The task publisher calls the DownloadLocalModel() function to obtain the hash of the local model and the local model of all nodes from IPFS.
• Step 6: After obtaining all the local models, the task publisher performs the model quality evaluation and calls UploadTestResult() to upload the test results, followed by calling UpdateReputation() to update the reputation of the winning nodes.
• Step 7: The task publisher checks the model quality and filters out the bad models, and aggregates the remaining models to obtain a new global model.
• Step 8: When the new global model is generated, the task publisher sends the updated global model to IPFS, and the corresponding hash value is uploaded to the blockchain network by calling the UploadGlobalModel() function.
• Step 9: The task publisher can make a payment to the winning nodes by calling the Payment() function.

The above describes one iteration of the working process in BCD-FL. The above steps will be repeated multiple times until the accuracy threshold is reached or the number of iterations reaches a predefined number. Algorithms 1 and 2 show the pseudo-code implementations of different processes in BCD-FL.

Algorithm 1 Processing Methods for Task Publisher in BCD-FL

Input:  Data Owners $\mathcal{N}$, FL Task Publisher p, Budget $\mathcal{B}$, Data Owners Bid Price Set $B_{\mathcal{N}}$, Data Owners Reputation Set ${Re}_{\mathcal{N}}$, Task Initialization Contract ${\mathcal{C}}_i$, Auction Contract ${\mathcal{C}}_{au}$, Federated Learning Contract ${\mathcal{C}}_{fl}$, Reputation Contract ${\mathcal{C}}_{re}$, Reward contract ${\mathcal{C}}_r$; Output:  Global Model $w_p$;   1: Initialization evaluation result $es$ = ∅, test result $ts$ = ∅, winning data owner set $\mathcal{M}$ = ∅, local model hash set ${\mathcal{H}}_{\mathcal{M}}$ = ∅;   2: Upload Initial Global Model $w_p\Rightarrow$ IPFS;   3: ${\mathcal{C}}_{fl}$.InitializeTask();   4: for$i=1$toTdo   5:    ${\mathcal{C}}_{au}$.StartAuction();   6:    SelectDataOwner(${Re}_{\mathcal{N}}$,$B_{\mathcal{N}}$,$\mathcal{B}$)$\Rightarrow \mathcal{M}$;   7:    ${\mathcal{C}}_{fl}$.DownloadLocalModel($\mathcal{M}$) $\Rightarrow {\mathcal{H}}_{\mathcal{M}}$;   8:    Download all local models $w_{\mathcal{M}}\Leftarrow$IPFS(${\mathcal{H}}_{\mathcal{M}}$);   9:    ModelQualityEvaluationDetection($w_{\mathcal{M}}$)$\Rightarrow es$;  10:    ${\mathcal{C}}_{fl}$.UploadTestResult($\mathcal{M}$,$es$) $\Rightarrow ts$;  11:    ${\mathcal{C}}_{re}$.UpdateReputation($\mathcal{M}$,$ts$);  12:    ModelFilteringAndAggregation($w_{\mathcal{M}}$)$\Rightarrow w_p$;  13:    Upload Global Model $w_p\Rightarrow$ IPFS;  14:    ${\mathcal{C}}_{fl}$.UploadGlobalModel($ipfsHash\left(w_p\right)$);  15:    ${\mathcal{C}}_r$.Payment($\mathcal{M}$);  16: end for  17: return $w_p$;

Algorithm 2 Processing Methods for Data Owner in BCD-FL

Input:  Local Data Size D, Bid Price b, Auction Contract ${\mathcal{C}}_{au}$, Federated Learning Contract ${\mathcal{C}}_{fl}$; Output:  Local Model $w_n$;   1: Initialization global model hash ${\mathcal{H}}_p$ = 0, new global model hash ${\mathcal{H}}_{p_{new}}$ = 0, local model = $w_n$;   2: for$i=1$toTdo   3:    ${\mathcal{C}}_{au}$.BidPrice(b,D);   4:    if Selected then   5:      ${\mathcal{C}}_{fl}$.DownloadGlobalModel() $\Rightarrow {\mathcal{H}}_{p_{new}}$;   6:      if ${\mathcal{H}}_p\ne {\mathcal{H}}_{p_{new}}$ then   7:         ${\mathcal{H}}_p$⇐${\mathcal{H}}_{p_{new}}$;   8:         Download global model $w_p\Leftarrow$IPFS(${\mathcal{H}}_p$);   9:         ModelTraining($w_p$)⇒$w_n$;  10:         Upload Local Model $w_n\Rightarrow$ IPFS;  11:         ${\mathcal{C}}_{fl}$.UploadLocalModel($ipfsHash\left(w_n\right)$);  12:      end if  13:    end if  14: end for  15: return $w_n$;

4.4. Optimization Goals

The goal of BCD-FL is to maximize the quality of the aggregated model during each iteration while ensuring individual rationality, computational efficiency, budget balance, and truthfulness. Assume that a task publisher p publishes a learning task with a task budget of $\mathcal{B}$ in iteration t, and each participating data owner i has a cost $c_i^t$ for the task. The task publisher selects the winning node by considering the reputation ${Re}_i^t$ and the bid $b_i^t$. We further define a variable $x_i^t$ to be 1 if the winning node i is selected in iteration t, and 0 otherwise. We use $r_i^t$ to denote the reward received by the node. To quantitatively calculate the gain for a node participating in the learning task, we define utility as follows.

Table 2. Involved notations.

Notations	Descriptions
$u_i^t$	The utility value of the node i in the t-th iteration.
$r_i^t$	The reward received by node i in the t-th iteration
$c_i^t$	The cost of node i in the t-th iteration
$b_i^t$	The bid of node i in the t-th iteratio
${Re}_i^t$	The reputation of node i in the t-th iteration
$x_i^t$	The selection status of node i
$w_i^t$	The local model parameters in the t-th iteration
$D_i$	Training datasets of the node i
$\mathcal{B}$	Budget
$\eta$	Learning rate

lists the primary notations and symbols used.

Definition 1

(Node Utility). When participating in the FL process, iteration t, node i has a utility gain as shown in Equation (6), which is calculated as the difference between the reward received and the associated cost.

$$u_i^t=\left\{\begin{array}{cc}{r}_i^t-c_i^t,\hfill & x_i=1\hfill \\ 0,\hfill & \mathrm{others}.\hfill \end{array}\right.$$

(6)

In BCD-FL, the process of maximizing the quality of the aggregated model has to satisfy the following constraints.

Individual Rationality: In each iteration t, node i participating in FL should yield a non-negative utility, i.e., $u_i^t\ge 0$.

Computational Efficiency: The incentive mechanism is capable of executing node selection and reward allocation in polynomial time.

Budget Balance: The total amount of compensation allocated to participating nodes cannot exceed the predefined budget.

Truthfulness: In iteration t, a node can only truthfully report its cost to obtain the optimal payoff. In other words, if node i reports untrue bids $b_i^t>c_i^t$ during the bidding process, then its utility ${\widehat{u}}_i^t$ should be smaller than its reported bid utility $u_i^t$, i.e., ${\widehat{u}}_i^t<u_i^t$.

5. The Incentive and Reputation Techniques in BCD-FL

In this section, we introduce the smart contract incentive mechanism based on reverse auction and the reputation mechanism based on model quality evaluation for the BCD-FL.

5.1. Incentive Mechanism Based on Reverse Auction (IMRA)

With a limited budget $\mathcal{B}$, the proposed reputation mechanism can facilitate the selection of the winning nodes to obtain a better global model. Taking account of the computational cost and system efficiency, we adopt reverse auction to incentivize high-quality nodes. In specific, each node i submits its bid information $b_i^t$ through a smart contract. BCD-FL selects a set of winner nodes ${\mathcal{M}}^t\subset \mathcal{N}$ and determines a payment set within the payment budget ${\mathcal{R}}^t=\left\{r_i^t|\forall i\in {\mathcal{M}}^t\right\}$. The IMRA process in BCD-FL can be formulated as Equations (7)–(10).

$$\underset{{\mathcal{M}}^t,{\mathcal{R}}^t}{max}\sum _{i\in {\mathcal{M}}^t}{Re}_i^t,$$

(7)

$$s.t.\sum _{i\in \mathcal{N}}{r}_i^t{x}_i^t\le \mathcal{B},$$

(8)

$$r_i^t\ge b_i^t,\forall i\in {\mathcal{M}}^t,$$

(9)

$$x_i^t\in \{0,1\},\forall i\in \mathcal{N}.$$

(10)

Equation (7) demonstrates the objective is to maximize the overall reputation value ${Re}_i^t$ and Equation (8) constraints the budget $\mathcal{B}$ of the task publisher. Equation (9) ensures that the reward amount of each winning node is greater than its claimed bid price. The output of this IMRA process is to determine the binary variable $x_i^t$ for each $i\in \mathcal{N}$, the reward $r_i^t$ for each node, and the set of winner nodes ${\mathcal{M}}^t$.

To efficiently handle the above IMRA process, here we propose a greedy algorithm as shown in Algorithm 3. Specifically, in Lines 4–7, we initialize the bidding prices and reputation values for the tth iteration. In Line 8, the algorithm sorts nodes in ascending order based on their unit reputation bid price $b_i^t/R{e}_i^t$, which is determined by their bid $b_i^t$ and reputation value ${Re}_i^t$. The ranking metric $b_i^t/R{e}_i^t$ indicates the bidding ability of node i. Then, the algorithm greedily adds node i to the set of winning nodes ${\mathcal{M}}^t$ according to their ranking until the total payment reaches the payment budget $\mathcal{B}$ as shown in Lines 9–12. Assuming that the ranking selects k nodes from front to back, but in this process, we pay the winning node i with a unit reputation bid of $(k+1)$, which means that the node receives a reward based on $(k+1)$ nodes in the ranking. In the tth iteration, the reward of node i is given by Equation (11), and Lines 15–18 show the updating process.

$$r_i^t=\left\{\begin{array}{cc}{Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t},\hfill & i\in {\mathcal{M}}^t\hfill \\ 0,\hfill & i\notin {\mathcal{M}}^t.\hfill \end{array}\right.$$

(11)

It is clear that the algorithm satisfies the constraints of Equation (8) and Equation (10), and the others are proved in Section 6.

Algorithm 3 Selecting and Paying Methods for Data Owners

Input:  Data Owners $\mathcal{N}$, FL Task Publisher p, Budget $\mathcal{B}$, Data Owners Bid Price Set $B_{\mathcal{N}}=\{b_1^t,...,b_n^t\}$, Reputation Contract ${\mathcal{C}}_{re}$, Reward contract ${\mathcal{C}}_r$; Output:  Select Result $x=\{x_1^t,...,x_n^t\}$, Payment $r=\{r_1^t,...,r_n^t\}$, Winning Node Set ${\mathcal{M}}^t$;   1: Initialization ${\mathcal{M}}^t$ = ∅, $k=1$, Data Owners Reputation Set $R{e}^t=\varnothing$;   2: Initialization $r_i^t$⇐ 0 for each $i\in \mathcal{N}$;   3: Initialization $x_i^t$⇐ 0 for each $i\in \mathcal{N}$;   4: for  $i=1$ to  N  do   5:    $p.{\mathcal{C}}_{re}$.QueryReputation(i) $\Rightarrow R{e}_i^t$;   6:    $R{e}^t\Leftarrow R{e}^t+\left\{R{e}_i^t\right\}$;   7: end for   8: Sort all $i\in \mathcal{N}$ in ascending order of $b_i^t/R{e}_i^t$;   9: while$\left(R{e}_k^t+{\sum }_{i\in {\mathcal{M}}^t}R{e}_i^t\right)\frac{b_{k+1}^t}{R{e}_{k+1}^t}\le \mathcal{B}$  do  10:    $k=k+1$;  11:    ${\mathcal{M}}^t\Leftarrow {\mathcal{M}}^t+\left\{k\right\}$;  12: end while  13: $k\Leftarrow k-1$;  14: $p.{\mathcal{C}}_r$.TransferFundToContract($\mathcal{B}$);  15: for  $i=1$ to  k  do  16:    $x_i^t\Leftarrow 1$;  17:    $r_i^t\Leftarrow {Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$;  18: end for  19: $p.{\mathcal{C}}_r$.Payment(${\mathcal{M}}^t$,r);  20: return $\left(x,r,{\mathcal{M}}^t\right)$;

5.2. Reputation-Based Model Quality Evaluation

After receiving the local models from the winning nodes, the task publisher can use a dataset for model quality evaluation. The evaluation of a local model’s quality should take into account how much it contributes to the global model. However, relying solely on an accuracy threshold to determine whether a local model is good or not can be challenging, as determining this threshold is not a straightforward process. We use a loss-based marginal approach to evaluate the quality of the uploaded local models. Specifically, we first aggregate the local models uploaded by participating nodes in each round to obtain a global model w and obtain a loss on the test set. Then, we remove the local model from node i and aggregate the remaining local models to obtain another global model $w^{\prime }$. The quality of the training model from node i in our iteration t can be defined as Equation (12).

$$m_i^t=Loss\left(w^{\prime }\right)-Loss\left(w\right).$$

(12)

This is because a better-quality local model is involved in the aggregation process, and the loss of the global model on the test set should be reduced or maintained. Based on the results from the model quality evaluation, we develop a reputation mechanism to reflect the reliability of a node and its model quality, which can be preserved in the blockchain to maintain its immutability and traceability as well as transparency. In specific, BCD-FL initializes each node with a reputation value $Re$= 1. At the tth round, based on the historical reputation values of node i and the current model quality evaluation results, the reputation of node i can be defined as in Equation (13).

$$R{e}_i^t=\left\{\begin{array}{cc}{\widehat{Re}}_i^{t-1}+{\rho }_{1}G\left(\frac{m_i^t}{{\sigma }_1}\right),\hfill & m_i^t\ge \delta \hfill \\ {\widehat{Re}}_i^{t-1}-{\rho }_{2}G\left(\frac{-m_i^t}{{\sigma }_2}\right),\hfill & m_i^t<\delta .\hfill \end{array}\right.$$

(13)

The variables in Equation (13) are explained as follows.

• ${\widehat{Re}}_i^{t-1}$ represents the historical reputation value of node i, which is the exponential moving average, and can be computed as the following formula:

  $${\widehat{Re}}_i^{t-1}=\alpha \times R{e}_i^{t-1}+(1-\alpha )\times {\widehat{Re}}_i^{t-2},$$

  (14)

  where $R{e}_i^{t-1}$ denotes the reputation value in the $t-1$th round and $\alpha$ is a smoothing factor. If the value of $\alpha$ is larger, the exponential moving average is more sensitive to the reputation value at the current moment. Otherwise, the past reputation values have a greater impact.
• $G(·)$ denotes the Gompertz [53] function, which measures the impact of quality detection on reputation, and translates the results of model quality evaluation into updating strength, as shown in Equation (15).

  $$y=a{e}^{-b{e}^{-cx}}.$$

  (15)
  In Equation (15), a, b, and c are coefficients. When $a=1$, $b=1$, and $c=5$, the function is plotted as shown in Figure 2.
• ${\rho }_1$ and ${\rho }_2$ are the reputation update values of unit updating strength. The update is triggered only when the training model quality is greater than or equal to $\delta$. Otherwise, the node will be penalized by a decrease in its reputation. The ${\sigma }_1$ and ${\sigma }_2$ are scaling factors to limit the updating strength in a certain interval.

This mechanism translates a node’s local model quality into an update strength, which is then combined with its historical reputation to determine its present reputation value. If a node contributes a subpar or malicious model, the reputation penalty update is triggered. The lower the quality of the model, the more severe the reputation decrement will be. On the other hand, nodes that contribute superior models will enhance their reputation value, increasing their chances of being selected by task publishers in the subsequent node selection process.

5.3. Model Aggregation

For better aggregation to obtain a global model, poor-quality local models need to be filtered. The local model of a node is accepted by BCD-FL only when $m_i^t\ge \delta$, otherwise it is discarded. The federated average aggregation algorithm generally follows Equation (16).

$$\mathit{w}=\sum _{i=1}^N\frac{D_i^t}{{\sum }_{i=1}^N{D}_i^t}{\mathit{w}}_i^t.$$

(16)

In Equation (16), $D_i^t$ denotes the size of the data volume used to train the local model by node i in iteration t, and ${\mathit{w}}_i^t$ denotes the local model parameters. It is noteworthy that the approach described above might not always preserve the truthfulness of the reported data volume or the quality of the local models provided by the nodes [54]. To improve the accuracy of our model, we propose an aggregation strategy that considers not only the data size of each node but also the reputation of each node. Specifically, given a set of accepted nodes ${\mathcal{M}}^{\prime }t$ and their local model parameters ${\mathit{w}}_i^t$, the global model $\mathit{w}$ will be computed as Equation (17).

$$\mathit{w}=\sum _{i\in {\mathcal{M}}^{\prime }t}\frac{R{e}_i^t{D}_i^t}{{\sum }_{i\in \mathcal{M}{’}^t}R{e}_i^t{D}_i^t}{\mathit{w}}_i^t.$$

(17)

6. Theoretical Analysis

In this section, we analyze how BCD-FL satisfies the computational efficiency, individual rationality, budget balance, and truthfulness as described in Section 4.4.

Theorem 1.

BCD-FL is computationally efficient with a time complexity of $\mathcal{O}(NlogN)$ for the processes of node selection and reward assignment, where $N=\left|\mathcal{N}\right|$ is the number of nodes in the set $\mathcal{N}$.

Proof of Theorem 1.

In Algorithm 3, Lines 4–7 involve querying the reputation of each node through the reputation contract, which has a time complexity of $\mathcal{O}\left(N\right)$. Line 8 has a time complexity of $\mathcal{O}(NlogN)$ for sorting nodes in ascending order based on their unit reputation bid price $b_i^t/R{e}_i^t$. Lines 9–12 find the top k nodes that satisfy the selected condition with a time complexity of $\mathcal{O}\left(N\right)$. Lines 15–18 determine the reward paid to all winning nodes with a time complexity of $\mathcal{O}\left(N\right)$. Thus, the total time complexity of Algorithm 3 is $\mathcal{O}(N+NlogN+N+N)=\mathcal{O}(NlogN)$. □

Theorem 2.

BCD-FL is individually rational.

Proof of Theorem 2.

In iteration t, if BCD-FL does not select node i, then its utility $u_i^t=0$. Otherwise, node i wins the bid with $b_i^t=c_i^t$. Then its reward amount is $r_i^t={Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$ and its utility $u_i^t=r_i^t-c_i^t={Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-b_i^t$. Since node i is successful in winning the bid, then we have $\frac{b_i^t}{R{e}_i^t}\le \frac{b_{k+1}^t}{R{e}_{k+1}^t}$, and $u_i^t={Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-b_i^t\ge 0$. Thus, for each node i, we have $u_i^t\ge 0$, proving that BCD-FL is individually rational. □

Theorem 3.

BCD-FL satisfies the budget balance.

Proof Theorem of 3.

In iteration t, we have a payment budget $\mathcal{B}$, and the reward for each node i is based on the $(k+1)$th node in the ranking and only the top k nodes obtain the reward. Therefore, we have ${\sum }_{i=1}^N{r}_i^t={\sum }_{i=1}^k{r}_i^t+{\sum }_{i=k+1}^N{r}_i^t={\sum }_{i=1}^k{Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, and since BCD-FL is performed under the condition of Line 9 of Algorithm 3, so ${\sum }_{i=1}^k{Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}\le \mathcal{B}$. We proved that BCD-FL satisfies the budget balance. □

Lemma 1.

According to Myerson’s theorem [55], the reverse auction mechanism is truthful and efficient only when the winner selection problem is monotonic and each winner’s payment is a critical payment.

• The selection rule is monotonic: In iteration t, if a node i wins by its bid price $b_i^t$, then node i will win by bidding at any bid price ${\widehat{b}}_i^t<b_i^t$.
• Critical payment: If node i wins with a bid price $b_i^t$, then node i can also win with another bid price ${\widehat{b}}_i^t$, but winning with $b_i^t$ gives node i the maximum payment, which is called the critical payment of node i. That is, the critical payment is the maximum bid that node i needs. If a node’s bid price is greater than the critical payment, it is unlikely to win the bid.

Theorem 4.

BCD-FL is truthful.

Proof of Theorem 4.

Assuming that node i wins by bidding price $b_i^t$, and if node i bids ${\widehat{b}}_i^t<b_i^t$ with constant bids from others, then it will be ranked higher than its original position and node i will still be selected. Thus, the selection rule is monotonic.

In iteration t, assuming that node i is selected by BCD-FL at the price of $b_i^t$, then the payment to node i is $R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$. If node i bids using the bid price of ${\widehat{b}}_i^t<R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, we choose the first failed node to be the $k+1$th bid to pay, i.e., $\frac{{\widehat{b}}_i^t}{R{e}_i^t}<\frac{b_{k+1}^t}{R{e}_{k+1}^t}$. Therefore, node i is still within the top k candidates and ${\sum }_{i=1}^k{Re}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}\le \mathcal{B}$. If node i bids using the bid price of ${\widehat{b}}_i^t>R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, then $\frac{{\widehat{b}}_i^t}{R{e}_i^t}>\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, and ${\sum }_{i=1}^{k+1}{Re}_i^t\frac{b_{k+2}^t}{R{e}_{k+2}^t}>\mathcal{B}$, which indicates node i is beyond our payment budget and cannot be selected. Thus, $R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}$ is the critical payment value for node i to win the bid with $b_i^t$.

We next examine two scenarios, namely the node being selected and not being selected, to elucidate why a node cannot enhance its utility by submitting an untrue bid.

Scenario One, Node Not Selected: Assume node i is not selected with its true bid $b_i^t$, resulting in a utility value $u_i^t=0$. From this, we can infer $\frac{b_i^t}{R{e}_i^t}>\frac{b_{k+1}^t}{R{e}_{k+1}^t}$. Even if node i increases its bid to ${\widehat{b}}_i^t>b_i^t$, we still have $\frac{{\widehat{b}}_i^t}{R{e}_i^t}$ greater than $\frac{b_{k+1}^t}{R{e}_{k+1}^t}$. As a result, node i still will not be selected as it exceeds the budget. Conversely, if node i submits a lower bid ${\widehat{b}}_i^t<b_i^t$, although node i may be selected, its utility ${\widehat{u}}_i^t=R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-c_i^t=R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-b_i^t$ will be less than 0. Thus, in any case, when node i is not selected, any untrue bid cannot bring a utility higher than the true bid.

Scenario Two, Node Selected: Assume node i is selected with its true bid $b_i^t$, we can infer $\frac{b_i^t}{R{e}_i^t}<\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, its utility is $u_i^t=R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-c_i^t=R{e}_i^t\frac{b_{k+1}^t}{R{e}_{k+1}^t}-b_i^t$. If node i submits a lower bid ${\widehat{b}}_i^t<b_i^t$, we have $\frac{{\widehat{b}}_i^t}{R{e}_i^t}$ less than $\frac{b_{k+1}^t}{R{e}_{k+1}^t}$, node i will still be selected and its utility remains unchanged. Conversely, if node i proposes a higher bid ${\widehat{b}}_i^t>b_i^t$, then node i will not be selected, its utility ${\widehat{u}}_i^t=0$. Thus, in any case, when node i is selected, any untrue bid cannot bring a utility higher than the true bid.

In summary, regardless of whether a node is selected or not, providing an untrue bid cannot increase its utility. □

7. Performance Analysis

7.1. Experimental Settings

We conducted extensive simulations with an Ubuntu 20.04 system of an Intel(R) Xeon(R) Platinum 8338C CPU @ 2.60 GHz and RTX 3090 (24 GB). The detailed system settings are as follows.

• PyTorch: We used Python 3.7.13 and PyTorch 1.13.0 to train and aggregate the local and global models.
• IPFS: IPFS uses version 0.7.0 and each IPFS client uses the "ipfs init" command for initial setup and the "ipfs daemon" command to start the daemon.
• Blockchain Network Setup: We have built a local blockchain network using Ganache. The smart contracts we develop are written in Solidity, a programming language specifically designed for creating contracts on the Ethereum blockchain.
• Dataset: In our experiments, the FL model is trained on the MNIST dataset, which is commonly used as a benchmark dataset for testing FL algorithms. The training set consists of 60,000 examples, whereas the test set has 10,000 examples. The dataset contains handwritten digits ranging from 0 to 9.
• Federated Learning Setup: The setup for FL is shown in

  Table 3. Parameter settings.

  Parameter	Setting
  Number of data owners	$\mathcal{N}$ = 20
  Task Publisher’Budget	$\mathcal{B}$ = 20
  Bid range	b = [1∼3]
  Data Size range	D = [1000∼3000]
  Percentage of malicious nodes	${\mathcal{N}}_e$ = [0, 0.1, 0.3, 0.5]
  Mislabeled data ratio	${\mathcal{R}}_e$ = [0∼1]
  Global epochs	E = 10
  Local epochs	e = 5

  , where we set 20 data owners and a budget of 20 for each round of task publishers. Additionally, we set the number of malicious nodes at random, where the value ${\mathcal{N}}_e$ = 0.5 indicates that half of the nodes in the network are malicious. When ${\mathcal{R}}_e$ = 1, it indicates that malicious nodes tampered with the data labels and set the mislabeled data ratio to 100%, which results in low-quality error samples. This occurs because the data of malicious nodes alter the data labels in accordance with the maliciousness of the nodes themselves. Additionally, we have set the parameters within our model for optimal performance and accuracy. Specifically, the values ${\rho }_1$ and ${\rho }_2$ are set to 0.1 and 0.5 respectively, while ${\sigma }_1$ and ${\sigma }_2$ are both set to 0.2. The value of $\delta$ is chosen to be −0.02. These parameter settings empower the model to effectively handle data from malicious nodes and maintain the integrity of our learning process.
• The model architecture: The DNN architecture of MNIST is presented in

  Table 4. DNN architecture for MNIST.

  Serial Number	Layer (Type)	Value	State_dict
  1	Input	(784,)	-
  2	fc1	(784,256)	fc1.weight fc1.bias
  3	Relu	-	-
  4	fc2	(256,128)	fc2.weight fc2.bias
  5	Relu	-	-
  6	fc3	(128,10)	fc3.weight fc3.bias

  . The architecture of the model is comprised of three layers that are fully connected, and the activation function employed is Rectified Linear Unit (ReLU). The model employs the cross-entropy loss function as its loss function, and the learning rate $\eta$ is assigned a value of 0.01.

7.2. Experimental Results

(1) Accuracy Analysis: To evaluate the performance of our proposed BCD-FL Federated Learning algorithm. We implement two benchmark approaches: a knapsack greedy mechanism and a bid price first mechanism. In the knapsack greedy approach, the data owners are sorted in a descending order according to their data volume and bid prices. The data owners with high data volume and low bids are greedily selected regardless of data quality and truthfulness. In the bid price first approach, the selection process prioritizes the data owner with the lowest bid price, irrespective of their level of truthfulness. The experimental design includes varying proportions of malicious nodes, specifically 0%, 10%, 30%, and 50%. To make the experiment most effective, the mislabeled data ratio (${\mathcal{R}}_e$) assigned to each malicious node is established at 1, indicating a data quality of 0 for these nodes. Furthermore, it is noteworthy that the knapsack greedy approach and the bid price first approach both make use of FedAvg [9] for the purpose of model aggregation.

Figure 3 shows the results after ten iterations. For scenarios where the proportion of malicious nodes is either 0% or 10%, the knapsack greedy mechanism and the bid price first mechanism do not differ much from our proposed algorithm in terms of accuracy. Additionally, The accuracy for the cases with 10% malicious nodes is also similar to the ones with 0% malicious nodes. This phenomenon can be attributed to the fact that the number of malicious nodes is relatively small, and therefore may not be selected in cases where there is a limited budget. However, when the proportion of malicious nodes increases to 30% or 50%, the performance of both the knapsack greedy mechanism and the bid price first mechanism deteriorates significantly. In contrast, our proposed algorithm continues to exhibit excellent performance.

In addition, our study investigates the impact of payment budget on model accuracy, specifically in scenarios where 50% of nodes are malicious. We allocated payment budgets of 30 and 40 in two separate instances and compared them against a baseline payment budget of 20. The mislabeled data ratio ${\mathcal{R}}_e$ associated with malicious nodes remained fixed at 1. Our experimental results, illustrated in Figure 4, suggest that increasing the payment budget may increase the selection of malicious nodes, but our algorithm still exhibits robust performance and improves accuracy.

(2) Reputation Analysis: In Figure 5, we increase ${\mathcal{R}}_e$ from 0 to 1 for 5 of the 20 nodes, while the data volume D is set to 3000 for all nodes for fairness, the bid price is set to 1 for all 5 of them, the payment budget is set to 20 for each iteration, and the initial reputation of all nodes is set to 1. We can see that nodes providing higher quality data have gained higher reputations, more than 1 in most cases. On the contrary, nodes that provide poor-quality data are penalized, leading to a decrease in reputation. In addition, the reputation decreases faster for nodes with worse data quality. This means that the possibility of selecting malicious or underperforming nodes is reduced, especially when the budget is limited. This reputation mechanism serves to incentivize the sharing of high-quality data while preventing dishonest or unqualified participation. As a result, it improves the overall performance of collaborative learning systems.

(3) Utility Analysis: Figure 6 shows how the data quality impacts the utility when changing the mislabeled data ratio. We can see that when the node’s mislabeled data ratio ${\mathcal{R}}_e$ is 0, the node’s utility will first increase and eventually stabilize. The utility of malicious nodes shows an initial increase followed by a sharp decrease. This situation arises because malicious nodes are initially selected by the model if they submit the lowest bid price. However, after two iterations, the reputation of such malicious nodes gradually decreases, leading to an increase in the unit reputation bid price. This leads to a continuous decrease in their ranking until they are no longer selected by the model. Notably, the utility decreases faster for nodes with poorer data quality, effectively preventing malicious or poor-quality nodes from participating in the FL process.

(4) Contract Cost Analysis: We evaluate the gas consumption of function calls for various types of contracts in Figure 7. We see that the gas consumption varies across functions, where the cost of uploading the test results of the model quality evaluation is the largest, with a gas consumption cost of 20,700, followed by the bidding function. The smallest cost is from the initiating auction function, which is only 47,000. Note that the gas consumption from all functions is within the acceptable range, being less than 250,000. Since the download local and global model functions and the query reputation function are set as view functions, they do not consume gas.

(5) Storage Cost Analysis: Assuming 20 nodes participate in the FL process with a budget of 20, there may be between 6 to 20 local models and one global model per iteration. Additionally, if we consider that the model has three levels of full connectivity and occupies approximately 0.89 MB, the storage cost per round will range from 6.23 MB to 18.69 MB. In BCD-FL, it uploads the model to IPFS and returns a 46B file hash to the smart contract through transactions. According to

Table 5. Comparison of storage capacity.

Name	One Iteration	After 10 Iteration
Traditional FL	6.23∼18.69 MB	62.3∼186.9 MB
BCD-FL	36 KB	360 KB

, the block size generated by each transaction is 36 KB per iteration.

The BCD-FL model has shown to be highly storage-efficient, requiring only 0.19% to 0.56% of the storage space compared to a model stored directly within the blockchain. Moreover, as the number of iterations or participants increases, this ratio decreases, further highlighting the storage advantages of uploading the model to the IPFS.

8. Conclusions

To improve the model quality in FL, we have proposed a BlockChain-based Decentralized Federated Learning (BCD-FL) model. The proposed model includes two effective mechanisms: a smart contract incentive mechanism based on reverse auction and a reputation mechanism based on model quality evaluation. The incentive mechanism uses a reverse auction method to identify data owners with low bids and high reputations under a limited payment budget. The reputation mechanism evaluates the quality of the training model and translates it into the reputation of the data owner. Our analysis and simulation results have demonstrated several desirable properties of the proposed BCD-FL, such as computational efficiency, individual rationality, budget balance, and truthfulness. Moreover, experimental results have shown that the proposed model is effective in terms of incentive, model aggregation, and storage overhead. In future work, we will study how to effectively enhance the robustness of the proposed model in practical systems.