Graph Neural Network Based Asynchronous Federated Learning for Digital Twin-Driven Distributed Multi-Agent Dynamical Systems

Abstract

The rapid development of artificial intelligence (AI) and 5G paradigm brings infinite possibilities for data annotation for new applications in the industrial Internet of Things (IIoT). However, the problem of data annotation consistency under distributed architectures and growing concerns about issues such as data privacy and cybersecurity are major obstacles to improving the quality of distributed data annotation. In this paper, we propose a reputation-based asynchronous federated learning approach for digital twins. First, this paper integrates digital twins into an asynchronous federated learning framework, and utilizes a smart contract-based reputation mechanism to enhance the interconnection and internal interaction of asynchronous mobile terminals. In addition, in order to enhance security and privacy protection in the distributed smart annotation system, this paper introduces blockchain technology to optimize the data exchange, storage, and sharing process to improve system security and reliability. The data results show that the consistency of our proposed FedDTrep distributed intelligent labeling system reaches 99%.

1. Introduction

With the rapid advancement of cloud computing, big data, artificial intelligence, and other cutting-edge technologies, the pace of digital transformation and intelligent evolution within enterprises has significantly accelerated. Emerging as a sustainable mechanism fostering the effective utilization of enterprise data, the concept of a data center has gained prominence since its inception in 2018, aiming to fortify future capabilities and address potential challenges that organizations may encounter. The data center architecture incorporates distinct layers, such as the development tool layer, data asset layer, asset management layer, data service layer, data operation system, data security system, and other specialized modules, with the focal point revolving around data assets and data services, both facilitated by data labels.

Data labels serve as the bridge between the data warehouse (DW) and the data mart, aligning with the DW while setting the stage for activities within the data mart. Data assets are inherently structured with tags, while data services function as value conduits channeling labels to be utilized by business stakeholders. These labels, derived from original data processing, directly empower business operations and drive business value creation. By comprising a label and a corresponding label value attributed to a specific object, a label streamlines data interpretation and utilization.

The utility of labels has expanded beyond the realm of the internet, reaching diverse sectors and encompassing varied subjects, from users and products to channels and marketing initiatives. In the digital domain, labels facilitate precise marketing endeavors, targeted messaging, and enhanced user engagement. In industrial realms, labels contribute to strategic categorization, intelligent search mechanisms, operational optimization, targeted marketing campaigns, service refinement, and intelligent managerial practices.

Categorically, data labels can be classified into fact labels, statistical labels [1], and algorithmic labels, which are aligned with factual categorizations, rule-based classifications, and model-driven analyses, respectively. Amidst the era of big data proliferation, data labeling techniques are swiftly evolving into intelligent labeling practices [2], increasingly harnessing machine learning and artificial intelligence methodologies to automate and refine the labeling process. Intelligent labeling encompasses fact labeling based on attribute relationships, rule labeling driven by predefined logic, and model labeling guided by algorithmic frameworks [3].

Data labeling is a key step in the field of machine learning and artificial intelligence with the application of data labels. Data labeling is divided into many types, usually classification labeling, bounding box labeling, segmentation labeling, key point labeling, sentiment labeling, semantic role labeling, sequence labeling, and so on.

This paper primarily focuses on intelligent labeling within distributed data center environments, highlighting two key challenges within this context:

(1)

The consistency of distributed intelligent labeling.

(2)

In a distributed annotation system, security problems such as network attacks and privacy disclosure easily occur during the process of data transmission.

Since data labeling requires a large number of samples, there are data privacy concerns when labeling data. Federated learning methods have been deeply studied in privacy preservation. For example, secure multiparty computing (SMC), homomorphic encryption (HE), differential privacy (DP), and trusted execution environments (TEE) are some of the techniques that play an important role in protecting data privacy. Federated learning, as a distributed machine learning method, allows for multiple devices or data sources to collaboratively train a model while maintaining data privacy locally, as the saying goes, “the data does not move, the model moves”. It can effectively protect data privacy, improve system flexibility, adapt to the characteristics of data distribution in a distributed environment, and improve network efficiency. At the same time, in the distributed intelligent labeling system, the asynchronous federal learning method can improve the efficiency of intelligent labeling model updating and reduce the communication cost and time; each device can be fine-tuned according to its own use of the environment and characteristics of the model. To improve the personalization of the model, the model response is made faster and more fault-tolerant, and provides higher utilization of resources, making for an excellent choice of distributed training methods. However, at the same time, federated learning privacy preservation poses some new challenges, such as the trade-off between computational efficiency, model accuracy, and privacy preservation, and how to maintain model performance while guaranteeing privacy. Few studies have combined federated learning privacy preservation with digital twin technology, which protects data privacy as well as security by utilizing prior knowledge to motivate federated learning models thereby enabling further performance enhancement of the models, hence the reason for this paper’s research.

The initial exploration concerning the challenge of distributed consistency involved the implementation of the Paxos algorithm to establish a framework that prioritizes achieving maximal consistency within distributed systems, albeit with a minimal probability of inconsistencies persisting [4]. Raft, a simplified rendition of Paxos, emerged as a refined paradigm in subsequent research endeavors [5]. Conversely, the Byzantine problem, initially conceptualized by Leslie Lamport in 1982, delves deeper into the nuances of attaining consensus within systems under adversarial conditions where select nodes may exhibit malicious behavior, thereby warranting a worst-case guarantee analysis.

To address the formidable consistency conundrum, innovative endeavors such as the PoW (proof of work) algorithm within blockchain design were introduced to impose economic disincentives on potential disrupters, reinforcing the pursuit of optimal consistency standards [6]. Despite ongoing advancements in algorithmic refinement, achieving an ideal solution to the consistency dilemma remains elusive. Thus, a pertinent question arises: how can algorithmic design expedite progress toward achieving near-ideal consistency levels?

Earlier studies often overlooked the symbiotic relationship between distributed systems and model interactions, notably neglecting integration with intricate models such as graph convolutional networks (GCNs) [7]. In contrast to traditional approaches, this paper focuses on leveraging previous findings on GCNs to propel advancements in system consistency [8]. Through rigorous simulation experiments, the efficacy of an intelligent annotation methodology anchored in GCN theory is substantiated, yielding a commendable consistency rate exceeding 95%. However, careful scrutiny of the experimental outcomes revealed a substantial lack of stability, as evidenced by pronounced fluctuations in the consistency curve.

Recent scholarly discourse has witnessed a burgeoning interest in the realm of digital twin (DT) technology [9], emphasizing the pivotal role of prior DT knowledge in fortifying overall model integrity and stability. Essentially a digital replica mirroring physical entities or environments, digital twins empower organizations to simulate real-world scenarios, engendering informed decision-making paradigms [10]. The integration of digital twin attributes, such as cost efficiency, precision, and adaptability, into intelligent annotation systems has a transformative impact on the fortification of distributed intelligent annotation systems while optimizing costs and augmenting data value [11].

In addition, the distributed data annotation raises several ethical and privacy concerns. One key issue is ensuring data anonymity, as even aggregated data can potentially reveal sensitive information. Additionally, the quality and fairness of annotations can be impacted by biases inherent in decentralized sources, leading to biased models. Ensuring transparency in how data are used and consent from participants is crucial. Moreover, robust security measures are needed to protect against data breaches and misuse.

The proposed research delineates a strategic path forward, envisaging the amalgamation of digital twin technology into the distributed intelligent annotation architecture as a potent solution to the identified challenges.

To address the consistency challenges inherent in intelligent labeling within distributed data centers while simultaneously ensuring robust security and privacy safeguards, it has become evident that the efficacy of the graph convolutional network (GCN) method alone may not suffice to consistently enhance system integrity, leading to observed instabilities. This study endeavors to elevate the standard of consistency by integrating the paradigm of digital twin mechanisms and leveraging the dynamic interplay between theoretical ideals and practical realities. Specifically, within the context of federated learning frameworks, we propose a novel approach that amalgamates digital twin (DT) and GCN models. This entails the deployment of a GCN model at each local secondary center, complemented by a DT model tasked with information exchange. Moreover, a contract-based reputation mechanism is designed to assess model fairness and bolster system consistency.

To further improve system security and privacy, we introduce a blockchain consensus methodology, which serves as a foundational framework to ensure comprehensive data integrity and confidentiality. Through this holistic approach, we aim to forge a robust and reliable system architecture that not only accentuates consistency but also upholds the stringent security and privacy imperatives essential for contemporary distributed data environments.

Federated learning, as a distributed machine learning method, allows for multiple devices or data sources to collaboratively train a model while maintaining data privacy locally, as the saying goes, “the data does not move, the model moves”. It can effectively protect data privacy, improve system flexibility, adapt to the characteristics of data distribution in a distributed environment, and improve network efficiency. At the same time, in the distributed intelligent labeling system, the asynchronous federal learning method can improve the efficiency of intelligent labeling model updating and reduce the communication cost and time. Each device can be fine-tuned according to its own use of the environment and characteristics of the model to improve the personalization of the model. This makes the model response faster and more fault-tolerant, and results in higher utilization of resources, making for an excellent choice of distributed training methods.

Our innovations and contributions are as follows:

(1)

To address the issues of poor data quality and the inability to introduce prior knowledge in a data-driven context, this paper proposes combining digital twins (DTs) with graph convolutional networks (GCNs) to design a hybrid and interpretable reputation assessment method. This approach allows for the incorporation of prior knowledge and enhances the model’s fairness based on data-driven conditions.

(2)

To enhance the fairness of federated learning, this paper introduces a joint model of digital twins (DTs) and graph convolutional networks (GCNs) into federated learning (FL). It measures the fairness of federated learning through a contract-based reputation incentive mechanism and ultimately improves the consistency of distributed models.

(3)

This paper introduces the method of blockchain integration in the context of federated learning (FL). It combines blockchain with the designed DT+GCN model to enable secure information exchange in FL scenarios.

(4)

Finally, through real-data experiments, we provide quantitative and qualitative analyses, demonstrating the potential benefits of our method in addressing contemporary and future challenges in the data middleware domain.

2. Related Work

This section delves into a comprehensive analysis of the research landscape encompassing digital twin technology, graph neural network technology, and federated learning. Initially, this study scrutinizes the evolution and utilization of digital twin technology within the domain of artificial intelligence information interaction, elucidating the synergistic potential unlocked through the integration of digital twin and blockchain technologies to amplify data interaction capabilities and efficiency within distributed scenarios. Subsequently, the interpretability facets of GCNs in recommendation systems are meticulously examined.

Moreover, a detailed exploration ensues regarding the current deployment and evaluation of incentive mechanisms and reputation frameworks within the framework of federated learning, alongside an investigation into their amalgamation with blockchain technology for enhanced security analysis. Finally, a comparative analysis delineates the distinctive attributes of our research vis-à-vis prior works, shedding light on the unique contributions and advancements brought forth by this study.

2.1. Digital Twins

Since its inception by Professor Grieves in 2003, digital twin technology has garnered increasing attention across diverse domains. Within the realm of digital twins, Wang Jing, Li Xinchun, and colleagues [12] conducted a thorough examination of the digital twin literature, delineated the influencing factors and conceptual framework, pinpointed pivotal contributions within the digital twin literature, highlighted research constraints, and outlined directions for future investigations.

In a distinct vein, Wang Baicun et al. [13] delved into an extensive exploration of the human digital twin landscape against the backdrop of Industry 5.0, proposing a nuanced three-dimensional human digital twin model comprising human entities, virtual entities, and interactive systems, exemplifying the efficacy of digital twin technology within interactive systems. Kobayashi, K. and collaborators [14] harnessed deep neural operators to bolster the generalizability of engineered systems, showcasing the prowess of operator learning methodologies in effectuating numerical approximations within intricate physical or complex spaces characterized by substantial parameter interactions.

Moreover, Xin Liu, Jiang Du, and colleagues [15] undertook a systematic review of digital twins pertaining to physical entities, virtual models, twin data, and application frameworks. Clarifying the digital twin concept, they elucidated the fundamental steps in virtual model construction grounded in the virtual model paradigm while encapsulating digital twin data acquisition and exchange protocol methodologies.

Additionally, Richard J. Somers et al. [16] integrated digital twins into physical systems, enriching these systems with data-driven and simulation-informed models interfaced with physical entities to furnish visualization capabilities, prognostic insights into future states, and enhanced communication channels. Notably, their work informs our exploration of information interaction among localized models in federated learning settings. Zhixiu Yao et al. investigated a graph attention multi-agent reinforcement learning method (GatMARL) [17], which aims to maximize service quality by combining digital twin technology with multi-cell MEC networks.

Similarly, Liu Shimin, Lu Yuqing, and collaborators [18] underscored the application of blockchain-based interaction methodologies within digital twin-centered manufacturing systems, positing that this approach can alleviate burdens on enterprise cloud infrastructures while concurrently augmenting data interaction capabilities and efficiencies. This proposition resonates with the core tenets of our study. Nevertheless, existing research predominantly concentrates on digital twin technologies and has yet to integrate graph neural networks and other cutting-edge technologies into their analyses.

2.2. GCN

In traditional graph convolutional neural network models, the adjacency matrix generated based on prior knowledge cannot reflect the actual spatial dependencies between stations. To address this issue, Zhao, J. et al. [19] proposed an adaptive graph convolutional network model (Adapt-GCN). This model can effectively adjust the correlation weights between adjacent stations and capture the spatial dependencies between nonadjacent stations. To address the challenges of error propagation and weak interpretability when integrating knowledge graphs into recommendation systems, researchers have proposed a new method called the knowledge-aware reasoning and graph convolution network (KR-GCN) [20]. This method introduces the GCN into recommendation systems, integrating user–item interactions and knowledge graphs into a heterogeneous graph. By utilizing graph convolutional networks with a path-level self-attention mechanism for discernment, this approach aims to enhance the relevance of the final explanations. Wu, W. and colleagues [21] introduced a novel shadow annotation strategy called Anno-GCN, which is based on graph convolutional networks. This strategy leverages additional GCN-labeled data and knowledge distillation to improve the performance of lightweight models. They utilize zero-shot learning to predict unseen categories, which helps address challenges such as dealing with unforeseen categories during training and insufficient annotated datasets. K Shiba et al. [22] proposed a method to extract useful information from the whole graph by using the position relationship between categories and edge types in the knowledge graph. Yang, L and colleagues [23] proposed a deep reinforcement learning solution based on graph convolutional networks (GCN-based DRL). By integrating the GCN into deep reinforcement learning (DRL) to capture the spatial dependencies of the network, they aimed to enhance the generalization performance of the proposed method. Yang, Y et al. [24] introduced a new interactive recognition network for cows, named SCIRNet, which constructs interbody graphs from the skeletal graphs of individual cows. They also introduced a multistream architecture that considers relative information between interacting cows to improve accuracy. In practice, they combined graphical features with image features extracted from interaction regions to capture visual representations of interaction areas, along with semantic priorities obtained from the dataset, to incorporate prior knowledge about the relationships between actions and interactions in cows. To address the lack of understanding and awareness of prior knowledge in the existing GCN network, they combined GCN with decision trees (DTs) to enhance fairness and increase the focus on prior knowledge research.

2.3. Federated Learning

The incentive mechanism in federated learning can be designed by multiple factors, and there are three main considerations: auction theory, game theory, and blockchain. In recent years, blockchain technology has seen rapid development and received widespread attention. Blockchain is mainly studied from the aspects of reputation management, security, and gradient update records. Specifically, the incentive mechanism can be designed from three aspects: one is based on a customer data contribution-driven incentive mechanism design, the other is based on a customer reputation-driven incentive mechanism design, and the third is based on a customer resource allocation-driven incentive mechanism design. Specifically, data contribution can be quantified in terms of “limitations”, including data quality and data size. Incentive design based on customer data contribution-driven methods from a data quality perspective includes the Shapley value, auction mechanism, contract theory, and mismatch. In terms of data quantity, we consider noncooperative games, deep reinforcement learning, cooperative games, security blockchains, and audit models. Finally, considering the heterogeneity of clients, parameter servers need to flexibly allocate communication and computing resources to maximize platform benefits.

For federated learning, the cost of additional communication and computing resources added by clients and aggregators has been a major concern of researchers. AS Parihar et al. also focused on resource allocation in distributed architectures [25]. They proposed a token-based resource allocation method for FANET that was implemented for dynamic behavioral node access under distribution. At the same time, due to the existence of negative events such as cyber attacks, there may be a risk of insecure privacy disclosure. Unlike the trust mechanism in traditional society, which relies on an “acquaintance society”, and the trust mechanism of the industrial era, which relies on specialized intermediaries, the trust mechanism of the digital age relies on blockchain technology. Blockchain is a machine of trust, providing a trusted distributed ledger that is shared and maintained by partners, who can conduct transactions based on a “machine of trust” even if they are not familiar with each other.

Blockchain technology has unique and distinct characteristics. First, blockchain refers to peer-to-peer networks that establish a multiparty collaborative digital network, which can ensure that participants can equally participate in social network cooperation. Second, the reliability of the source of transaction data is guaranteed through the verification of the cryptographic digital signature of the transaction party. Third, blockchain introduces a distributed database, chain data structure, hash algorithm, and other technologies through multiparty participation and verification to improve the transparency of transaction records, and cannot be tampered with so as to ensure security and credibility. Fourth, the consensus algorithm is introduced to achieve consistent trading results based on consensus rules and ensure the credibility of trading rules. Finally, smart contract technology is introduced through the computer system to execute the contract according to the agreed-upon content to ensure the trust of the transaction contract. In the above aspects, many scientists have devoted considerable manpower and resources to researching key blockchain issues. A. Ahmed et al. [26] proposed a fair and reliable federated learning incentive mechanism by proposing a federated learning incentive scheme based on reverse auctions and trust reputation to select reliable customers and fairly reward customers with limited budgets. The key is that reputation can reflect credibility and reliability. Yang Danni, Ji Yun et al. [27] proposed an incentive mechanism based on contract theory, which is applied to asynchronous federated learning. By adaptively adjusting the local model training cycle, the delay can be reduced, and the accuracy of the test can be improved to maximize the utility. At the same time, Ao Xiong and Chen Yu et al. [28] studied the true and reliable federated learning incentive mechanism based on the reputation mechanism and reverse auction. This incentive mechanism can not only encourage more customers with high data quality and a high reputation to participate in federated learning at a lower cost but also increase the economic benefit of federated learning tasks by 31% and the accuracy rate by 5%. It reached 98%. At present, the 6G era has gradually entered the trend of the times, and the dynamic nature of data has become increasingly prominent. Zhu Ye and Liu Zhiqiang et al. [29] proposed a dynamic incentive and reputation mechanism to improve the energy efficiency and training performance of federated learning. The Stackelberg game-based incentive method has been widely studied because it can adjust the optimal energy consumption in time according to the changes in available clients during the federated learning process. Incentive methods based on Stackelberg games have also attracted the attention of other famous researchers. Wang Zhilin et al. [30] studied the key role of blockchain in federated learning and proposed a blockchain-based incentive mechanism for federated learning joint resource allocation. Chen Yuming, Zhou Hui et al. [31] proposed a Stackelberg game-based FL multifactor incentive mechanism (SGMFIFL), and they designed a multifactor reward function based on reputation, accuracy and reward rate. High-reputation and highly accurate data owners receive more rewards; J Kang et al. [32] introduced reputation as a measure of mobile device reliability and trustworthiness, and at the same time, they designed a reputation-based worker selection scheme for reliable federated learning. Yu Hai et al. [33] designed an equity-conscious federarated learning incentive program. Q Song et al. [34] proposed a federated learning model based on trust aggregation under a new architecture of digital twin-enabled industrial IoT to ensure high-performance customer contributions. Therefore, on the basis of the above research, this paper integrates digital twins with federated learning and introduces blockchain for integration expansion and research.

3. Model

This section aims to clarify our interpretation and perception of the basic structure, establishing a coherent basis for subsequent elaboration of the core principles of the proposed architecture. The conceptual model is shown in Figure 1.

3.1. DT+GCN

To fill the gaps, DTs are used to achieve dynamic awareness of complex IoT environments and enhance the interconnection between heterogeneous mobile clients, as shown in Figure 2. By combining the original GCN model with DTs, interpretability can be realized, and prior knowledge can be introduced, which enhances the fairness of model information exchange in the case of a new credit value.

3.1.1. DT System Model

The smart marking digital twin is a system model that combines digital twin technology and smart marking technology to aid in and optimize the data annotation process. Digital twin technology uses digital models to simulate actual systems, thereby improving system understanding and efficiency. Smart marking technology, on the other hand, utilizes artificial intelligence techniques to automatically or semi-automatically perform data annotation. Together, these two technologies can improve the speed and accuracy of data annotation, thereby accelerating the process of training and optimizing machine learning models. The specific steps of the digital twin system model are as follows:

(1)

Data collection: the system collects various types of data such as environment, devices, and user behavior in real time through sensor networks and IoT devices.

(2)

Data fusion and processing: the distributed architecture enables data to be processed in parallel on multiple nodes, and a large quantity of data is preprocessed, cleaned, and analyzed through algorithms such as machine learning (e.g., deep learning).

(3)

Digital twin construction: based on the collected data, a virtual, real-time updated physical system model is constructed, which reflects the physical state and behavior of the real world.

(4)

Intelligent labeling: using the trained model, automatically identify and classify data, and label each entity or behavior.

(5)

Decision support and optimization: based on the feedback from marking, the system can provide real-time decision support, optimize production processes, maintenance strategies, etc., and perform predictive maintenance.

(6)

Feedback closed loop: the results of actual operation will further update the digital twin model, forming a closed loop of continuous learning and improvement.

DTs: The secondary data center participating in the training is represented as $N=\left\{1,2,\dots ,n\right\}$. Therefore, the client DTs model $D{T}_i^c$ corresponding to the secondary data center $i$ is represented as follows:

$$D{T}_i^c\left(t\right)=\left\{f_i^t\left(x\right),k_i\left(t\right),c_i\left(t\right)\right\}$$

(1)

where $f_i^t\left(x\right)$ represents the class score that client $i$ sends to the server at time $t$; $k_i\left(t\right)$ represents the computing power that client $i$ sends to the server at time $t$ and represents the energy consumption generated by a round of training for the client at time $t$. In the realm of federated learning, the computational capabilities of participating devices serve as a critical resource constraint, exerting a significant impact on the convergence rate and training expenses. In addition, we use the packet loss rate ${\tilde{c}}_i\left(t\right)$ and CPU frequency deviation ${\tilde{k}}_i\left(t\right)$ to measure the DT error and calculate the power in the communication environment. On one front, network congestion may trigger an elevated packet loss rate, necessitating the retransmission of packets and consequently elevating the energy consumption levels within the system. On the flipside, the CPU frequency stands out as a pivotal factor in dictating the velocity of computational prowess. In general, computing power escalates within a specific range as the CPU frequency rises. This progression culminates in the acquisition of a finely calibrated DTs model:

$$D{T}_i^c\left(t\right)=\left\{f_i^t\left(t\right),k_i\left(t\right)+{\tilde{k}}_i\left(t\right),c_i\left(t\right)+{\tilde{c}}_i\left(t\right)\right\}$$

(2)

By harnessing the capabilities of the client DT, vital physical entity data are preserved and transmitted to the aggregator, facilitating a self-calibration process that hinges on error mapping within the DT. This approach aids in curbing data inaccuracies and in iteratively fine-tuning the system for optimal performance. The aggregator side $DTs\left(t\right)$ retains the DT error set resulting from the discrepancy between the client’s local update and the global update, along with the distribution of the client’s reputation across the network coverage. The local loss function, incorporating a weighted cosine similarity value, computes the loss function, guiding the aggregator in iteratively updating and training the global model until it converges to the minimum loss.

3.1.2. Historical Opinion Model Based on GCN

Our previous research focused on the process of intelligent annotation based on a GCN, and the results showed that the consistency of distributed intelligent annotation can only be maintained at least 95%, but it has no greater effect. In this paper, it is used as a historical model to combine with the current model to form recommendations. Specifically, each local data center trains the GCN model, and we set the credit value for each secondary station $reputation=\left\{attention,parameter,MAE\right\}$. The weights of the corresponding coefficients in $reputation$ are ${\psi }_1$, ${\psi }_2$, and ${\psi }_3$, where ${\psi }_1+{\psi }_2+{\psi }_3=1$. For the first secondary data center, its similarity to the previous state is as follows:

$$SIM\left(L_1,J_1\right)=1-DISS=reputatio{n}_1$$

(3)

For different secondary data centers, the composition of reputation is different. Take a secondary data center as an example:

(1)

Attention 1: The 10 feature weights formed by private data are equivalent to the topology formed by the connections between 10 nodes, thus generating attention. The greater the attention is, the better the reputation.

(2)

Parameter 1: Private model parameters trained by private subsidiary 1 data. The higher the parameter is, the better the reputation.

(3)

The data of the verification set are substituted into the trained private model, and the difference value MAE11 is obtained.

Thus, $DISS\left(L_1,J_1\right)$ is the normalized difference between subsidiary 1 and its previous state, defined as follows:

$$DISS\left(L_1,J_1\right)={\psi }_1\ast attentio{n}_1+{\psi }_2\ast paramete{r}_1+{\psi }_3\ast MA{E}_{{11}^{\prime }}=reptutatio{n}_1$$

(4)

Reputation2 consists of the following parts:

(1)

Substitute private data 1 into $MA{E}_{d12}$ of the private data obtained by private model 2.

(2)

$MM{D}_{1\&2}$ between public data 1 and public data 2.

(3)

The difference between parameter 1 and parameter 2 is $MA{E}_{p1\&2}$.

Similarly, based on the similarity between subsidiary 1 and subsidiary 2,

$$SIM\left(L_1,J_1\right)=1-DISS\left(L_1,L_2\right)$$

(5)

where $DISS\left(L_1,L_2\right)$ is the normalized difference between subsidiary 1 and subsidiary 2, defined as follows:

$$DISS\left(L_1,L_2\right)={\psi }_1\ast MA{E}_{d12}+{\psi }_2\ast MM{D}_{1\&2}+{\psi }_3\ast MA{E}_{p1\&2^{\prime }}=reputatio{n}_2$$

(6)

For subsidiary 1, the final reputation is as follows:

$$Globa{l}_1\ast finalre{p}_1=reputatio{n}_1\ast mo{d}_1+reputatio{n}_2\ast mo{d}_2+\dots +reputatio{n}_6\ast mo{d}_6$$

(7)

$$w^{\ast }=\underset{w\in R^L}{\arg \min }f=\underset{w\in R^L}{\arg \min }{\displaystyle {\sum }_{i\in V}{f}_i}$$

(8)

The forecast output of the model is Pr. We consider the forecasts of labeled data and unlabeled data at the same time and calculate the loss through the mean absolute error (MAE) as the minimized loss function:

$$loss={\displaystyle \frac{1}{L}}{\displaystyle \sum _{i=1}^{L}loss\left(y,\mathrm{Pr}\right)+}{\displaystyle \frac{1}{U}}{\displaystyle \sum _{i=1}^{L+U}loss\left(\widehat{y},\mathrm{Pr}\right)}$$

(9)

$$MAE\left(X,h\right)={\displaystyle \frac{1}{m}}\left|h\left(x_i-y_i\right)\right|$$

(10)

It should be noted that there are components in our explainable framework under a GCN: (i) accurate predictions; (ii) regular terms (but fit); (iii) smooth terms.

$$f_i={\displaystyle \frac{1}{2}}{‖L_i^{{\displaystyle \frac{1}{2}}}\left({\widehat{y}}_i-{\tilde{y}}_i\right)‖}^2+{\displaystyle \frac{x_i}{2}}{‖w‖}^2+{\displaystyle \frac{{\eta }_i}{2}}Tr\left({\widehat{y}}_i{ }^T{\tilde{L}}_i{\widehat{y}}_i\right)$$

(11)

Graph neural network interpretability can be predicted by an interpretable predictor, or interpretability can be enhanced by an attentional mechanism. The latter will be set to predict a node’s labeling for an edge, but this labeling cannot jointly consider the graph structure and node feature information to interpret the prediction. We mainly take the interpreter as an example here to describe the interpretable prediction process in detail as follows:

The interpreter predicts by generating explanations to identify the most influential model in a subgraph of the computational graph with a subset of node features. Given a node i, specifying that the graph is G-represented, and the goal is to identify subgraphs Gs and features $N_S=\left\{\left.n_j\right|i_j\in G_S\right\}$ that use information M to bring out the importance, the optimization framework of the interpreter:

$$\max M\left(Y,\left(G_S,N_S\right)\right)=H\left(Y\right)-H\left(\left.Y\right|G=G_S,N=N_S\right)$$

(12)

3.1.3. Global Model

The interaction model of DTs and GCNs can enhance the interpretability of the model: using the original and current interactions of DTs, we can set the DT in this paper to the original interaction and the GCN to the current interaction. Therefore, the final global model is as follows:

$$Global\text{\_}{T}_c^j\left(i\right)=w_{GCN,t}\cdot GCN\text{\_}{T}_c^j\left(i\right)+w_{DT,t}\cdot DT\text{\_}{T}_c^j\left(i\right)$$

(13)

3.2. TFL-DT: Trust-Based Reputation (Evaluation/Calculation)

In the distributed scenario, the transmission process of parameters between models will have the same cost, privacy, and data security problems. Therefore, combining DT with FL forms distributed federated learning (DFL). In each round of communication, the secondary data center is trained locally based on its own dataset. Each secondary data center then achieves model aggregation through P2P communication of the model updates received from the neighboring data center to achieve consensus on the global update. Therefore, our main work in this section is to leverage a new incentive-based federated learning framework that incorporates a contract-based reputation mechanism that can optimize the accuracy of the model. At the same time, federated learning technology is refined to process non-IID data, which facilitates information interaction and data fusion in distributed scenarios and improves distributed consistency.

3.2.1. Federated Learning Framework

First, since digital twin technology is carried out in a distributed situation, and at the same time, it is necessary to consider the security and privacy protection of the system after the introduction of blockchain, this paper is considered to be a digital twin in the federated learning scenario, and the following should first determine the concept representation of parameters in federated learning: First, combined with the application scenario of this paper, federated learning trains a single shared model separately in a distributed data center and minimizes the global objective function $\min F\left(m\right)$, where $m$ represents the model parameters, and the global model loss function $F\left(m\right)$ is the average of the loss function $F_i\left(m\right)$ for each local $F\left(m\right)={\displaystyle \frac{1}{n}}{\displaystyle {\sum }_{i=1}^n{F}_i\left(m\right)}$. First, the local secondary data center trains the private data to generate a local model. Where the private data are represented as $\left(X_k,Y_k\right)$, they come from different data distributions of $k$ secondary data centers $P_k\left(x,y\right)$. In addition, parameter $m_k\in R_d$ of the trained local model uploaded by each secondary data center is sent to the primary data center for aggregation of the global model. Then, the local secondary data center is locally updated, and the gradient method is constantly iterated to optimize the global model. Here, $n_k$ is defined as the number of local samples, $\eta$ represents the learning rate, and $\eta \nabla F_k\left(m_k\right)$ represents the gradient vector. Then, the federated learning loss function and the global aggregate optimization local objective function are as follows:

$$F_i\left(m^k\right)={\displaystyle \frac{1}{n_k}}{\displaystyle \sum _{i\in P_i}{f}_i\left(m^k\right)}$$

(14)

$$m^k\leftarrow m^k-\eta \nabla F_k\left(m^k\right)$$

(15)

In this section, we focus on a federated learning framework based on incentives, in which we refer to a contract-based approach to reputation mechanisms, where incentives are primarily based on Stackelberg’s interclient incentives. At the same time, a reputation model including direct trust and indirect trust is designed. Direct trust is computed based on the interactive engagements between employees and customers, with the quantity of interactions denoted by T. Indirect trust considers the impact $D_k$ of the quality of the data provided by the worker on the generated local datasets.

Step 1: System initialization.

In this step, the system initializes the model parameters and reputation values for each participant (e.g., clients and servers).

Step 2: Local model training.

Individual clients utilize local data for model training, enhancing both the personalization and adaptability of the model.

Step 3: Trust-based global model aggregation.

In this step, leveraging trust value computations, the server aggregates the model parameters from each client to derive a unified global model.

Step 4: Trust calculation.

Through direct and indirect trust, the system calculates the reputation value of each client, which helps to assess its contribution and trustworthiness.

Step 5: Trust update.

Finally, the system updates its reputation value in preparation for the next round of model aggregation and incentive calculation.

3.2.2. Trust Behavior Model

In this section, to evaluate the trust level a user places in a virtual twin for engagement in distributed learning, it is imperative for the virtual twin to comprehensively document the user’s behaviors throughout the distributed learning process. First, the following parameters are specified: the number of virtual twins in the system, expressed as the number of user devices in the system, and $v{t}_j$, which represents the identity of the $j$ virtual data twin. The trust evidence of the user recorded under context c can be formalized as four times, as shown in Equation (15). Here, $B{M}_i^c$ is an ordered set representing the behavior of a virtual twin in context c, recording the behavior of $u_t$ when interacting with $v{t}_j$, and $R{L}_i^c$ is a set recording the trust of $u_i$ recommended by other virtual twins in context c. Here, $u_i$’s interaction with $v{t}_j$ means that $u_t$ participates in distributed learning in $v{t}_j$. The proposed trust calculation function is expressed as follows:

$$T{E}_{i,j}^c=〈B{M}_i^c,R{L}_i^c,L\text{\_}T{P}_i^c,NL\text{\_}T{P}_i^c〉$$

(16)

In the trust calculation function proposed above, $v{t}_j$ can obtain $u_i$’s local trust profile (denoted as $L\text{\_}T{P}_i^c$) and nonlocal trust profile (denoted as $NL\text{\_}T{P}_i^c$). Due to the limitation of storage resources, we assume that the maximum length of $B{M}_i^c$ is G and that the length of $B{M}_i^c$ is denoted as $i{g}_{i,c}$. $b{h}_{i,k}^c$ is each element in $B{M}_i^c$ that records the behavior of $u_i$ interacting with $v{t}_j$ for time $k$. $ac{c}_{i,k}$ represents the degree of anomaly, and $dela{y}_{i,k}$ represents the delay in uploading the local model after iterating the behavior of the $k$ interaction between $u_t$ and $v{t}_j$. It is represented by Equation (6). $ac{c}_{i,k}$ indicates that virtual twins test the anomalies of the trained local models uploaded by each secondary data station through the local model anomaly detection method. The smaller the $ac{c}_{i,k}$ value is, the more abnormal the detection result. The more abnormal the test results are, the greater the $ac{c}_{i,k}\in \left[0,1.\right]$

$$B{M}_i^c=〈b{h}_{i,k}^c〉,i\in \left[1,M\right],k\in \left[1,{\lg }_{i,c}\right]$$

(17)

$$b{h}_{i,k}^c=〈u_i,c,ac{c}_{i,k},dela{y}_{i,k}〉$$

(18)

where $R{L}_i^c$ represents the trust set of the context and $r{l}_{k,c}^i$ represents the trust of virtual twin $v{t}_k$ in user device $u_i$.

$$R{L}_i^c=\left\{\left.r{l}_{k,c}^i\right|k\ne j\right\}$$

(19)

$$r{l}_{k,c}^i=〈v{t}_k,r{t}_{i,k}^c,h_{i,k}^c,t〉$$

(20)

where $recom\text{\_}{T}_c\left(i\right)$ represents the recommended trust value for $u_i$ in the context. The calculation is based on $u_i$’s behavior when interacting with $v{t}_j$, and $recom\text{\_}{T}_c\left(i\right)$ is obtained according to the recommended trust information of other twins $u_j$.

$$T_c^j\left(i\right)=w_{l,i}\cdot local\text{\_}{T}_c^j\left(i\right)+w_{r,i}\cdot recom\text{\_}{T}_c^j\left(i\right)$$

(21)

Equation (20) shows the function of virtual twin $v{t}_j$’s global trust value for user device $u_i$ in context c. $T_c^j\left(i\right)$ is related to the local trust value and the recommended trust value and is multiplied by the corresponding weight. The local trust value $local\text{\_}{T}_c^j\left(i\right)$ is related to two factors. One is related to the trust value of virtual twin $v{t}_j$ interacting with $u_i$ during direct interaction. The other factor has to do with historical interactions, which is $hist\text{\_}{T}_c^j\left(i\right)$. Therefore, the local trust value $local\text{\_}{T}_c^j\left(i\right)$ is expressed as follows:

$$local\text{\_}{T}_c^j\left(i\right)=w_{curr,i}\cdot curr\text{\_}{T}_c^j\left(i\right)+w_{hist,i}\cdot hist\text{\_}{T}_c^j\left(i\right)$$

(22)

3.2.3. Combining Local and Recommended Views

After receiving the shared data from the data provider, the data requester has a subjective opinion (i.e., a local opinion) about each data provider based on the interaction history. When forming the final opinion, the local opinion should still be taken into account. In this section, we combine the recommended opinion in summary B with the local opinion in section c to infer the final result. The specific process is as follows: In the direct interactive reputation opinion, the task publisher evaluates the reputation of the worker:

$$b_{i\to j}^{t_y}+d_{i\to j}^{t_y}+u_{i\to j}^{t_y}=1$$

(23)

where $b_{i\to j}^{t_y},d_{i\to j}^{t_y},u_{i\to j}^{t_y}\in \left[0,1\right]$

$$\begin{array}{c}{b}_{i\to j}^{t_y}=\left(1-u_{i\to j}^{t_y}\right){\displaystyle \frac{{\alpha }_i^{t_y}}{{\alpha }_i^{t_y}+{\beta }_i^{t_y}}}\\ d_{i\to j}^{t_y}=\left(1-u_{i\to j}^{t_y}\right){\displaystyle \frac{{\beta }_i^{t_y}}{{\alpha }_i^{t_y}+{\beta }_i^{t_y}}}\\ u_{i\to j}^{t_y}=1-q_{i\to j}^{t_y}\end{array}$$

(24)

where $b,d,u$ are the trusted, untrusted, and uncertain number of positive or negative interactions during time periods $\alpha$ and $\beta$, respectively, and $q$ is the data transmission probability. Therefore, direct reputation can be expressed:

$$T_{i\to j}^{t_y}=b_{i\to j}^{t_y}+a{u}_{i\to j}^{t_y}$$

(25)

where $a\in \left[0,1\right]$ represents the coefficient of the degree of residual effect of uncertainty. After the interaction effect is generated, the weight of the positive interaction is m, and the weight of the negative effect is n ($m\le n$ and $m+n=1$).

$$\begin{array}{c}{b}_{i\to j}^{t_y}=q_{i\to j}^{t_y}{\displaystyle \frac{m{\alpha }_i^{t_y}}{m{\alpha }_i^{t_y}+n{\beta }_i^{t_y}}}\\ d_{i\to j}^{t_y}=q_{i\to j}^{t_y}{\displaystyle \frac{n{\alpha }_i^{t_y}}{m{\alpha }_i^{t_y}+n{\beta }_i^{t_y}}}\\ u_{i\to j}^{t_y}=1-q_{i\to j}^{t_y}\end{array}$$

(26)

Indirect reputation opinion is determined by using the modified cosine function to define the similarity factor between task publishers $i$ and $x$:

$$Sim\left(i,x\right)={\displaystyle \frac{{\displaystyle {\sum }_{j\in c}\left(D_{i\to j}-{\overline{D}}_i\right)\left(D_{x\to j}-{\overline{D}}_x\right)}}{\sqrt{{\displaystyle {\sum }_{j\in N}\left(D_{i\to j}-{\overline{D}}_i\right)\sqrt{{\displaystyle {\sum }_{j\in X}{\left(D_{x\to j}-{\overline{D}}_x\right)}^2}}}}}}$$

(27)

where $D_{i\to j}$ represents the reputational opinion of $i$ to $j$, ${\overline{D}}_i$ represents the mean value of $i$’s reputation opinion on C, $D_{x\to j}$ represents the mean value of $x$’s reputation opinion on $j$, and ${\overline{D}}_x$ represents the mean value of $x$’s reputation opinion on C. Furthermore, the total weight of referrer $x$’s indirect reputation opinion is calculated as follows:

$${\overline{\omega }}_{i\to x}={\epsilon }_{i\to x}\times Sim\left(i,x\right)$$

(28)

where $\epsilon$ is the predetermined parameter. We integrate the indirect reputation opinion into a global recommendation reputation opinion.

$$\begin{array}{c}{b}_{x\to j}^{rec}={\displaystyle \frac{1}{{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}}}}{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}{b}_{x\to j}}\\ d_{x\to j}^{rec}={\displaystyle \frac{1}{{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}}}}{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}{d}_{x\to j}}\\ u_{x\to j}^{rec}={\displaystyle \frac{1}{{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}}}}{\displaystyle \sum _{x\in X}{\overline{\omega }}_{i\to x}{u}_{x\to j}}\end{array}$$

(29)

The comprehensive reputation opinion is expressed as follows:

$$\left\{\begin{array}{c}{b}_{i\to j}^{final}={\displaystyle \frac{b_{i\to j}{u}_{x\to j}^{rec}+b_{x\to j}^{rec}{u}_{i\to j}}{u_{x\to j}^{rec}+u_{x\to j}-u_{x\to j}^{rec}{u}_{x\to j}}}\\ d_{i\to j}^{final}={\displaystyle \frac{d_{i\to j}{u}_{x\to j}^{rec}+d_{x\to j}^{rec}{u}_{i\to j}}{u_{x\to j}^{rec}+u_{x\to j}-u_{x\to j}^{rec}{u}_{x\to j}}}\\ u_{i\to j}^{final}={\displaystyle \frac{u_{x\to j}^{rec}{u}_{i\to j}}{u_{x\to j}^{rec}+u_{x\to j}-u_{x\to j}^{rec}{u}_{x\to j}}}\end{array}\right.$$

(30)

3.3. Consistency Checking under Blockchain

Reputation management within blockchain technology can be accomplished through the multi-powered subjective logic model for reputation computation or through federated blockchain technology to securely store reputations in a decentralized manner. Hence, in this section, we introduce blockchain technology that facilitates communication among DFL clients via blockchain ledgers for secure model exchange and aggregation. Blockchain technology offers transparency, decentralization, and sustained distributed data storage along with consistency verification to uphold data security and privacy. However, the challenges of ensuring reliable data storage, achieving traceability, and consistent verification of data interactions persist as vital research endeavors. To tackle these challenges, we propose the blockchain distributed consistency verification system (BDCVS) model, comprising four core components: the IoT device (DO), source chain (SC), target chain (TC), and audit chain (AC). The placement of IoT devices in each secondary data center, establishment of source and target chains in the physical model (GCN model training smart tag model) and mechanism model (digital twin model), and utilization of tagged public data in the audit chain are key aspects of the BDCVS model.

3.3.1. Consistent Goals under Blockchain

Our expected goals for the blockchain distributed consistent system are as follows:

(1)

Consistency: The BDCVS can ensure the consistency of data interactions between the source chain and the target chain. When the mechanism model where the TC resides does not update the data or is in a modified state, the model does not pass the AC consistency test.

(2)

Privacy: In the process of cross-chain information data interaction between SCs and TCs, any privacy information of DO cannot be obtained.

(3)

Dynamic: The auxiliary verification information form (AVF) is introduced to perform data update operations with minimal overhead and dynamic information interaction to improve the verification efficiency.

(4)

Security: By introducing the advanced gamma multisignature scheme AGMS, the system model encrypts the SC and TC between the two secondary middle stations, which can ensure the security of data update interaction and other operations.

The specific steps are as follows:

• System initiation

The IoT device DO in each secondary center in BDCVS generates public key and private key pair $\left\{pk=e,sk=\left(d,u\right)\right\}$ and outputs public parameter $par=\left\{g,e,N,H_1,\Gamma ,A\right\}$, where $e$ is the public key, $d$ represents the private key, computes RSA model $N=pq$, $H_1$ is the hash function, $H_1:{\left\{0,1\right\}}^{\ast }\to Z_N$, $\Gamma$ is a pseudorandom permutation $\Gamma :{\left\{0,1\right\}}^{k_1}\times {\left\{0,1\right\}}^{{\log }_2^n}\to {\left\{0,1\right\}}^{{\log }_2^n}$, and $A:{\left\{0,1\right\}}^{k_2}\times {\left\{0,1\right\}}^{{\log }_2^n}\to Z_N$ is a random function.

2.

Data processing

To protect the security and privacy of the original dataset $M$, DOS preprocesses the data of $M$, turns it into blind data $M^{\ast }$, generates authentication labels, and constructs the BV-MHT and auxiliary authentication information from the AVF. DOS uploads the processed dataset to SC. DOS built a batch-validated Merkle hash tree (BV-MHT) based on the processed data $M^{\ast }$ to facilitate efficient and secure storage. Then, the DOS construct validation auxiliary information table (AVF), which saves validation costs due to the number of important search nodes, realizes batch verification and speeds up the process of batch verification.

3.

Data uploading

SC encrypts uploaded content using advanced gamma multisignatures (AGMSs), ensuring security and privacy in cross-chain interactions between SC and TC and detecting unauthorized store access and tampering. DO passes the blind data and corresponding labels to the audit chain.

4.

Data updating

When DOS wants to update the data request in the SC at will, DOS generates a data update request and sends it to the SC, which acts as a request to req the data update and sends the data update request to the TC. When the SC receives a data update request from the DOS, the SC updates and synchronizes the update with the TC to ensure that the SC and TC are consistent.

5.

Data auditing

AC is an entity that is supervised by marked public data and established by a body, with standards and guidelines, responsible for auditing, certification, and storage.

(1)

The TC sends the certificate and signature $p{k}_{D{O}_T}$ to the AC for consistency verification.

$$proo{f}_{TC}=\left\{{\zeta }^{\left(t{s}_1\right)},{\mu }^{\left(t{s}_1\right)},g^{\left(t{s}_1\right)},u^{\left(t{s}_1\right)}\right\}$$

(31)

(2)

The SC generates an audit certificate and sends the certificate and signature to the AC for consistency verification.

$$proo{f}_{SC}=\left\{AV{F}_{c\times t},h_{root}^{\left(t{s}_1\right)},{\left\{h_{\lambda \phi }^{\left(t{s}_1\right)}\right\}}_{\phi \in \left[1,c\right]}\right\}$$

(32)

(3)

After receiving proofs from SC and TC, AC computes verification information $proo{f}_{SC}$.

$${\Omega }^{\left(t{s}_1\right)}={\displaystyle \prod _{\phi \in \left[1,c\right]}{g}^{\wedge {\phi }^{\left({{ }^{ts}}_1\right)}\cdot h^{\left({{ }^{ts}}_1\right)}{\phi }^{\mathrm{mod}N}}}$$

(33)

(4)

AC checks the signatures of $p{k}_{D{O}_S}$ and $p{k}_{D{O}_T}$, uses Equation (33) to verify the correctness of $proo{f}_{TC}$. If the verification fails, TC may not store it as needed, and AC notifies SC of the abnormal situation at this time. If the verification succeeds, AC generates audit log $t{s}_1$ on the blockchain for further inspection.

$${\left({\zeta }^{\left(t{s}_1\right)}\right)}^e\cdot g^{{ }^{\left(t{s}_1\right)}}\left(\mathrm{mod}N\right)={\Omega }^{\left(t{s}_1\right)}\cdot g^{\mu \left(t{s}_1\right)}\left(\mathrm{mod}N\right)$$

(34)

3.3.2. Security Verification under Blockchain

The introduction of AGMS into BDCVS in this paper ensures reliable consistency of dynamic cross-chain data. AGMS (advanced gamma multisignature) is an advanced gamma multsignature scheme. Gamma multisignature is a multiparty digital signature scheme that allows for multiple signers to sign the same document or transaction to ensure the integrity and authenticity of the document. The AGMS scheme is improved and optimized on the basis of the traditional multsignature scheme and has higher security and efficiency. It adopts a signature algorithm based on elliptic curve cryptography, which uses a shorter signature length and faster signature speed. If all entities in the BDCVS (each data center) follow the prescribed procedure in an honest and diligent manner, the relevant parameters can be calculated with completeness and accuracy. At this point, the AC ensures audit consistency between the SC and the TC, thus improving the reliability and credibility of the blockchain data interaction consistency problem.

3.3.3. Analysis of the Advantages of Blockchain

Blockchain technology has significant advantages in distributed intelligent annotation systems, especially in terms of security, reliability, and data integrity. First of all, blockchain ensures that data cannot be tampered with once they are written through encryption and chain structure, which provides a guarantee for the security of the labeled data. Then, using smart contracts and permission management mechanisms, blockchain can restrict access to data, ensuring that only authorized users can mark and access them. At the same time, the decentralized nature of the blockchain makes the system not dependent on a single central server, reducing the risk of a single point of failure. Each node keeps a complete copy of the data, ensuring that the data are stored in multiple locations in the network, improving the fault tolerance of the system. It is worth noting that among the specific blockchain protocols, Ethereum can support blockchain platforms for smart contracts, which can be used to implement complex business logic and data management, and Hyperledger is a blockchain framework designed for enterprise-level applications, emphasizing privacy and performance. In this paper, the blockchain is integrated with deep learning models to demonstrate good performance. On the one hand, the blockchain is integrated with digital twin (DT) technology by creating digital copies of physical entities, and the blockchain can ensure the security and consistency of these copies while utilizing deep learning models for real-time analysis and prediction. On the other hand, the combination of blockchain technology with graph convolutional networks (GCNs) has also been effective. While GCN can be used for feature extraction and pattern recognition when dealing with data with complex network structures, the blockchain ensures data integrity and traceability. In these ways, blockchain technology can significantly improve the security, reliability, and data integrity of distributed intelligent labeling systems, while integration with deep learning models can further enhance the system’s intelligent analysis and prediction capabilities.

3.4. The Diagram of the Proposed Method

Finally, the diagram of the proposed method is summarized for the system process in Figure 3; the detailed procedures are given as follows: First, the real-time data are input into the system—the first step, that is, to carry out the GCN model for training—to obtain the prediction data, and input it into the DT mechanism model, which produces a part of the historical data to return to the GCN incentive model. The other part of the prediction data continues to be inputted downward to the DT data model, after the recommender system, to improve the quality of the content and the conversion rate, and to increase the dependence on the model and the frequency of use and reduce the burden of screening information, making the recommendation more intelligent. After the recommendation system, the consistency of the data is judged by the reputation mechanism, and if the consistency test is passed, the next step of blockchain technology security verification is carried out; otherwise, they are sent back to the recommendation system for re-recommendation. Finally, the data that pass the security verification are output as the final attribute of the system, and the data that do not pass the security verification are returned to the blockchain technology to continue the security verification until they pass the security verification and can be output.

4. Experiment

4.1. Datasets

In this paper, the smart meter dataset in London was selected as the dataset for experimental verification, which is mainly labeled by the energy consumption level displayed by the meter. This dataset contains energy consumption readings from 5567 London households participating in the UK Power Networks-led Low Carbon London project between November 2011 and February 2014.

The dataset is the result of measuring the smart meters according to half-hourly measurements, which contains all the information on the households in the panel (their acorn group, their tariff), specifically daily information, such as the number of measures, the minimum, maximum, average, median, sum, and standard. For application of the model, we selected January 2012 to December 2013 as time interval of this study, and found 358 households (also named as LCLids) with data covering this 2-year span. After interpolating missing values, we reduced the resolution of the datasets to hourly values by summing up two subsequent half-hour values. However, we did not further trim outliers as in the previous work but kept the data in their original state to retain the volatility in individual electricity consumption.

4.2. Experiment

To comprehensively evaluate the performance of the proposed FedDTrep federated learning algorithm, this experiment compares it with several advanced algorithms in the current field, including FedAvg [35], FedProx [36,37], and MOON [38]. In the conventional federated learning framework, the FedAvg (federated averaging) algorithm is employed, wherein each client autonomously trains a model using local data and subsequently transmits the updated model to a centralized server. The server merges these updates equally to update the global model. However, when not independently co-distributed, model performance may be degraded because the variability in updates across clients is not taken into account. FedProx (federated proximal algorithm), an optimization algorithm designed in a federated learning environment, introduces a regularization term, which encourages clients to update the global model more “gently”, aiming to deal with the possible non-uniform distribution of the data among different clients. FedProx can be seen as a generalization and reconstruction of FedAvg. MOON is a longitudinal joint learning algorithm. The MOON algorithm, a concise and efficient model-level federated learning algorithm, is an improvement on FedAvg.

In the experiments, the regular term weight hyper parameter $\mu$ adopted by the FedProx algorithm is set to 0.1 for all the datasets, while for the MOON algorithm, the regular term weight hyper parameter $\mu$ is set to 0.1 for all the datasets, and the temperature hyper parameter $\tau$ for comparison learning is set to 0.5 [35]. To ensure the fairness of the experiment, all algorithms participating in the comparison, including the FedDTrep algorithm in this study, adopted uniform settings on all datasets: the local training cycle $E$ was set to 2, the learning rate $\eta$ was set to 0.01, and the batch size $B$ was set to 64. To access the model performance, the accuracy index is introduced as accuracy = Num(correct predicting)/Num(total predicting).

4.2.1. Comparison of the Results of the Algorithms for Different Participating Individuals

Convergence speed and accuracy are important in distributed intelligent marking consistency verification. We first explore how the number of participating local models affects the performance of our designed modeling algorithm. In Figure 4, Figure 5 and Figure 6, we compare the results when the number of participating local models is set to different values. Meanwhile, the side-by-side comparison compares our designed FedProx algorithm with the rest of the existing federated learning algorithms, and sets them under synchronous federated learning and asynchronous federated learning frameworks, respectively.

A.

Performances of data analysis using five local models

From Figure 4, it can be seen that the convergence trends of the four methods are similar; from an overall point of view, they are all accompanied by a small degree of fluctuation, but the overall upward trend eventually converges. With respect to the number of iterations, approximately 15 to 100 before the beginning of the model, there is a clear increase in accuracy, and then the magnitude of the slowdown begins to increase and is accompanied by fluctuations until after 180 iterations, when they are gradually lowed and eventually converge to a flat, and it is not difficult to see that our proposed FedDTrep algorithm is ultimately higher than the other three algorithms in terms of the algorithm’s accuracy, which converges at approximately 55%. The other three algorithms are FedProx, FedAvg, and MOON algorithms in descending order of accuracy. Compared to the synchronous case, the model accuracy in the asynchronous case on the right side is significantly improved from the original, more than 50% to more than 92%. The overall trend of change is similar, both from the beginning of approximately 20 iterations to the ground 50 iterations when the model accuracy is significantly improved, and from 50 iterations to 100 iterations, the accuracy improvement amplitude relative to the previous reduction. However, the model accuracy is still a larger improvement, from 100 iterations to 200 iterations after the relative stabilization, the final convergence in 92% of the meaning. The FedDTrep algorithm proposed in this paper is more accurate than the other three algorithms. It is worth noting that after switching from synchronous to asynchronous federated learning, the global model accuracy based on the FedDTrep algorithm is smoother than that of the other three algorithms, with fewer upward and downward variations; moreover, the accuracy increases relatively smoothly and finally converges to 94%.

Figure 4. The left-hand and right-hand side show a comparison of the accuracy of the four algorithms of the model for the synchronous and asynchronous cases.

Figure 4. The left-hand and right-hand side show a comparison of the accuracy of the four algorithms of the model for the synchronous and asynchronous cases.

B.

Performances of data analysis using 10 local models

In terms of mimicking the experimental procedure of the previous figure, in order to further illustrate the superiority of our approach in distributed smart marking consistency verification, we conducted an experimental comparison with the same paradigm as in Figure 3 and increased the number of models participating in the local training to 10, and the results are shown in Figure 5.

Figure 5. The left and right sides show the accuracy comparison of the four algorithms of the global smart marking model for the synchronous and asynchronous cases, respectively, with the number of local models being 10.

Figure 5. The left and right sides show the accuracy comparison of the four algorithms of the global smart marking model for the synchronous and asynchronous cases, respectively, with the number of local models being 10.

In Figure 5, both plots show an increase in precision compared to Figure 4. In terms of the horizontal comparison, the accuracy of the synchronized federated learning case is still lower than that of the asynchronous federated learning case. Vertically, for the synchronous case, the accuracy of local intelligent marking model training is improved to approximately 55% for all four algorithms. Compared with the accuracy derived by the FedDTrep algorithm with 5 local participants in Figure 5, the accuracy curve derived by the FedAvg algorithm with 10 local participants in Figure 5 increases more smoothly. Similarly, for the asynchronous federated learning case, the accuracies of the final global model smart labeling training all converge to approximately 94%.

C.

Performances of data analysis using of 15 local models

Similarly, to fully illustrate the superiority of our approach for distributed smart marking consistency verification, we increased the number of models involved in local training to 15, and the smart marking accuracy is shown in Figure 6.

Figure 6. Comparison of the accuracy of the four algorithms of the global intelligent marking model for the synchronous and asynchronous cases with 15 local models on the left and right sides.

Figure 6. Comparison of the accuracy of the four algorithms of the global intelligent marking model for the synchronous and asynchronous cases with 15 local models on the left and right sides.

In terms of the horizontal comparison, the accuracy of the synchronous federation learning case is still lower than that of the asynchronous federated learning case. In the longitudinal analysis of the accuracy of local intelligent marking model training within synchronous and asynchronous federated learning scenarios, our proposed FedDTrep algorithm demonstrates a noteworthy enhancement in model training accuracy compared to the other three existing algorithms. Specifically, our algorithm achieves convergence rates of around 58% and 94.6% for the synchronous and asynchronous cases, respectively.

4.2.2. Different Degrees of Heterogeneity

Undertaking the comparison experiments from Figure 4, Figure 5 and Figure 6, we continue to increase the number of models participating in local training to 20; meanwhile, in order to more scientifically reasonably and comprehensively verify the superiority of our method, we investigated the experiments according to the dimensions with different degrees of heterogeneity, and the results of the experiments are shown in Figure 7.

In Figure 7, the three graphs all represent the case when the number of local models involved in training is 20, but they differ in the degree of heterogeneity, i.e., the difference in similarity when different local models involved in training are selected for information interaction. Specifically, from the three graphs, we can see that although the final model training accuracies all converge to approximately 97%, the combination of local models with greater similarity results in a more accurate degree of global model training and a smoother increase in accuracy. From left to right, we can observe a gradual smoothing of the accuracy curve, indicating that the model similarity is greater when it is locally interacting with information.

4.2.3. Different Numbers of Malicious Nodes

In addition to considering the two factors of the number of models participating in local training and the different degree of heterogeneity, the presence of malicious nodes can also have a damaging impact on the accuracy and consistency of global model training, and the impact of different numbers of malicious nodes varies. The specific experimental results are shown in Figure 8, and similarly, the left side shows the synchronous federated learning scenario and the right side shows the asynchronous federated learning framework.

As in the previous Figure 4, Figure 5, Figure 6 and Figure 7, the comparison graphs between the centralized training and the FedDTrep method under the synchronous federated learning framework on the left side and the asynchronous federated learning framework on the right side are still followed in Figure 8. From the left and right graphs, we can see the same points. In terms of the general trend, when the number of malicious nodes is the same, the accuracy of the smart marking model of the centralized training method is lower than the accuracy of the FedDTrep method. As the number of malicious nodes increases, both centralized training and FedDTrep methods show a decreasing trend. A comparison of the two graphs shows that the accuracy of model training under synchronous federated learning is significantly lower than the accuracy of model training under the asynchronous federated learning framework. Taking the blue curve of the FedDTrep-malicious-1 method with the best training results as an example, as the number of iterations increases, the accuracy during synchronous training on the left side eventually stabilizes can reach approximately 86%, while the accuracy during asynchronous training on the right side converges at approximately 99.6%. This suggests that malicious nodes can have a destructive effect on the accuracy of model training for global smart labeling, and the degree of such destruction becomes more serious as the number of malicious nodes increases. Moreover, compared with the centralized training method, the FedDTrep algorithm proposed in this paper achieves excellent accuracy. The effect of both centralized and FedDTrep algorithm training shows that the asynchronous federated learning framework is better than synchronous learning, which also indicates that asynchronous federated learning can avoid memory bottlenecks and computational efficiency problems when training on large-scale distributed data, which highlights its advantages. In addition, asynchronous federated learning has better generalization ability, and can train models on different local data distributions, thus obtaining better model generalizability.

4.2.4. Performance of Algorithms with Different Numbers of Participating Nodes

The optimization comparison of our method with three other existing federated learning algorithms is shown in the previous experimental figure, which demonstrates the superiority of our proposed FedDTrep algorithm. In this section, we mainly explore the performance of the algorithms from the perspective of DTs, as shown in

Table 1. This table shows the performance of the algorithm with different number of participating nodes.

n	Arithmetic	Synchronization Accuracy Rate (%)	Asynchronous Accuracy Rate (%)	Client Average Download Latency (ms)	Client Mean Computing Delay (ms)	Client Average Upload Delay (ms)	Client Mean Total Time Delay (ms)
10	FedAvg	85.81	94.87	0.810	0.138	2.399	3.347
	Fedrep	86.03	95.84	0.826	0.137	2.405	3.368
	FedDTrep	87.38	97.05	0.666	0.135	2.202	3.003
15	FedAvg	87.68	94.98	0.832	0.137	2.411	3.368
	Fedrep	88.59	95.57	0.820	0.140	2.400	3.360
	FedDTrep	89.63	97.82	0.685	0.125	2.130	2.952
20	FedAvg	88.52	95.52	0.828	0.139	2.396	3.363
	Fedrep	89.05	96.71	0.821	0.139	2.406	3.366
	FedDTrep	90.51	97.75	0.652	0.131	2.155	2.938
25	FedAvg	88.77	95.61	0.816	0.139	2.444	3.399
	Fedrep	89.19	96.89	0.825	0.141	2.406	3.368
	FedDTrep	90.86	98.03	0.690	0.135	2.164	2.995
30	FedAvg	89.37	96.86	0.831	0.138	2.420	3.386
	Fedrep	90.82	97.86	0.819	0.139	2.424	3.382
	FedDTrep	91.84	98.00	0.667	0.135	2.178	2.983

The performance metrics for FedDTrep are shown in bold.

. We choose the basic FedAvg algorithm and the Fedrep algorithm without DT to compare with our proposed FedDTrep algorithm and consider the perspective of the number of participating locally trained models as well as the various performances for comparative analysis. As shown in Table 1, this paper counts a total of six algorithmic performances of the three algorithms FedAvg, Fedrep, and FedDTrep, which are synchronous accuracy, asynchronous accuracy, average client download latency, average client computation latency, client usual upload latency, and client usual total latency, with the first two performance parameters in percent and the last four performance parameters in milliseconds. With the increase in the number of nodes, i.e., the number of local models participating in the training of the global information interaction model, the performance of each algorithm under the three methods is enhanced, and it can be seen that among the three algorithms, the accuracy of the three algorithms increases and the latency decreases in turn. Moreover, a side-by-side comparison shows that the accuracy of the intelligent marking model with synchronous federated learning is lower than that of asynchronous federated learning, indicating that the comprehensive performance effect of FedDTrep proposed in this paper is the strongest.

5. Conclusions

In this work, we propose FedDTrep, a novel reputation mechanism-based distributed solution dedicated to efficient smart marking operations for cross-domain devices in distributed data middles. FedDTrep combines two strategies, digital twins and graph convolutional neural network (GCN), using a federated learning scheme with reputation-based trust mechanism, thus significantly reducing computational and communication overhead while utilizing blockchain technology to guarantee security and privacy during data transmission. The experimental results show that our FedDTrep system significantly minimizes computational and communication latency and improves the accuracy rate while guaranteeing accuracy and privacy compared to the FedAvg scheme and Fedrep scheme with federated learning only, as well as the MOON scheme.

In distributed systems with heterogeneous data, the proposed method trains across devices without centralizing data, benefiting privacy-sensitive annotation environments. It handles diverse data but must address data quality and annotation standard variations. Reputation-based aggregation and DT methods help manage skewed data distributions. In resource-limited and communication-heavy environments, the method scales well by using decentralized data sources for annotation, though efficient aggregation algorithms and adaptive communication are crucial. Each node locally annotates data, contributing to the global model without centralized transfer. Digital twins provide real-time simulations and feedback, enhancing scalability in dynamic environments. The proposed method supports scalable data annotation by combining decentralized learning with real-time simulations.

While digital twin-based distributed federated intelligent annotation systems provide some basic security protections, there is still room for improvement in global execution efficiency, model defensibility, convergence, and system training scalability and migration. These frameworks need to further integrate more efficient privacy-preserving techniques and improve defenses against malicious attacks. And, although the research on digital twin technology for federated learning privacy protection has made some progress, it still needs to overcome the challenges of efficiency, practicality, multiparty collusion attacks, and security frameworks to achieve more secure and efficient privacy protection.