Toward Sustainable Model Services for Deep Learning: A Sub-Network-Based Solution Integrating Blockchain with IPFS and a Use Case in Intelligent Transportation

Abstract

In the era of deep learning as a service, ensuring that model services are sustainable is a key challenge. To achieve sustainability, the model services, including but not limited to storage and inference, must maintain model security while preserving system efficiency, and be applicable to all deep models. To address these issues, we propose a sub-network-based model storage and inference solution that integrates blockchain and IPFS, which includes a highly distributed storage method, a tamper-proof checking method, a double-attribute-based permission management method, and an automatic inference method. We also design a smart contract to deploy these methods in the blockchain. The storage method divides a deep model into intra-sub-network and inter-sub-network information. Sub-network files are stored in the IPFS, while their records in the blockchain are designed as a chained structure based on their encrypted address. Connections between sub-networks are represented as attributes of their records. This method enhances model security and improves storage and computational efficiency of the blockchain. The tamper-proof checking method is designed based on the chained structure of sub-network records and includes on-chain checking and IPFS-based checking stages. It efficiently and dynamically monitors model correctness. The permission management method restricts user permission based on the user role and the expiration time, further reducing the risk of model attacks and controlling system efficiency. The automatic inference method is designed based on the idea of preceding sub-network encrypted address lookup. It can distribute trusted off-chain computing resources to perform sub-network inference and use the IPFS to store model inputs and sub-network outputs, further alleviating the on-chain storage burden and computational load. This solution is not restricted to model architectures and division methods, or sub-network recording orders, making it highly applicable. In experiments and analyses, we present a use case in intelligent transportation and analyze the security, applicability, and system efficiency of the proposed solution, particularly focusing on the on-chain efficiency. The experimental results indicate that the proposed solution can balance security and system efficiency by controlling the number of sub-networks, thus it is a step towards sustainable model services for deep learning.

1. Introduction

Deep learning has entered the era of large models. Due to limitations in computing resources, expertise, and time, it is challenging for ordinary users to deploy, maintain, and utilize deep models. This has given rise to the concept of Deep Learning as a Service (DLaaS) [1,2]. In DLaaS, model services are at the core. Two fundamental model services are to store models for model owners, referred to as model storage, and to generate results for model users using the stored models, referred to as model inference. Sustainability is an essential requirement for model services. Model services are provided through a service system. Sustainable model services require the system to be able to provide services stably over the long term, which necessitates the system to ensure the security of the model. A model involves intellectual property and service quality, and if it is attacked, related model services may become disrupted or ineffective, resulting in losses for both model owners and users. Particularly in fields closely related to public safety, such as intelligent transportation, attacks on models can lead to significant incidents. Various attacks and protection methods for deep models have been widely discussed. Model security vulnerabilities include model tampering [3], model inputs tampering [4,5], model extraction [6] and model leakage [7]. Currently, methods to protect model security can be divided into two main categories. One category includes algorithms to defend against attacks [8]. The other focuses on secure model storage techniques to ensure model security by minimizing the possibility of attacks [9]. Sustainable model services also require the system to be applicable to all deep models. Only a widely applicable model service system can continuously meet various demands; otherwise, it may be replaced by more versatile systems. Sustainable model services also involve the optimal utilization of resources, necessitating the system to balance storage efficiency and computational efficiency while ensuring security. If the efficiency of a model service system is low, it may face obsolescence because it cannot meet the requirements for resource sustainability.

Existing works have already explored to maintain sustainability for deep model services. Blockchain [10], as a highly secure digital ledger technology, is commonly used as a secure storage platform for building a model service system. To further enhance the security and efficiency of the system, integrating blockchain with other technologies has become a recent research focus. Common technologies include data encryption [11,12], auditing and monitoring [13,14], access control [15], and the InterPlanetary File System (IPFS) [16]. How to design and integrate specific techniques to create a secure, efficient, and highly applicable system that supports sustainable model services for deep learning is an issue worthy of in-depth research. To address this issue, ref. [17] introduced an analytical framework for machine learning that combines blockchain with data encryption to protect model security, while also proposing the use of a binary-based analysis storage format (ASTORE) to store models, improving the system efficiency. Ref. [18] combined blockchain with data encryption, auditing and monitoring, proposing a method called DeepRing to protect deep neural networks. This method can prevent model tampering. Ref. [19] introduced a deep model evaluation framework. This work suggested using the IPFS as an off-chain auxiliary part to store the entire model, only recording the model in the blockchain, managing model permissions using single-attribute-based on-chain access control, and utilizing trusted off-chain computational resources for centralized model inference. This approach is versatile and practical to some extent.

The above-mentioned works introduce new ideas for maintaining the sustainability of deep model services but also have some limitations. Ref. [17] only employs data encryption to protect model security. Although it proposes using a new format for model storage, it has not discussed how to manage model usage permissions or introduce auditing and monitoring mechanisms to prevent model tampering, resulting in significant security gaps. Additionally, ref. [17] does not provide a corresponding inference method for models in the ASTORE format. While DeepRing [18] significantly improves security, it assumes that each layer has weights, biases, and activation functions, and that each layer has only one subsequent layer. This means that the method cannot handle complicated network architectures and lacks universality. Moreover, storing the entire model in the blockchain and performing model inference on-chain significantly reduces system efficiency. The solution proposed in [19] has issues as well. On one hand, it does not restrict usage time for models. On the other hand, model information is concentrated in the IPFS, which could lead to the leakage of all model information if the model address is compromised. Additionally, the centralized model inference also poses security risks.

To achieve sustainable model services for deep learning that have good applicability and balance security and system efficiency, we propose a sub-network-based solution for model storage and inference that integrates blockchain with IPFS. The specific contributions are as follows:

(1)

A highly distributed storage method based on blockchain and IPFS is proposed. Specifically, a deep model is decomposed into a series of sub-networks, dividing the whole model into intra-sub-network and inter-sub-network information. Sub-network files are uploaded to the IPFS, while the encrypted address of the sub-networks and their connection information are organized into a chained structure recorded in the blockchain. This method makes it difficult to leak the complete model information and helps alleviate the storage burden in the blockchain, thus enhancing system efficiency, and can achieve a balance between security and system efficiency by adjusting the number of sub-networks.

(2)

A two-stage tamper-proof checking method based on the chained structure of sub-network records is designed. This method checks the information of stored models, including an on-chain checking stage and an IPFS-based checking stage. It can dynamically and continuously monitor model correctness, further protecting the model from tampering caused by security vulnerabilities in the blockchain and the IPFS.

(3)

A double-attribute-based model usage permission management method is presented. This method employs attribute-based on-chain access control, granting permission to a user based on the role and the expiration time. Only authorized users with permission are allowed to perform inference using the model but are restricted from accessing and downloading the model. This enhances supervision on users, further mitigates security risks, and controls system efficiency.

(4)

An automatic inference method based on preceding sub-network encrypted address lookup is proposed. This method divides the entire model inference process into distributed off-chain sub-networks inference processes. Model inputs and sub-network outputs are uploaded to the IPFS, and their encrypted address are recorded in the blockchain. This records and pretects the inference results while further alleviating the on-chain storage and computational burden.

(5)

A smart contract called Model Contract is designed to deploy all four methods mentioned above into the blockchain, enabling them to be automatically executed in a decentralized manner.

(6)

A use case in the field of intelligent transportation is provided to demonstrate the enhancement of vehicle detection reliability using the proposed solution. The sustainability of this solution is analyzed based on this use case, including its security, applicability, and system efficiency. Additionally, an analysis of the on-chain efficiency sustainability of this solution is conducted based on a simulation experiment.

The rest of this paper is organized as follows. Section 2 presents key issues, related technologies, and existing works of model services for deep learning. Section 3 describes the proposed solution. In Section 4, we provide a use case in intelligent transportation and conduct analyses on the proposed solution. Section 5 serves as the conclusion to this paper.

2. Related Works

In this section, we first discuss the key issues of model services for deep leaning, then introduce the fundamental technologies for building a system for model services, and finally present the current research status of model services for deep learning.

2.1. Key Issues of Model Services for Deep Learning

The development of deep models has shown a trend towards larger scales and more complex architectures. Early deep convolution neural networks were constructed by stacking convolutional and fully connected layers together. The scales have evolved from small to large over time. For instance, LeNet-5 [20], proposed in 1998, includes two convolutional layers and three fully connected layers, with around 60 thousand parameters. AlexNet [21], introduced in 2012, comprises eight convolutional layers and three fully connected layers, with approximately 60 million parameters. Visual Geometry Group (VGG) networks [22] were introduced in 2014, and the VGG-16 has thirteen convolutional layers and three fully connected layers, in total around 138 million parameters. As models grew in scale, there were architectural innovations. For example, ResNet [23], proposed in 2015, introduced residual connections. Inception [24], also proposed in 2015, featured parallel branches with various-sized convolution kernels. More recently, Google’s Transformer [25] architecture gained attention. It is designed for sequence data and incorporates encoder-decoder architecture, multi-head attention and self attention mechanisms, skip connections, and layer normalization layers, and can handle long-range dependencies effectively. This success led to models built on this architecture, including Google’s Bidirectional Encoder Representations from Transformers (BERT) [26] and OpenAI’s Generative Pre-trained Transformers (GPT) [27]. The BERT-large has around 300 million parameters, while GPT-3 has about 175 billion parameters. Deep models like BERT and GPT fall into the category of large models.

The security of deep models is also becoming increasingly important, especially in fields closely related to public safety, such as intelligent transportation. The security vulnerabilities of models involve the following aspects. On one hand, if a model can be easily accessed by attackers, they can manipulate the model. Manipulation can take the form of directly deleting or damaging the existing architecture and parameters of the model, as well as implanting specific architectures such as backdoors causing erroneous outputs when the input triggers the backdoor condition. On the other hand, if model information is easily obtained by attackers [7], they can study the model and add designed perturbations to the model input, resulting in incorrect outputs. These attack forms are referred to as adversarial attacks [5] in the literature. Adversarial attacks initiated based on complete model information are known as white-box attacks, while those initiated based solely on the model’s input and output are known as black-box attacks. Additionally, if attackers have unlimited access to the model, they can also use machine learning to extract models for illicit purposes [6]. Currently, methods to protect model security can be categorized into two types. One enhances the model robustness against attacks by designing algorithms, for example, adversarial training techniques [8] to withstand adversarial attacks. The other type explores secure storage techniques [9] to ensure model security by minimizing the likelihood of attacks. The proposed model storage method in this paper falls into the latter category.

Training, deploying, maintaining and using deep models are difficult tasks for ordinary users due to limitations related to computing resources, expertise, and time. This challenge has given rise to the concept of Deep Learning as a Service (DLaaS) [1,2]. The core of DLaaS is model services, which includes two fundamental services: model storage, involving the storage of models for model owners, and model inference, involving the generation of results for model users using the stored models. Model services are provided through a service system. The system needs to be capable of consistently delivering services over the long term while considering the efficient utilization of resources to ensure the sustainability of model services. Specifically, the system must ensure the security of models to protect intellectual property and service quality. The system also needs to be applicable to all deep models to ensure good service versatility. Additionally, the system must balance storage and computational efficiency to conserve resources and energies while maintaining model services.

2.2. Fundamental Technologies for Building Model Service Systems for Deep Learning

There are many fundamental technologies involved in building model service systems for deep learning. Technologies related to this paper include data encryption, access control, auditing and monitoring, blockchain, and the IPFS.

Data encryption is one of the most prevalent secure data technologies [11,12,28]. It involves converting data into ciphertext, ensuring that authorized users can decrypt the data. Encryption techniques can be divided into symmetric and asymmetric encryption, each differing in terms of encryption and decryption processes, speed, and transmission security. Firstly, symmetric encryption [29,30] uses one key for both encryption and decryption, whereas asymmetric encryption [31,32] employs two keys: a public one and a private one. Secondly, symmetric encryption and decryption are faster, suitable for processing large volumes of data. Asymmetric encryption and decryption take more time and are relatively slower, making them more suitable for small-scale data. Taking security and efficiency into consideration, the solution proposed in this paper adopts symmetric encryption.

Access control is a frequently used information security technology [15], and has been applied in various fields such as online courses [33], medical device supply chains [34], port supply chains [35], and the Internet of Things [36]. It refers to managing or restricting users’ access permissions to data, ensuring the information inaccessible to unauthorized users. The solution proposed in this paper adopts the Attribute-Based Access Control (ABAC) [37]. The ABAC determines access based on users’ attributes, providing a better solution for complex security requirements. The general process of verifying access control permissions is as follows: access control policies are predefined by data owners to establish rules for resource access control. Access control requests are submitted by users, and then compared against the predefined access control policies. If they match, authorization is granted; otherwise, access is denied. Centralized access control can pose risks. Having access control policies stored centrally can lead to a single-point-of-failure. And, centralized access control requires third-party supervision, also introducing the tampering risk.

Auditing and monitoring are technologies used to keep the security and compliance of data in various systems [9,14], originally applied in financial systems. These techniques aim to ensure that data is adequately protected to prevent data leaks, tampering, or other security vulnerabilities. In the field of data storage, event logs are used to record important events that occur in the storage system, such as accesses, modifications, deletions, etc., for the purpose of auditing. Integrity monitoring is employed to prevent and detect data tampering, verifying the integrity of data through techniques like digital signatures [13]. Real-time monitoring involves ongoing checks on the storage system to promptly identify anomalies. Compliance checks refer to examining whether the stored data adheres to specific requirements.

Blockchain is a highly secure digital ledger technology, characterized by decentralization, distributed consensus, and immutability [10,38]. It is often used as a secure data storage platform. The decentralization implies that data is stored across multiple nodes, enhancing system robustness. The distributed consensus mechanism uses consensus algorithms to verify and confirm transactions; only verified transactions can be added, preventing malicious transactions and tampering. The immutability refers to the fact that each block includes the hash value of the preceding block, forming a chained data structure. Modifying information within a block would change the hash values of all subsequent blocks, making tampering difficult. Furthermore, blockchain supports smart contracts, enabling automated on-chain operations and reducing the risk of third-party intervention. Despite its high security, the security of the blockchain is not absolute. For instance, the 51% attack is a common attack on the blockchain [39]. Attackers gain control over more than 51% of the computational power or the number of validating nodes in the blockchain network, enabling them to maliciously manipulate and attack the network. In such cases, attackers can tamper with information in the blockchain, reverse transactions, initiate attacks, and disrupt the confirmation of legitimate transactions, thereby compromising the data security.

The IPFS is a decentralized peer-to-peer file storage system [16]. Unlike traditional web protocols that rely on location-based addressing, the IPFS uses content addressing. This means that a file’s hash value, known as the Content Identifier (CID), serves as its unique identifier. This approach allows files to be located and retrieved based on their content, independent of specific servers or locations. This content-addressing method helps achieve secure data storage and can also be used to detect file tampering. Compared to traditional centralized cloud storage services, the IPFS offers advantages such as decentralized storage and high-performance data transmission. Unlike the blockchain, the IPFS does not support smart contracts. The IPFS has already been employed for secure storage of industrial images [40], medical data [41], and vehicle network data [42]. While the IPFS provides a certain level of security, its security is not absolute. Factors such as the presence of malicious nodes, CID leakage, or external attacks can impact the security and availability of data.

2.3. Current Research Status of Model Services for Deep Learning

Integrating multiple technologies effectively based on blockchain is a recent research focus in the field of building model service system for deep learning. Ref. [17] proposed the integration of blockchain and data encryption, utilizing a specialized binary data format known as Analyzed Store Format (ASTORE) to store machine learning models, thus enhancing data efficiency within the system. However, ref. [17] did not discuss model usage permissions or an auditing and monitoring mechanism for model tamper resistance, nor did they elucidate how model inference occurs within the ASTORE format. Ref. [18] combined data encryption, auditing and monitoring, and blockchain to introduce a protective approach for deep neural networks named DeepRing. This method designs each layer of the deep network as a block, with each block containing functionalities to store layer parameters, compute layer outputs, update a ledger after output computation, and validate subsequent layer outputs to prevent model tampering. Nonetheless, DeepRing assumes that each layer possesses weights, biases, and activation functions, and that each layer only connects to one another layer. This limitation implies that the approach cannot handle complicated network structures. Furthermore, storing the entire model in the blockchain could potentially have an adverse impact on system efficiency, as it requires a significant amount of on-chain storage space and on-chain computational resources. To alleviate the storage and computational burden in the blockchain, ref. [19] proposes the use of the IPFS as an off-chain auxiliary storage for model files [43], reserving the blockchain solely for recording model-related information. Moreover, ref. [19] deploys access control through a smart contract to manage model usage permissions, while model inference is performed using trusted off-chain computational resources. One drawback of the solution proposed by [19] is that, if the CID of a model is leaked, all model information is compromised. Additionally, adopting a centralized processing approach for model inference can introduce security vulnerabilities. Overall, current research in model services for deep learning has not yet proposed mechanisms that are capable of effectively balancing security and system efficiency, and achieved good applicability, the study of sustainability in deep model services is still in its early stage.

3. Proposed Sub-Network-Based Solution Integrating Blockchain with IPFS for Deep Model Storage and Inference

To address the issues faced by current deep model services, a sub-network-based solution integrating blockchain with IPFS for fundamental model services, including storage and inference, is proposed. This solution incorporates mechanisms for balancing security and system efficiency, and based on this solution, systems supporting sustainable model services for deep learning can be developed. The solution involves four aspects: storage, tamper-proof checking, permission management, and inference, as illustrated in Figure 1. Subsequently, a detailed description will be provided for each aspect. The functions employed in the algorithms are listed in

Table 1. Descriptions of functions.

Function	Description
$\mathrm{Size}\left(\right)$	Returning the length of a list
$\mathrm{UploadToIPFS}\left(v\right)$	Uploading the file v to the IPFS and retrieve its CID
$\mathrm{Add}\left(r\right)$	Adding an element r to a list or dictionary
$\mathrm{Encrypt}(x,k)$	Encrypting x with key k
$\mathrm{Hash}(x_1,x_2,\dots ,x_n)$	Returning the identifier of a record through $x_1,x_2,\dots ,x_n$
$\mathrm{APIstub}.\mathrm{PutState}(x,v)$	Uploading v to the blockchain and assigning x as its index
$\mathrm{Marshal}(x_1,x_2,\dots ,x_n)$	Packaging $x_1,x_2,\dots ,x_n$
$\mathrm{Linux}.\mathrm{time}.\mathrm{Now}\left(\right)$	Retrieving the current time in Linux system
$\mathrm{APIstub}.\mathrm{GetState}\left(x\right)$	Searching with index x from the blockchain state database
$\mathrm{Reverse}\left(\right)$	Reversing the elements of a list
$\mathrm{Decrypt}(x,k)$	Decrypting x with key k
$\mathrm{DownloadFromIPFS}\left(x\right)$	Downloading the file with CID x from the IPFS
$\mathrm{Inference}(L,M)$	Using a input list L to infer the output of a network M

.

3.1. Model Storage

Considering the model security and the system efficiency, the proposed solution extends the storage space of the blockchain using the IPFS. Model storage employs a highly decentralized approach. Model owners first decompose the deep model to be stored into a series of sub-networks according to their needs, thereby dividing the information of the entire model into the information within the sub-networks, i.e., the files corresponding to the sub-networks, and the information between the sub-networks, i.e., the interconnections between sub-networks. When storing the model, the first step involves uploading files of the sub-networks to the IPFS, obtaining the corresponding CID for each file. Subsequently, the smart contract encrypts the CIDs using the key provided by the owner, i.e., $ModelKey$. Then, based on the interconnections between sub-networks provided by the owner, all encrypted CIDs are uploaded to the blockchain as records of the sub-networks in the blockchain. The structure of this record is designed as follows:

$$SNRecord=\{SNID,PreSNID,PreSNEncryCID,SNEncryCID\},$$

where $SNEncryCID=\mathrm{Encrypt}(CID,ModelKey)$ represents the encrypted CIDs of the sub-network file in the IPFS using the $ModelKey$. $PreSNEncryCID$ is a list of encrypted CIDs of the previous sub-networks adjacent to the current sub-network.

$$SNID=\mathrm{Hash}(PreSNID,PreSNEncryCID,SNEncryCID)$$

(1)

serves as the identifier for the current sub-network record. $PreSNID$ refers to the identifier of the preceding record, making the $PreSNID$ of the first record empty. Based on the sub-network records in the blockchain, the model’s record in the blockchain is designed as follows:

$$ModelRecord=\{ModelID,SNID\_List,ModelName,OwnerID,Time\},$$

where $SNID\_List$ is a list composed of $SNID$s from the sub-network records, and its order represents the sequence of recording the sub-networks. We do not impose any restrictions on the order of recording the sub-networks, and the order of $SNID\_List$ does not need to match the sequence in which the sub-networks appear in the model.

$$ModelID=\mathrm{Hash}(SNID\_List,Time).$$

(2)

$ModelName$, $OwnerID$, and $Time$ are the model name, the identifier of the model owner, and the time of recording the model, respectively. The design of the sub-network records and the model record is related to tamper-proof checking, permission management, and automated inference, which will be explained in the following sections.

To accomplish decentralized and efficient model management, a smart contract named Model Contract (MC) has been designed. AddModel() is a method within MC, used for uploading the sub-network files to the IPFS and recording the sub-networks and the model in the blockchain, which is displayed in Algorithm 1. FindByID() is another method within MC, serving the purpose of searching for a specific record within the blockchain.

Algorithm 1 AddModel(): Uploading sub-network files of a model to the IPFS and recording the sub-networks and the model in the blockchain

Require: $SNF\_List,PreSNF\_List,ModelKey,ModelName,OwnerID$

Ensure: $\mathrm{OK}$

1: for$k=1$ to $SNF\_List.\mathrm{Size}\left(\right)$ do   2: $\hspace{1em} SNCID\leftarrow \mathrm{UploadToIPFS}(SNF\_List[k\left]\right)$   3: $\hspace{1em} SNCID\_List.\mathrm{Add}\left(SNCID\right)$   4: end for   5: $PreSNID\leftarrow \mathrm{Null}$   6: $SNID\_List\leftarrow \mathrm{Null}$   7: for $k=1$ to $SNCID\_List.\mathrm{Size}\left(\right)$ do   8: $\hspace{1em} PreSNCID\leftarrow \left[SNCID\right|SNF\in PreSNF\_List\left[k\right]\mathrm{and}SNCID\in SNCID\_List]$   9:     for $j=1$ to $PreSNCID.\mathrm{Size}\left(\right)$ do 10: $\hspace{1em} PreSNEncryCID.\mathrm{Add}\left(\mathrm{Encrypt}\right(PreSNCID\left[j\right],ModelKey\left)\right)$ 11:     end for 12: $\hspace{1em} SNEncryCID\leftarrow \mathrm{Encrypt}(SNCID\_List[k],ModelKey)$ 13: $\hspace{1em} SNID\leftarrow \mathrm{Hash}(PreSNID,PreSNEncryCID,SNEncryCID)$ 14: $\hspace{1em} \mathrm{APIstub}.\mathrm{PutState}(SNID,\mathrm{Marshal}(PreSNID,PreSNEncryCID,SNEncryCID\left)\right)$ 15: $\hspace{1em} PreSNID\leftarrow SNID$ 16: $\hspace{1em} SNID\_List.\mathrm{Add}\left(SNID\right)$ 17: end for 18: $Time\leftarrow \mathrm{Linux}.\mathrm{time}.\mathrm{Now}\left(\right)$ 19: $ModelID\leftarrow \mathrm{Hash}(SNID\_List,Time)$ 20: $\mathrm{APIstub}.\mathrm{PutState}(ModelID,\mathrm{Marshal}(SNID\_List,ModelName,OwnerID,Time\left)\right))$ 21: return $\mathrm{OK}$

3.2. Model Tamper-Proof Checking

The proposed solution chooses to store the model in the IPFS and the blockchain due to their immutability. However, as mentioned in Section 2, this immutability is not absolute. In order to further prevent model tampering caused by security vulnerabilities in the IPFS and the blockchain, we designed a two-stage tamper-proof checking method for the stored models. Inspired by the anti-tampering mechanism of the blockchain, the identifier of the current sub-network record, i.e., $SNID$, is designed as a function of the identifier of the preceding sub-network record, i.e., $PreSNID$, while also being related to the encrypted CIDs in the record, i.e., $PreSNEncryCID$ and $SNEncryCID$, as shown in Equation (1). This implies that sub-networks are recorded using a chained structure. If $PreSNID$, $PreSNEncryCID$, or $SNEncryCID$ changes, $SNID$ will also change. Due to this design, in the proposed method, the model owner needs to back up the $SNID\_List$, referred to as $OrigSNID\_List$, and all $SNEncryCID$s, referred to as $OrigSNEncryCID$s, upon completion of the initial model storage. This backup is used for tamper-proof checking and restoring the model information in the blockchain.

Tamper-proof checking should be completed before the model is delivered for use. It includes two stages: the first stage checks whether the model information in the blockchain has been tampered with, and the second stage checks whether the model information in the IPFS has been tampered with. Specifically, in the first stage, a comparison is made between the current $SNID\_List$ and $OrigSNID\_List$ to determine their consistency. This comparison is performed in reverse order starting from the last element of both lists, until the first inconsistent $SNID$ is found. The reason for the change in this $SNID$ is that the corresponding $PreSNEncryCID$ or $SNEncryCID$ has been altered. The model owner examines the specific reason for the change in this record and re-records the corresponding sub-network and model. The process of checking tampered sub-network records is designed as a method in the smart contract MC, as illustrated in Algorithm 2. For the new model record, repeat the above checking and restoration process until the $SNID\_List$ of the latest model record is identical to $OrigSNID\_List$. This concludes the first checking stage. In the second stage, the smart contract decrypts the $SNEncryCID$ of each sub-network record and retrieves the corresponding CID from the IPFS. If the CID is not found, it indicates that the sub-network file has been tampered with or deleted. In this case, the model owner needs to re-upload the file of that sub-network to the IPFS. The process of checking tampered sub-network files is designed as one of the methods in the smart contract MC, as illustrated in Algorithm 3. The second checking stage concludes when the CIDs of all sub-networks recorded in the blockchain can be retrieved from the IPFS.

Algorithm 2 TPCI(): Checking whether the model information in the blockchain has been tampered with in the first stage of the tamper-proof checking

Require: $OrigSNID\_List,ModelID$

Ensure: $\mathrm{OK}\mathrm{or}\mathrm{Error}$

1: $ModelRecord\leftarrow \mathrm{APIstub}.\mathrm{GetState}\left(ModelID\right)$   2: if $ModelRecord==\mathrm{Null}$ then   3:   return $\mathrm{Error}(“\mathrm{The}\mathrm{model}\mathrm{record}\mathrm{cannot}\mathrm{be}\mathrm{found}.$”)   4: end if   5: $f\leftarrow \mathrm{Null}$   6: for $k=1$ to $OrigSNID\_List.\mathrm{Size}\left(\right)$ do   7:   if $OrigSNID\_List.\mathrm{Reverse}\left(\right)\left[k\right]!=ModelRecord.SNID\_List.\mathrm{Reverse}\left(\right)\left[k\right]$ then   8: $\hspace{1em} f\leftarrow OrigSNID\_List.\mathrm{Size}\left(\right)-k+1$   9:   end if 10: end for 11: if $f!=\mathrm{Null}$ then 12:   return $\mathrm{Error}(“\mathrm{The}f\mathrm{th}\mathrm{subnetwork}\mathrm{record}\mathrm{has}\mathrm{been}\mathrm{tampered}\mathrm{with}!$”) 13: end if 14: return $\mathrm{OK}$

Algorithm 3 TPCII(): Checking whether the model information in the IPFS has been tampered with in the second stage of the tamper-proof checking

Require: $ModelID,ModelKey$

Ensure: $\mathrm{OK}\mathrm{or}\mathrm{Error}$

1: $ModelRecord\leftarrow \mathrm{APIstub}.\mathrm{GetState}\left(ModelID\right)$   2: for $k=1$ to $ModelRecord.SNID\_List.\mathrm{Size}\left(\right)$ do   3: $\hspace{1em} SNRecord\leftarrow \mathrm{APIstub}.\mathrm{GetState}(ModelRecord.SNID\_List[k\left]\right)$   4: $\hspace{1em} SNCID\leftarrow \mathrm{Decrypt}(SNRecord.SNEncryCID,ModelKey)$   5: $\hspace{1em} response\leftarrow \mathrm{DownloadFromIPFS}\left(SNCID\right)$   6:     if $response==\mathrm{Null}$ then   7:        break;   8:     end if   9: end for 10: if $response==\mathrm{Null}$ then 11:    return   $\mathrm{Error}(“\mathrm{The}\mathrm{subnetwork}\mathrm{with}\mathrm{the}SNCID\mathrm{has}\mathrm{been}\mathrm{tampered}\mathrm{with}\mathrm{in}\mathrm{the}$ IPFS!”) 12: end if 13: return $\mathrm{OK}$

3.3. Model Permission Management

In addition to strengthening the supervision of model correctness, the proposed solution also enhances the restrictions on usage permission to the model. This helps maintain model security and system efficiency. By deploying the ABAC model in the blockchain, it manages model usage permissions based on user roles and expiration time. Specifically, a permission is defined as

$$Permission=\{UserID,Role,ModelID,Time\}.$$

Here, $UserID$ represents the user’s identifier, $Role$ indicates the user’s identity, which can be an owner or a visitor. Owners have the authority to add the visitor role for other users. The visitors are only granted permission to utilize the model but are not allowed to access or download the model. Users without any roles have no permissions regarding the model. $ModelID$ is the identifier of the model. $Time$ represents the expiration time for this permission. The record structure for each permission in the blockchain is set as $\{PermissionID,Permission\}$. $PermissionID$ is the identifier, calculated as

$$PermissionID=\mathrm{Hash}(Permission.UserID,Permission.ModelID,Permission.Role).$$

User’s usage permission to the model must undergo an permission verification, following these specific steps: first, the user submits an permission request as

$$Request=\{UserID,ModelID,Role\}.$$

Then, the identifier for this request is generated as

$$RequestID=\mathrm{Hash}(Request.UserID,Request.ModelID,Request.Role).$$

Permission verification attempts to retrieve the permission with $PermissionID$ equal to $RequestID$ from the on-chain state database. If no permission is found, it means the user does not have the right to use the model. If a permission record is found, further permission verification is conducted based on the expiration time. After the permission verification is successful, the model owner checks the model and provides the latest $ModelID$ and $ModelKey$ to the smart contract, which are not disclosed to the user.

To achieve decentralized model usage permission management, the smart contract MC has been designed with the AddPermission(), QueryPermission() and CheckPermission() methods. AddPermission() is used to add permission records to the blockchain, where owner permissions are added by the smart contract during model uploading, and visitor permissions are added by the owner. QueryPermission() utilizes the FindByID() method defined in Section 3.1 to search for permission records. CheckPermission() handles permission verification by first generating a model usage permission request, then using QueryPermission() to search for a permission record. Subsequently, it further verifies the permission based on the expiration time and returns the verification result, as outlined in Algorithm 4.

Algorithm 4 CheckPermission(): Checking a user’s permission

Require: $Request$

Ensure: $\mathrm{OK}\mathrm{or}\mathrm{Error}$

1: /** Generating a request for using the model. **/   2: if $Request.UserID==\mathrm{Null}\mathrm{or}Request.ModelID==\mathrm{Null}\mathrm{or}Request.Role==\mathrm{Null}$ then   3:   return $\mathrm{Error}(“\mathrm{The}\mathrm{request}\mathrm{is}\mathrm{incomplete}.$”)   4: end if   5: $RequestID\leftarrow \mathrm{Hash}(Request.UserID,Request.ModelID,Request.Role)$   6: /** QueryPermission(). **/   7: $Permission\leftarrow \mathrm{APIstub}.\mathrm{GetState}\left(RequestID\right)$   8: if$Permission==\mathrm{Null}$then   9:   return $\mathrm{Error}(“\mathrm{The}\mathrm{permission}\mathrm{has}\mathrm{not}\mathrm{been}\mathrm{submitted}.$”) 10: end if 11: /** Verifying permission with expiration time. **/ 12: if $Request.Role==Permission.Role\mathrm{and}Request.Role==\mathrm{Owner}$ then 13:   return $\mathrm{OK}$ 14: else if $Request.Role==Permission.Role\mathrm{and}Request.Role==\mathrm{Visiter}$ then 15:   if $\mathrm{Linux}.\mathrm{time}.\mathrm{Now}\left(\right)>Permission.Time$ then 16:     return $\mathrm{Error}(“\mathrm{The}\mathrm{permission}\mathrm{has}\mathrm{expired}.$”) 17:   else 18:     return $\mathrm{OK}$ 19:   end if 20: end if 21: return $\mathrm{Error}(“\mathrm{The}\mathrm{request}\mathrm{is}\mathrm{invalid}.$”)

3.4. Model Inference

In response to the proposed model storage method, an automatic inference method based on encrypted CID lookup of preceding sub-networks has been designed. This method is independent of the order of sub-network recording. Model users upload the model input, i.e., $ModelInput$, to the IPFS and provide the encrypted key for the model output, i.e., $OutputKey$, which is also used for encrypting and decrypting the CID of the model input. Model owners provide the $ModelID$ and $ModelKey$. The smart contract invokes the sub-networks by looking up the encrypted CID of the preceding sub-networks, following these steps: firstly, using the $ModelID$, the smart constract calls the $SNID\_List$ of the $ModelRecord$, then iterates through the $SNID\_List$ to find $SNRecord$s with an empty $PreSNEncryCID$. These $SNRecord$s correspond to the sub-networks that appear earliest in the model. And thus, these sub-networks can be inferred using the provided model inputs, the results of the sub-network inference are stored in the IPFS, with the encrypted CIDs of these outputs being stored in the blockchain. Subsequently, the $SNEncryCID$ of sub-networks that have undergone inference are recorded in a list, referred to as $InferedSNEncryCID\_List$. Among the remaining $SNRecord$s, the smart contract searches for $SNRecord$s with $PreSNEncryCID$ being a subset of the current $InferedSNEncryCID\_List$. This signifies that the inputs for these sub-networks have been generated, enabling inference to be performed on them. The outputs of these sub-networks are stored in the IPFS, and the encrypted CIDs of the outputs are stored in the blockchain. The smart constract updates the $InferedSNEncryCID\_List$ thereafter, then continue following the above process to perform inference on the remaining sub-networks and record their outputs in the blockchain until all sub-networks have been inferred, that is, until the lengths of $InferedSNEncryCID\_List$ and $SNID\_List$ are the same. The record of a model output in the blockchain is designed as follows:

$$\begin{array}{cc}\hfill ModelOutputRecord=& \{ModelOutputID,ModelID,ModelInputEncryCID,\hfill \\ & SNOutput\_Dict,Time\},\hfill \end{array}$$

where $SNOutput\_Dict=\{SNEncryCID:SNOutputEncryCID\}$, and

$$ModelOutputID=\mathrm{Hash}(ModelID,ModelInputEncryCID,Time).$$

(3)

Here, $ModelInputEncryCID$ is the encrypted CID of the $ModelInput$, $SNOutputEncryCID$ is the encrypted CID of the outputs of the sub-networks. We have designed the above process as a method named ModelInference() within the smart contract MC, as shown in Algorithm 5. There are three implementation methods for sub-network inference: off-chain inference, on-chain inference, and hybrid inference. Off-chain inference refers to the scenario where, during automated inference, the sub-networks in the IPFS are not uploaded to the blockchain. Instead, the smart contract allocates off-chain trusted computing resources randomly, the corresponding computing platform, using the decrypted CIDs, retrieves the sub-network and input files from the IPFS and performs the off-chain inference, then uploads the encrypted CIDs of the output file to the blockchain. On-chain inference refers to the scenario where, during automated inference, the sub-network and input files in the IPFS are gradually uploaded to the blockchain as the smart contract invoked. Then the smart contract uses on-chain computing resources to perform the inference. Hybrid inference refers to a situation where, during automated inference, the smart contract can utilize both off-chain and on-chain computing resources. Algorithm 5 utilizes the off-chain inference method; see Algorithm 6.

Algorithm 5 ModelInference(): Performing model inference after tamper-proof checking and permission granting

Require: $ModelID,ModelKey,OutputKey,ModelInput$

Ensure: $\mathrm{OK}$ or $\mathrm{Error}$

1: $ModelInputCID\leftarrow \mathrm{UploadToIPFS}\left(ModelInput\right)$   2: $ModelInputEncryCID\leftarrow \mathrm{Encrypt}(ModelInputCID,OutputKey)$   3: $SNOutput\_Dict\leftarrow \mathrm{Null}$   4: $InferedSNEncryCID\_List\leftarrow \mathrm{Null}$   5: $InferedSNID\_List\leftarrow \mathrm{Null}$   6: while $InferedSNID\_List.\mathrm{Size}\left(\right)!=SNID\_List.\mathrm{Size}\left(\right)$ do   7: $\hspace{1em} CurrSNID\_List\leftarrow ModelRecord.SNID\_List-InferedSNID\_List$   8: $\hspace{1em} CurrSNInputEncryCID\_List\leftarrow \mathrm{Null}$   9: $\hspace{1em} CurrSNEncryCID\_List\leftarrow \mathrm{Null}$ 10:    for $k=1$ to $CurrSNID\_List.\mathrm{Size}\left(\right)$ do 11: $\hspace{1em} \hspace{1em}SNRecord\leftarrow \mathrm{APIstub}.\mathrm{GetState}(CurrSNID\_List[k\left]\right)$ 12:        if $SNRecord.PreSNEncryCID\subseteq InferedSNEncryCID\_List$ then 13: $\hspace{1em} \hspace{1em}\hspace{1em}CurrSNEncryCID\_List.\mathrm{Add}(SNRecord.SNEncryCID)$ 14:          if $SNRecord.PreSNEncryCID==\mathrm{Null}$ then 15: $\hspace{1em} \hspace{1em}\hspace{1em}\hspace{1em}CurrSNInputEncryCID\_List.\mathrm{Add}\left(\right[ModelInputEncryCID\left]\right)$ 16:          end if 17: $\hspace{1em} \hspace{1em}\hspace{1em}CurrSNInputEncryCID\_List.\mathrm{Add}\left(\right[SNOutput\_Dict(SNRecord.$$PreSNEncryCID\left)\right])$ 18: $\hspace{1em} \hspace{1em}\hspace{1em}InferedSNID\_List.\mathrm{Add}(SNRecord.SNID)$ 19:        end if 20:    end for 21:    for $k=1$ to $CurrSNEncryCID\_List.\mathrm{Size}\left(\right)$ do 22: $\hspace{1em} \hspace{1em}CurrSNOutputEncryCID\leftarrow \mathrm{OffChainSNInference}(CurrSNInputEncryCID\_$$List\left[k\right],$ 23: $\hspace{1em} \hspace{1em}CurrSNEncryCID\_List\left[k\right],ModelKey,OutputKey)$ 24:        if $CurrSNOutputEncryCID==\mathrm{Error}$ then 25:          return $\mathrm{Error}$ 26:        end if 27: $\hspace{1em} \hspace{1em}SNOutput\_Dict.\mathrm{Add}(CurrSNEncryCID\_List[k]:CurrSNOutputEncryCID)$ 28:    end for 29: $\hspace{1em} InferedSNEncryCID\_List.\mathrm{Add}(CurrSNEncryCID\_List)$ 30: end while 31: $Time\leftarrow \mathrm{Linux}.\mathrm{time}.\mathrm{Now}\left(\right)$ 32: $ModelOutputID\leftarrow \mathrm{Hash}(ModelID,ModelInputEncryCID,Time)$ 33: $\mathrm{APIstub}.\mathrm{PutState}(ModelOutputID,\mathrm{Marshal}(ModelID,ModelInputEncryCID,$ 34: $SNOutput\_Dict,Time)$ 35: return $\mathrm{OK}$

Algorithm 6 OffChainSNInference(): Performing sub-network inference through off-chain computation

Require: $SNInputEncryCID\_List,SNEncryCID,ModelKey,OutputKey$

Ensure: $SNOutputEncryCID$ or Error

1: $SNInput\_List\leftarrow \mathrm{Null}$   2: for $k=1$ to $SNInputEncryCID\_List.\mathrm{Size}\left(\right)$ do   3: $\hspace{1em} SNInputCID\leftarrow \mathrm{Decrypt}(SNInputEncryCID\_List[k],OutputKey)$   4: $\hspace{1em} SNInput\leftarrow \mathrm{DownloadFromIPFS}\left(SNInputCID\right)$   5: $\hspace{1em} SNInput\_List.\mathrm{Add}\left(SNInput\right)$   6: end for   7: $SNCID\leftarrow \mathrm{Decrypt}(SNEncryCID,ModelKey)$   8: $SN\leftarrow \mathrm{DownloadFromIPFS}\left(SNCID\right)$   9: $SNOutput\leftarrow \mathrm{Inference}(SNInput\_List,SN)$ 10: if $SNInput\_List==\mathrm{Null}\mathrm{or}SN==\mathrm{Null}\mathrm{or}SNOutput==\mathrm{Null}$ then 11:   return $\mathrm{Error}(“\mathrm{The}\mathrm{inference}\mathrm{fails}.$”) 12: end if 13: $SNOutputCID\leftarrow \mathrm{UploadToIPFS}\left(SNOutput\right)$ 14: $SNOutputEncryCID\leftarrow \mathrm{Decrypt}(SNOutputCID,OutputKey)$ 15: return $SNOutputEncryCID$

4. Experiments and Analyses

In this section, we first apply the proposed solution to vehicle detection in intelligent transportation, demonstrating how this solution enhances the reliability of vehicle detection by improving model security through a functional demonstration. Then, a security and applicability analysis of the proposed solution is conducted. Finally, we present experimental results and analysis regarding system efficiency of the proposed solution, consisting of two parts: an analysis of system efficiency based on the above use case and an analysis of the on-chain efficiency through results of a simulation experiment. All experiments were conducted using an platform based on the Hyperledger Fabric consortium blockchain [44]. A distributed container network comprising four nodes was built on the Docker Swarm platform. Data encryption was performed using the Advanced Encryption Standard [30]. The hashing algorithm used is SHA-256 [45], and when using SHA-256, all inputs were concatenated into a single string before being input into the algorithm.

4.1. A Use Case in Intelligent Transportation

4.1.1. Background and Settings

Vehicle detection is an important issue in intelligent transportation, and methods based on deep learning have become a research focus in this field [46]; one category of methods is based on deep learning-based foreground segmentation [47,48]. This experiment is conducted based on a foreground segmentation network [49]. The network consists of an encoder, a Feature Pooling Module (FPM), and a decoder. Two skip-connection modules are used between the encoder and decoder to merge low-level and high-level features, thereby improving detection performance. We trained this model using a portion of frames from the “highway” video in the CDnet2014 dataset [50] along with their annotations, making it a vehicle detection network, as shown in Figure 2. The training process strictly followed the experimental settings in Section 2 of [49]. And the used training code is from https://github.com/lim-anggun/FgSegNet_v2 (accessed on 10 August 2023). Figure 3a displays three video frames from “highway” that were not used for training, while Figure 3b shows the inference results of the trained model on these three video frames. It can be observed that the model performs well in detecting vehicles in different scenarios of the same scene. All vehicles are segmented, and their contours are fairly accurate. However, if the model encounters a tampering attack, the detection results can change significantly. Figure 3c shows the inference results of injecting Gaussian noise into the skip connection module SN5’s convolutional kernels. It can be seen that all detection results become unrecognizable. In real-life scenarios, computer vision companies develop deep models for vehicle detection and provide detection services to users, such as traffic management departments, through a model service system. Vehicle detection must be reliable, and if the model is subjected to the tampering attacks mentioned above during the service, it could lead to severe incidents. Sustainable model services are crucial to ensuring reliable vehicle detection.

4.1.2. Functional Demonstration

In this section, we demonstrate how the proposed solution, based on the vehicle detection model in Figure 2, ensures the reliability of detection from the perspectives of storage, tamper-proof checking, permission management, and inference. The following experiment involves two users, User1 and User2, with User1 acting as the model owner, and User2 as the model requester. Two servers, each equipped with a RTX 3090 GPU, were used for model inference.

First, the vehicle detection model was well-distributed and stored based on the blockchain and the IPFS. The dispersion of model information increases the difficulty of obtaining the complete model information and prepares for tamper-proof checking, permission management, and inference in terms of data structure. The model owner initially divided the model into sub-networks. There are various ways to achieve this model division, and one of these ways is depicted in Figure 2. The model was divided into nine sub-networks. Subsequently, based on this division, the model owner uploaded these sub-network files to the IPFS, as illustrated in Figure 4a. In Figure 4a, SN1.h5, SN2.h5,…, SN9.h5 represent the files corresponding to the sub-networks, and beneath each file name is its CID. The interconnections among the nine sub-networks are defined using two lists:

$$\begin{gathered} SNF\_List=[\mathrm{SN}1,\mathrm{SN}2,\mathrm{SN}4,\mathrm{SN}5,\mathrm{SN}6,\mathrm{SN}7,\mathrm{SN}9,\mathrm{SN}8,\mathrm{SN}3], \\ PreSNF\_List=\left[\right[],[\mathrm{SN}1],[\mathrm{SN}3],[\mathrm{SN}2],[\mathrm{SN}1],[\mathrm{SN}4],[\mathrm{SN}6,\mathrm{SN}8],[\mathrm{SN}5,\mathrm{SN}7],[\mathrm{SN}2\left]\right]. \end{gathered}$$

In this setup, $SNF\_List$ keeps track of the order in which the sub-networks are uploaded to the IPFS, while $PreSNF\_List$ records the preceding sub-networks adjacent to each sub-network in $SNF\_List$. By using the AddModel() method in the smart contract MC, the records of the sub-networks and the model were stored in the blockchain, completing the model storage process. Subsequently, in Figure 4b, we used the FindByID() method in MC, indicated by blue underlines, to display the records of SN8, SN3, and the model in the blockchain. As can be seen, the record of SN3 is preceded by the record of SN8, therefore the $PreSNID$ of the record of SN3 is the $SNID$ of the record of SN8. The $PreSNEncryCID$ of the record of SN8 is the $SNEncryCID$ of SN5 and SN7, and the $PreSNEncryCID$ of the record of SN3 is the $SNEncryCID$ of SN2. The order of $SNID\_List$ in the record of the model is consistent with that of $SNF\_List$. The model owner backed up the $SNEncryCID$ from each sub-network record, indicated by red underlines, and the $OriginalSNID\_List$ from the model record, also indicated by red underlines, is the basis for model tamper-proof checking.

Although the model has been stored in both the blockchain and the IPFS, its security is not absolute, and there is a possibility of model tampering due to security vulnerabilities. To ensure the reliability of detection, it is essential to ensure the correctness of the vehicle detection model. This requires a tamper-proof checking method to dynamically and continuously monitor the correctness of the model. For the model tamper-proof checking, we demonstrate the results under two scenarios. The first scenario involves the tampering of model information in the blockchain. Instead of directly tampering with the model information in the blockchain, we uploaded a tampered sub-network record to the blockchain. Based on the design of the sub-network record and model record structures, the $SNID$ of subsequent sub-network records changed as a result, leading to changes in the model record. Therefore, we also uploaded the subsequent sub-network records and the corresponding model record to the blockchain. Specifically, for the sub-network SN8, we removed the encrypted CID of SN5 from its $PreSNEncryCID$. As a result, in Figure 5a, the $PreSNEncryCID$ of SN8’s record has changed compared to Figure 4b, causing a change in its $SNID$. The subsequent record after SN8’s record, which is the record of SN3, also experienced $SNID$ changes. Ultimately, the model record has changed as well. We utilized the AddModel() method from the smart contract MC to upload the tampered sub-network and model records to the blockchain, and then utilized the FindByID() method from MC to display these records, as indicated by blue underlines. By inputting the $OriginalSNID\_List$ and the $ModelID$ of the tampered model record, we conducted tamper-proof checking on the model using the TPCI() method in MC, as indicated by blue underlines. The results are shown in Figure 5b. It can be observed that the $SNID$ of the first tampered sub-network record, which corresponds to the record of SN8 marked with a red underline, has been detected during the checking and indicated as an error, as highlighted by a red underline. The second scenario involves tampering with the information stored in the IPFS. We deleted a file belonging to one of the sub-networks in the IPFS. Specifically, the file of SN5 was deleted. By inputting the original model record’s $ModelID$ and the CID encryption key $ModelKey$, as indicated by a blue underline, we conducted tamper-proof checking on the model using the TPCII() method in the smart contract MC. The results are shown in Figure 6. It is evident that the CID associated with SN5 cannot be retrieved, resulting in errors highlighted by red underlines.

The vehicle detection model service is not open to the public. Considering model security, service providers need to control user access to the model. In the proposed solution, this is achieved through role-and-time-based permission management. After storing the model in the IPFS and the blockchain, User1 called the AddPermission() method of MC to grant User2 permission to use the model. Subsequently, User2 generated and verified a request to use the model by invoking the CheckPermission() method of the smart contract MC. Figure 7a illustrates the successful addition of a permission record through the QueryPermission() method of MC. During the query, the $RequestID$ was used as input, as indicated by the blue underline. The result marked with a red underline confirms the existence of this permission record, indicating that User1 allows User2 to use the model. The time displayed here represents the expiration time of the permission. After successfully granting permission, User1 input the model information into the smart contract for model inference without providing this information to User2. When the permission expires, User2 called the CheckPermission() method again to verify the request to use the model, as indicated by the blue underline in Figure 7b, and the system displays that the permission has expired, as shown by the red underline.

The model inference produces the vehicle detection results, and this step also considers the security of both the model and the detection results. Once the permission granting was successful, User1 input the $ModelID$ and $ModelKey$ into the ModelInference() method of the smart contract MC and User2 input the $ModelInput$ and $OutputKey$ into ModelInference(). The automatic model inference process began. In our experiment, considering the model security, the inference of sub-networks took place in two off-chain servers. Sub-network SN1 started the inference, followed by SN2 and SN6. Next were SN3 and SN5, then SN4, SN7, SN8, and finally SN9. $ModelInput$ and the outputs from each sub-network were sequentially uploaded to the IPFS, see Figure 8a. The $ModelInput$ is a frame from the “highway” video named “highway_in000019.jpg”. The CIDs of the model input and sub-network inference outputs were encrypted and then uploaded to the blockchain as parts of the record of the model inference result. Figure 8b displays the records of the model outputs in the blockchain by invoking the FindByID() method, as indicated by a blue underline. The $SNOutput\_Dict$, marked with a red underline, is a dictionary containing the mapping of encrypted CIDs between each sub-network and its inference result. We have printed them all for reference.

4.2. Security and Applicability Analysis

In this section, we begin by comparing the proposed solution with three existing solutions [17,18,19] based on the used technologies and their applicability to deep models, as shown in

Table 2. Comparisons of the proposed solution and some existing solutions on security and applicability.

Perspective	Solution of [17]	DeepRing [18]	Solution of [19]	Proposed Solution
Storage	Blockchain (the model is in the blockchain.)	Blockchain (the model is in the blockchain.)	IPFS + Blockchain (the model is in the IPFS.)	IPFS + Blockchain (the model is in the IPFS and the blockchain.)
Data Encryption	Considered (the type is not specified.)	Asymmetric encryption and symmetric encryption	Symmetric encryption	Symmetric encryption
Auditing and Monitoring	Not considered	Layer-wise asymmetric encryption and decryption	Not considered	Tamper-Proof checking
Access Control	Not considered	Not considered	ABAC based on user roles	ABAC based onuser roles and expiration time
Inference	Not considered	Not considered	Centralized inference	Distributed inference
Applicability to Deep Models	All	Part	All	All

. We then proceed to further analyze the security and applicability of the proposed solution in the context of the use case presented in Section 4.1.

In the proposed model storage method, the entire model information is not fully stored in either the IPFS or the blockchain. Instead, the information of the entire model is divided into intra-subnetwork information and inter-subnetwork connection information. The information within a sub-network is stored in the IPFS, while the inter-subnetwork connection information is stored in the blockchain. Compared to the solutions proposed in [17,18,19] that store the model fully in the blockchain or the IPFS, the proposed model storage method is more decentralized, making it more difficult to steal the complete model information and thereby enhancing security. Moreover the proposed model storage method is not constrained by model architectures, model division methods, or sub-network recording sequences, making it applicable to all deep models. In terms of data encryption, we have chosen a symmetric encryption method to strike a balance between security and system efficiency.

A method for tamper-proof checking on stored models is presented in the proposed solution. This auditing and monitoring method has not been considered in existing solutions [17,18,19]. This method effectively prevents subsequent errors caused by tampered models, thereby enhancing model security. For example, in the first scenario presented in Section 4.1.2, due to the proposed model inference method invoking sub-network files in the IPFS according to the sub-network records in the blockchain, when the $SNEncryCID$ of sub-network SN5 is deleted from the $PreSNEncryCID$ of SN8’s record, issues arise with SN8’s input, causing the automated inference fails. In the second scenario presented in Section 4.1.2, although the sub-network records in the blockchain are correct, there is no SN5 file available in the IPFS for invocation, causing SN5 to fail to be inferred, leading to a failure of the model inference. However, it does not mean that automated inference cannot function properly whenever model information is tampered with. Consider the scenario in Figure 3c where Gaussian noise is added to the convolution kernel of SN5—if its file in the IPFS is tampered with, there is a change in its CID. And the changed CID is encrypted and uploaded to the blockchain, thus tampering with the model information in the blockchain as well. In this case, automated inference can proceed normally, but the inference results will be compromised. To rectify this scenario, the TPCI() method can be used to check and restore the tampered parts in the blockchain, followed by using the TPCII() method to check and restore the tampered parts in the IPFS.

In the proposed solution, we enhance model security by utilizing permission management based on user roles and expiration times, a feature not considered in existing solutions [17,18,19]. While [19] also employs the ABAC for granting permissions to users, this method is solely based on user roles. In contrast, our proposed permission management method combines user roles and expiration times, aligning more closely with the practical requirements of model services for deep learning. Furthermore, unlike [19], in our solution, once usage permissions are granted for a model, the model owner does not provide users with the checked and updated $ModelID$ or $ModelKey$; instead, this information is directly input into the smart contract. As a result, users can only use the model without being able to access or download it, further enhancing model security.

We also introduce a distributively executable model inference method in our proposed solution, which has not been considered in existing solutions [17,18,19]. The implementation of automated inference involves three possible strategies: off-chain inference, on-chain inference, and hybrid inference. Off-chain inference has lower security, as there is a possibility of model leakage or tampering when off-chain computational resources are compromised. Nevertheless, unlike [19], due to the highly decentralized nature of our storage method, an individual computing resource are used only for several sub-networks during inference, and the connection information of sub-networks is stored in the blockchain, minimizing the likelihood of complete model information leakage. On-chain inference offers higher security during the inference process, with a lower likelihood of model leakage or tampering. However, this will impose a significant storage and computational burden on the blockchain. For hybrid inference, the security and system efficiency fall between that of off-chain inference and on-chain inference.

4.3. Experimental Results and Analyses on System Efficiency

4.3.1. System Efficiency Analysis Based on the Use Case in Intelligent Transportation

In this part, we analyze the system efficiency of the proposed solution based on the use case presented in Section 4.1. In the experiment conducted in Section 4.1, we collected experimental results on nine aspects reflecting system efficiency, involving required storage space, time consumed for data transmissions, and time consumed for computations. The results are presented in the third column of

Table 3. Comparisons of the proposed solution and the solution proposed in [19] on some aspects of the system efficiency.

Perspective		Proposed Solution	Solution Proposed in [19]
Storage	Storage space required in IPFS during model storage (MB)	35	64
	Storage space required in blockchain during model storage (${10}^{-3}$ MB)	$0.75$	$0.06$
	Storage space required in blockchain during model inference (MB)	132	$0.81$
	Storage space required in blockchain during model inference (${10}^{-3}$ MB)	$0.34$	$0.09$
Data Transmission	Time consumed for file uploading to IPFS during model storage (s)	$0.77$	$0.34$
	Time consumed for file uploading to IPFS during model inference (s)	$1.25$	$0.13$
	Time consumed for file downloading from IPFS during model inference (s)	$1.93$	$0.53$
Computation	Time consumed for hashing during model storage (${10}^{-6}$ s)	$33.86$	$1.75$
	Time consumed for off-chain model inference (s)	$6.02$	$6.76$

. Under the same experimental conditions, we also conduct the same experiment using the solution presented in [19], which does not involve model division and stores the entire model file in the IPFS while recording the model in the blockchain. The results are shown in the last column of Table 3. As analyzed in Section 4.2, it is already known that the proposed solution offers better security compared to the solution proposed in [19].

In terms of storage space, we measured the space required in both the blockchain and the IPFS during the model storage and inference processes. In the proposed solution, during the model storage, nine sub-network files were stored in the IPFS, consuming less space compared to storing the entire model file in the IPFS. However, recording the models in the blockchain requires nine sub-network records and one model record for the proposed solution. In contrast, the solution without model division only requires one model record. It is evident that the proposed solution consumes more on-chain space. The proposed solution actually requires less total storage space in the model storage. During the model inference, the inference results are stored in the IPFS, occupying a significant amount of storage space, its on-chain record includes the mappings between encrypted CIDs of each sub-network and its inference result, which increases the required on-chain storage space. Regarding data transmission, the main time consumption is concentrated on uploading data to the IPFS and downloading data from the IPFS. In the proposed solution, during the model storage, the sub-network files were uploaded to the IPFS. We measured the time it took to upload each sub-network file, as shown in Figure 9c. Comparing Figure 9a,c, it is evident that the uploading time is related to the file size, but not solely dependent on it; it is also influenced by factors like network speed. The same applies to downloading, as shown in Figure 9e. Although the size of the entire model file is close to two times of the size of the sub-network files, the time it takes to upload the entire model file to the IPFS is only less than half of the time it takes to upload all the sub-network files. The model inference involves the uploading and downloading of the model input and sub-network inference results in the IPFS, as well as the downloading of sub-network files. We also measured the time for uploading and downloading each file, as shown in Figure 9d–f. Because it involves the uploading and downloading of intermediate model output results, the proposed solution consumes significantly more time than the solution that does not divide the model into sub-networks. In terms of computation, we respectively examine the time consumed for on-chain hashing during model storage and the time consumed for off-chain sub-network inference. On-chain hashing during model storage is used to compute identifiers for the sub-network records and the model record. The specific time consumption can be seen in Figure 10a. Because ten hashing calculations are required, the time consumed is significantly higher compared to when the model is not divided into sub-networks, which requires only one hashing. The proposed solution employs off-chain inference for sub-networks. Although off-chain inference has lower security compared to on-chain inference, it is currently a more feasible inference method considering system efficiency. Off-chain inference is conducted on two servers, and we measured the inference time for each sub-network, as shown in Figure 10b. In this process, inference for SN2 and SN6 can be performed simultaneously, and so can SN3 and SN5, resulting in a shorter total time compared to the scenario where the model is not divided and inferred on a single server. It should be noted that because each server is equipped with only one RTX 3090 GPU, the inference time here does not meet the requirements for real-time detection.

From the above experimental results, it can be seen that while the proposed solution has certain advantages in aspects such as the space required in the IPFS during model storage and the time consumed for off-chain inference, it still consumes more in terms of overall space and time compared to the solution without model division. This is the cost of increased security. We found that the number of sub-networks is an important parameter in the proposed solution, and all aspects examined in Table 3 are related to it. In the use case in Section 4.1, dividing the vehicle detection model into nine sub-networks may be not the best choice. As the number of sub-networks increases, the security of the model improves, but the storage and computations required by the system also increase. In the practical use of the proposed solution, a balance between security and system efficiency can be achieved by selecting an appropriate number of sub-networks to maintain the sustainability of model services. This mechanism, which can balance security and system efficiency, is a feature of the proposed solution that existing solutions [17,18,19] have not considered.

4.3.2. On-Chain Efficiency Analysis Based on A Simulation Experiment

In this part, we analyze the efficiency and sustainability of blockchain from the perspective of on-chain storage and computation. We first conduct a simulation experiment and then proceed with the analysis based on the results of this experiment. To simplify the experiment, we make the assumption that, except for the first sub-network of the model, each subsequent sub-network has only one preceding sub-network. Since the proposed solution does not impose any restrictions on the order of recording sub-networks, we also assume that the first recorded sub-network is not the first sub-network of the model. Based on these assumptions, the simulation experiment examines the impact of increasing the number of sub-networks on on-chain storage and computation. Specifically, we consider the number of sub-networks to be 2, 3, 4, 5, 6, 7, 8, 9, and 10, and analyze the on-chain space required by the sub-network and model records, as well as the time consumed by hashing for these records. The experimental results are shown in Figure 11.

From the results, it can be observed that the on-chain storage space required by the sub-network and model records, as well as the model inference result record, exhibits a trend of basic linear growth with the increase in the number of sub-networks. When the number of sub-networks is 10, these records occupy approximately 1.5 KB of storage space. Therefore, in the proposed solution, the space occupied by model and its inference result records in the blockchain is relatively small, providing room for other on-chain information, such as permission records, ensuring sustainability in terms of storage. During the model storage, hashing is required to compute identifiers for all sub-network records and the model record, as shown in Equations (1) and (2). The time consumed for this operation also exhibits basic linear growth with the increase in the number of sub-networks. During the model inference, hashing is only needed once for recording the inference result. Since the length of the input string remains unchanged, see Equation (3), the time consumed for this operation remains relatively constant. When the number of sub-networks is 10, the total time consumed for on-chain hashing is approximately 40 $\mathsf{\mu }$s. Thus, in the proposed solution, the computational time involved in on-chain recording of the model and its inference results is not substantial. The complexity of model tamper-proof checking and on-chain computations for model inference is also related to the number of sub-networks. With reasonable control over the number of sub-networks, the computational load within the blockchain remains manageable. Furthermore, the proposed permission management method can restrict model usage based on user roles and expiration times, which also helps control the computational load in the blockchain. Therefore, on-chain computations in the proposed solution exhibit sustainability.

5. Conclusions

This paper proposes a sub-network-based solution integrating blockchain and IPFS for model storage and inference. This solution aims to balance model security and system efficiency while being widely applicable, serving as a foundation for the development of systems that support sustainable model services. The proposed solution utilizes a highly decentralized storage method, dividing a deep model into sub-networks and inter-sub-network components. Sub-networks are stored in the IPFS, while connectivity between sub-networks, as well as records of sub-networks and models, are stored following a chained structure in the blockchain. To prevent model security issues arising from system vulnerabilities, a two-stage tamper-proof checking method is designed to monitor model security. In addition to enhancing the supervision of model correctness, the proposed solution strengthens control over model usage by employing permission management based on user roles and expiration times. The solution introduces an automated inference method and utilizes the IPFS for storing model inputs and sub-network outputs, alleviating the on-chain storage burden. Trusted off-chain computing resources are used for distributed inference, mitigating the on-chain computational burden. The proposed solution is applicable to all deep models. We applied this solution to vehicle detection in intelligent transportation and analyzed its security, applicability, and system efficiency. Experimental results demonstrate that the proposed solution offers good security and applicability. By choosing an appropriate number of sub-networks and employing the dual-attribute-based model permission management method, model security and system efficiency can be balanced, ensuring the sustainability of on-chain storage and computational resources. Issues for further research on this proposed solution include determining how to select the number and division method of sub-networks for a given deep model, as well as whether there are quantitative criteria for the selection. We will address these issues in future work.