Cuckoo-Store Engine: A Reed–Solomon Code-Based Ledger Storage Optimization Scheme for Blockchain-Enabled IoT

Abstract

As the distributed ledger technology underlying cryptocurrencies such as Bitcoin and Ethereum, blockchain has empowered various industries, such as supply chain management, healthcare, government services, e-voting, etc. However, the ever-growing ledger on each node has been the main bottleneck for blockchain scalability as the network scale expands, which worsens in blockchain-enabled IoT scenarios with resource-limited devices. With the support of the Reed–Solomon (RS) code, the Cuckoo-Store (CS), a ledger storage optimization engine, is proposed in this paper to dramatically decrease the storage burden on each node by encoding the ledger as data segments with redundancy and distributing them to multiple nodes. These distributed data segments can be collected and decoded using RS code to recover the original ledger. Furthermore, the Cuckoo filter (CF) is used to guarantee the integrity of the encoded segments, which helps detect the forged segments and facilitates the process of ledger recovery. Theoretical analysis and simulation results show that the CS engine can decrease the storage in each node by more than 94%, and the original ledger can be recovered efficiently with acceptable communication overheads.

1. Introduction

As the distributed ledger technology (DLT) underlying cryptocurrencies such as Bitcoin and Ethereum, blockchain implements decentralization, immutability, transparency, trustlessness, and traceability, with the support of P2P networking, cryptography, chained structure, and the consensus algorithm [1,2]. With the development of industrial digitization, the blockchain has played an essential role as the trust anchor for multi-party cooperation and resource sharing in various fields, such as digital twins (DT), healthcare, smart grids, and supply chain management [3,4,5]. A typical application is the blockchain-enabled Internet of Things (IoT), where all sensors act as the blockchain nodes to maintain the entire ledger, and collected data are exchanged in the form of transactions [6]. For example, blockchain-enabled IoT has been applied in various fields, such as Industry 4.0, smart cities, and identification [7,8,9].

However, the ever-growing ledger on each node has been the main bottleneck for blockchain scalability as the network scale expands, which worsens in blockchain-enabled IoT scenarios with resource-limited devices. The limited storage scalability is caused by the blockchain storage model, where all nodes have to maintain the replicas of the entire ledger to guarantee the core characteristics of the blockchain, such as decentralization, transparency, and traceability [10]. By the end of March 2023, the storage for the ledger had exceeded 470 GB and 900 GB in Bitcoin and Ethereum, and the growth rates were predicted to reach 50 GB and 400 GB per year, respectively [11]. Considering the expansion rate of the IoT scale (projected to rise from 10 billion devices in 2018 to 64 billion in 2025 [12]), the storage requirements for IoT devices to participate in blockchain activities independently will rise rapidly (10× or even 100×), which damages the decentralization and further limits the scalability of the transaction throughout, in addition to the consensus efficiency.

Various solutions have been proposed to address storage issues, including the schemes of role division, channel, sharding, and storage clusters. In [1,13], the nodes are divided into light and full nodes according to their resources. The former only maintains the lightweight chain formed by the block headers, while the latter maintains the entire ledger, including all headers and bodies. Correspondingly, the full nodes can participate in all the blockchain activities independently, while the light nodes have to rely on the full nodes with simplified payment verification (SPV). When using role division, the storage burden on the light node is decreased, but with the cost of damaged decentralization and robustness. As indicated in [14], a blockchain with more nodes performing independently has better decentralization and robustness. Poon et al. constructed an off-chain transaction channel called the Lightning Network [15]. Once the channel is constructed, all transaction executions are transferred into the off-chain. Only the transactions for channel opening and closing are recorded on-chain, which decreases the ledger growth rate. However, the main purpose of the Lightning Network is to improve the throughput, which makes nodes more vulnerable to attacks because both parties involved in the transaction must be online and must log in with their private keys. A sharding scheme named Elastico was proposed in [16] to improve blockchain scalability, where all nodes are divided into multiple consensus units called shards, and each node only maintains the shard-related ledger to dramatically decrease the local storage burden. However, the complicated architecture is necessary for cross-shard communication and committee reshuffling, which improves the deployment difficulty. Perard proposed the Erasure Code-based low storage (ECLS) blockchain in [17], where the block is encoded by RS code, and each node only maintains a subset of the encoded data instead of the entire ledger. The original block could be recovered based on the RS code’s maximum distance separable (MDS) property after distributed subsets are collected correctly. However, there is no efficient mechanism to inspect the integrity of the collected subsets, which means the ledger recovery process is easily interrupted due to low fault tolerance. Similarly, Qi et al. proposed a reliable storage partition scheme, RS-Store, for the blockchain ledger in [18] by combining the RS code and the Byzantine Fault Tolerance (BFT) consensus protocol, where the storage burden on each node is decreased by distributing the encoded transactions into multiple nodes. However, the RS-Store is designed for the permissioned blockchain and makes a huge modification to the original workflow, resulting in compatibility issues when applying the scheme to permissionless blockchains such as Bitcoin and Ethereum. Furthermore, there are cluster storage schemes based on other techniques, such as the schemes proposed in [19,20], with the support of the residue number system (RNS) and distributed hash table (DHT), respectively. These schemes can decrease the storage burden on each node but have individual limitations in terms of communication overheads and integrity.

In this paper, a storage optimization engine, Cuckoo-Store (CS), is proposed to dramatically decrease each node’s storage burden with enough fault tolerance, where the RS encoding is used to translate the block bodies as encoded segments with redundancy, and Cuckoo filter (CF) is used to guarantee the integrity of the encoded results and facilitate the ledger recovery process. With the support of the CS, each node selects subsets of the encoded results when synchronizing the new blocks. They can recover the original ledger by requesting the neighbor nodes for the other subsets and perform the decoding algorithm provided by the RS code, where the CF helps to detect the forged responses via membership checks and checks whether the decoded segments are correct. The major contributions of this paper are summarized as follows:

1.

We first propose the CS engine to dramatically reduce each node’s storage burden without damaging the core functions of the blockchain. The CS engine is consensus-algorithm-independent and slightly modifies the original block structure and workflow, lowering the deployment difficulty;

2.

The RS encoding is used to translate the block body as encoded segments with redundancy. The multiple packed transactions are grouped as the basic unit to be processed and are represented by a matrix containing the raw and redundant data. The encoded segments are distributed into multiple nodes to decrease the storage burden on each node. The original block can be recovered with the support of the decoding algorithm of the RS code, where the grouped data segments are still the basic units to be recovered, and the redundant data provide the extra capability for error detection and correction to avoid interruptions caused by a transmission error or node crash. The recoverability of the RS code guarantees the traceability of the blockchain after the CS engine is applied;

3.

We use the CF to guarantee the integrity of the encoded results during the block proposal. During the ledger recovery process, the CF is used to detect the forged segments from the malicious nodes before decoding, which facilitates the recovery process by preventing a number of forged segments from participating in the decoding process and removing the detected malicious nodes from the neighbor list. After decoding, nodes can also verify the correctness of decoded segments by CF. With the support of the CF, the CS engine can provide better fault tolerance and more efficient error detection and correction than the existing RS-based schemes;

4.

We implement the PoC (Proof of Concept) deployment of the CS engine. The simulation results show that with the CS engine’s support, the storage for the ledger on each node is decreased dramatically (more than 94%), which is comparable with the existing schemes. Furthermore, the original ledger can be recovered more efficiently in the CS engine, which means the system remains stable even under malicious attacks.

The rest of this paper is organized as follows. The basic techniques underlying the CS engine are introduced in Section 2. Section 3 describes the construction of the CS engine, including the processes for ledger distribution and recovery. The theoretical analysis and simulation are performed in Section 4 and Section 5, respectively. Finally, the paper is concluded in Section 6.

2. Related Techniques

This section introduces the techniques related to the construction of the Cuckoo-Store engine. The blockchain’s chain structure and transaction lifecycle are first described in Section 2.1. Then, the encoding and decoding processes of the Reed–Solomon code are introduced in Section 2.2. Finally, the Cuckoo filter is introduced in Section 2.3, including its construction and membership check process.

2.1. Blockchain Ledger Structure and Transaction Lifecycle

Blockchain is a back-linked list of blocks in chronological order, where each block is composed of a body and a header. The body records all confirmed transactions, which guarantees the blockchain’s decentralization, immutability, and traceability [21,22]. The header maintains the metadata related to the body. As shown in Figure 1, all transactions packed in the body are hashed together in pairs to obtain the Merkle root, which is stored in the header to guarantee the body’s integrity. There are other fields in the header, such as the timestamp, the difficulty target, the nonce, and the parent hash. The timestamp is a unique serial number to determine the order of the blocks. The difficulty target is a numeric value used to adjust the mining difficulty according to the estimated computing power, ensuring that the blocks are constructed efficiently. The nonce is also a numeric value repeatedly altered as the input of the hash process, referred to as Proof-of-Work (PoW), to meet the difficulty level restrictions. The parent hash is the previous block’s hash value, and the hash of this block will be referenced in the next header, which links the adjacent blocks tightly. Any modification to a block causes an enormous recalculation of all subsequent blocks, making the blockchain’s deep history immutable unless under the 51% attack [23].

Taking Ethereum as an example, for a transaction described formally by tx = {txid, (sender, recipient, value), nonce, sig}, the tuple of (sender, recipient, value) indicates the amount of the transferred tokens, and accounts on both sides. The nonce is the number of transactions issued by the sender, and sig is the sender’s signature. Finally, tx can be indexed by txid, which is the unique identifier, via the hash function [24,25]. The transaction is the smallest unit to drive the blockchain system. Transactions are created, relayed, and validated by the nodes and packed into the blocks. The state transitions occur when the transaction is executed and confirmed. The workflow of the blockchain is described in detail as follows:

1.

A new transaction is generated with a complete structure and propagated to the network;

2.

The nodes collect the transactions into their waiting area, called the mempool. Then, these transactions are checked locally to ensure the transferring funds are available and the signatures are valid. The valid transactions remain in the mempool until the packing rules are met, whereas the invalid ones are excluded;

3.

The nodes pack the transactions in the mempool into new blocks, execute them, and update local state sets. Then, PoW is processed independently. The node satisfying the nonce with the difficulty target propagates its block to the network;

4.

The nodes receiving the new block check its validity and append it to the tip of the local chain.

In particular, Bitcoin and Ethereum implement simple payment verification (SPV) for light nodes to confirm transactions with the help of the full nodes. For a specific transaction tx, the light node sends txid to the full node, which returns the Merkle branches corresponding to txid. Then, the light node can confirm the transaction by recalculating the Merkle root with the received branches and compare the result with the Merkle root maintained in the local header.

2.2. Reed–Solomon Code

Error-correcting code is an encoding technique capable of error detection and correction by adding redundancy during the encoding process [26]. Benefiting from the redundancy, the encoded results can always be decoded correctly as long as the errors (such as data loss or forgery) do not exceed the upper limits of the capabilities for error detection and correction, which improve with more redundancy [27]. Thus, error-correcting coding is widely used in data backup and distributed storage [28,29].

The Reed–Solomon (RS) code is a typical implementation of error-correcting code based on the arithmetic in finite fields, where the data are encoded and decoded in bytes by Encoder and Decoder, respectively [30]. In particular, a specific encoding pattern of the RS can be represented formally as RS(n, p), where p is the number of bytes for the original data and n is the number of bytes for the encoded result with redundant data of q bytes added during the encoding. Thus, there is n = p + q.

In the RS(n, p), the original data, D, of b bits can be encoded as follows. First, D is divided in bytes into p parts ($\mathit{d}=[d_1,d_2,\dots ,d_p]$) after padding $(8p-b)$ zero bits, where each part $d_i(1\le i\le p)$ takes one byte $(0\le d_i\le 2^8)$. Then, the Encoder can calculate the redundant vector $\mathit{c}=[c_1,c_2,\dots ,c_q]$ based on d as follows:

$${\mathit{c}}^{\mathit{T}}=G·{\mathit{d}}^T=\left[\begin{array}{cccc}{g}_{11}& g_{12}& \cdots & g_{1p}\\ g_{21}& g_{22}& \cdots & g_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ g_{q1}& g_{q2}& \cdots & g_{qp}\end{array}\right]·\left[\begin{array}{c}{d}_1\\ d_2\\ ⋮\\ d_p\end{array}\right]=\left[\begin{array}{c}{c}_1\\ c_2\\ ⋮\\ c_q\end{array}\right],$$

(1)

where $G=\left[g_{ij}\right](1\le i\le q,1\le j\le p)$ is the generator matrix with q rows and p columns. Finally, the encoding result $D_{enc}$ can be obtained by concatenating d and c as

$${\mathit{D}}_{\mathbf{enc}}\leftarrow Encoder\left(D\right)=[\mathit{d},\mathit{c}],$$

(2)

where $D_{enc}$ is a set with $n=p+q$ elements.

Based on the properties of the RS code, the Decoder can recover the original data, D, based on the encoding result $D_{enc}$ or its subset $D_{sub}$ with no less than $\lceil p+q/2\rceil$ elements ($D_{sub}\subset D_{enc}$, and $|D_{sub}|\ge \lceil p+q/2\rceil$) as follows:

$$D_{dec}\leftarrow V^{-1}·Vector\left(D_{sub}\right),$$

(3)

where V is the decoder matrix with n rows and p columns and $V^{-1}$ is the inverse matrix of V. V can be obtained by concatenating a p-order identify matrix $I_{p\times p}$ and the generator matrix $G_{q\times p}$ as follows:

$$V=\left[\begin{array}{c}{I}_{p\times p}\\ G_{q\times p}\end{array}\right]=\left[\begin{array}{cccc}1& 0& \cdots & 0\\ 0& 1& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & 0\\ g_{11}& g_{12}& \cdots & g_{1p}\\ g_{21}& g_{22}& \cdots & g_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ g_{q1}& g_{q2}& \cdots & g_{qp}\end{array}\right].$$

(4)

In the process of RS-encoded data transmission, some data bits may be corrupted, resulting in inconsistencies between the received data and the encoded data. Through the decoding process of the RS code, these errors can be corrected to make the received data consistent with the original data. For RS(n, p) code, it can correct up to no more than $t=\lfloor (n-p)/2\rfloor$ errors at locations that are known and provided to the algorithm [31].

In practice, the fast Fourier transform (FFT) and its inverse transform (IFFT) could be used to accelerate matrix multiplication and inversion in a finite field with a time complexity of $O\left(nlogn\right)$ for encoding and decoding [32,33]. Furthermore, the ISA-L library provides an efficient implementation for RS encoding and decoding with high performance and low latency with the support of single instruction multiple data (SIMD) instructions [34].

2.3. Cuckoo Filter

The Bloom filter (BF) is a probabilistic, space-efficient data structure that checks set membership quickly [35,36]. The Bloom filter is a bit array with $S_{BF}$ bits and K hash functions ($h_k,k=1,2,\dots ,K$). Initially, all bits in the bit array are set to 0 s. To insert a new item, x, to a set, S, all locations indexed by $h_k\left(x\right)$ should be set to 1 s. Correspondingly, the membership of x in S can be determined if all locations indexed by $h_k\left(x\right)$ in the bit array are 1 s. There will be a false positive for a BF with $C_{BF}$ items inserted, which means BF reports yes by mistake for an item outside of S [37].

The Cuckoo filter (CF) is an improvement of BF with a more compact size and higher query efficiency by introducing the new components of buckets and fingerprints [38,39,40]. Unlike the BF, the CF divides the underlying bit array into multiple buckets as the units to store elements, where the maximum capacity of a bucket is preset as $C_{buk}$. Typically, the number of buckets, $N_{buk}$, in a CF is determined as [39]

$$N_{buk}=\lceil \frac{C_{CF}}{C_{buk}}\rceil ,$$

(5)

where $C_{CF}$ is the maximum capacity of elements that can be inserted.

The fingerprint is the unique identifier for each element in a bucket, which is obtained with a specific hash function, $h_S$, but has length, fl, as follows:

$$finger\left(x\right)=h_S\left(x\right)[:fl],(S=1or2).$$

(6)

Typically, $fl$ can be calculated as [38]

$$fl=⌈log\left(\frac{2·C_{buk}}{FPP}\right)⌉,$$

(7)

where FPP is the false positive rate. The reason for the misjudgment is that two different elements may have the same fingerprint and the same storage location.

Based on $C_{CF}$ and $fl$, the size of the CF ($S_{CF}$) can be determined as follows:

$$S_{CF}=fl·C_{CF}=\frac{fl·N_x}{\gamma },$$

(8)

where $N_x$ is the number of stored elements and $\gamma$ represents the already stored elements divided by $C_{CF}$, also called the load factor [38]. The performance of the CF is affected by $\gamma$, which should be determined carefully to achieve a good trade-off between the false positive rate and the CF’s size [39]. Typically, $\gamma$ is approximately 95% [40]. Initially, $S_{CF}$ bits in the buckets are set to 0 s.

An example of the CF with two hash functions ($h_1$ and $h_2$) and eight buckets ($N_{buk}$ = 8) is shown in Figure 2 [36]. The process to insert a new element, x, with the fingerprint, $finger\left(x\right)$, in Set S is as follows:

1.

Calculate $h_1\left(x\right)$ using the SHA-256 function and insert x into bucket $b_1$ if it is not full, where $b_1=h_1\left(x\right)$ mod $N_{buk}$, and $h_1$ is calculated as follows:

$$h_1\left(x\right)=SH{A}_{256}\left(x\right);$$

(9)

2.

If bucket $b_1$ is full, pick up an old element, y, kick it out of bucket $b_1$ and reinsert it into one of the other buckets. Typically, y is the element first inserted into bucket $b_1$. Then, insert x into bucket $b_1$;

3.

For the old element y kicked out of bucket $b_1$, its new bucket, $b_2$, can be determined based on $h_2\left(y\right)$ mod $N_{buk}$, where $h_2\left(y\right)$ is calculated as follows:

$$h_2\left(y\right)=h_1\left(y\right)\oplus SH{A}_{256}\left(h_1\left(y\right)[:fl]\right).$$

(10)

Correspondingly, the membership of x in S can be determined using two-step checks as follows [37]:

1.

Calculate finger(x) and locate the corresponding two candidate buckets indexed by $h_1\left(x\right)$ mod $N_{buk}$ and $h_2\left(x\right)$ mode $N_{buk}$;

2.

If any one of the inspected buckets reports yes for the existence of x, the membership of x in S is confirmed with $FPP$.

Figure 2. Insert a new element x in the Cuckoo filter.

Figure 2. Insert a new element x in the Cuckoo filter.

Thus, the CF could be used in various scenarios, such as network routing and data storage, by providing membership checks [41,42]. The CF could also be applied to Bitcoin and Ethereum to replace the BF used [43,44].

For a set or vector, S, the polynomial commitment (PC) schemes could also provide the membership proof for an element x to indicate x is in S. Typical PCs (including FRI [45], IPA [46], and KZG [47]) are compared in

Table 1. The typical polynomial commitment (PC) schemes.

Technology	Cryptographic Assumption	Proof Size	Verification Delay
FRI	Hashes only	Large (10–200 KB)	Medium
IPA	Elliptic Curves	Medium (1–3 KB)	High
KZG	Elliptic Curves, Pairings, Trust Setup	Small (≈500 B)	Low

for their cryptographic assumption, proof size for a commitment, and verification delay.

Among the above PCs, the KZG scheme has the most compact proof size and the shortest time for verification. However, KZG is based on the operations on the elliptic curve cyclic groups and the proofs are checked based on the pairing. Based on the widely used paring-friendly curve BLS12_381, compared with the insertion and check of the Cuckoo filter (hash operations), commitment/proof generation and checks in KZG are extremely expensive. Furthermore, the construction of KZG relies on the trusted setup, which increases the security risk. As a probabilistic structure, the Cuckoo filter has a false positive, which is inferior to the PCs. Furthermore, as indicated in [39], the Cuckoo filter can be well designed to reduce the false positive rate.

The Merkle tree can also be used for membership checks, which is called a vector commitment scheme and is the underlying technique of SPV, as described in Section 2.1. The SPV scheme provides the membership proof for a leaf node, which is called the Merkle branch. If a node can provide the Merkle branch related to a leaf, it must store the hashes of all leaves, which means all hashes of leaves should be maintained locally, conflicting with our purpose of storage optimization. Compared with the PCs and the Merkle tree, the Cuckoo filter has a more compact commitment and shorter check time, which is more suitable for blockchain applications.

3. Design of the Cuckoo-Store Engine

In this section, the system model of the Cuckoo-Store engine is first introduced in Section 3.1. The processes of ledger encoding and decoding are introduced in Section 3.2 and Section 3.3, respectively. Additionally, the Cuckoo filter is introduced into the encoding and decoding process to address the practical issues in Section 3.4. Finally, a demonstration of the CS engine is given in Section 3.5.

3.1. System Model

As shown in Figure 3, the Cuckoo-Store engine is formed from four modules, and all modules cooperate to interact with the blockchain network. The RS module is the core of the CS engine, which implements the functions of RS encoding and decoding by Encoder and Decoder, respectively. The data verification module can be used to validate the received data during the decoding process by the Block and Filter verifiers. The data storage module is divided into three parts: temp data storage, segments storage, and cuckoo filter storage. These parts are responsible for maintaining the temporary data, encoded segments, and the CF, respectively. The data request module is designed to interact with the other nodes, where the local reader is used to respond to the data request. The decode and remote reader are used to obtain segments and blocks from the neighbor node, respectively.

In practice, the CS engine could be implemented as the middleware, and any nodes with the CS engine could adopt a new storage model for the blockchain ledger. The basic idea is to encode the new block to multiple encoded segments with smaller sizes and distribute them among multiple nodes, which will decrease the storage burden for the ledger on each node and guarantee the recoverability of the ledger with the support of the RS Decoder. For clarity, the nodes can be divided into senders to issue the new transactions, validators to verify the transactions, and miners to propose new blocks. Each node can have multiple roles simultaneously. In the blockchain with the CS engine, the senders and validators can issue and validate the transactions as usual, and the process of block proposal and synchronization can be modified as follows:

1.

Block proposal. The miner packs the transactions into a new block. Based on the packed transactions, with RS Encoder’s support, the miner encodes the block bodies into multiple encoded segments and inserts these segments into a Cuckoo filter as step 3. Then, the hash of the CF is inserted into the new header to guarantee the integrity and consistency of the encoding results with other fields (parent hash, Merkle root, etc.). Then, the miner participates in the consensus process (such as PoW), and the recognized new block will be broadcast to the network;

2.

Block validation and synchronization. The validators who receive the new block stop the local PoW and validate the new block. Different from the original blockchain, in addition to normal checks (such as nonce and Merkle root), the nodes should repeat the processes of encoding and insertion to the CF to validate the integrity of the encoding results, with the support of the RS Encoder and block verifier as step 1 and step 2. The valid block will be appended to the tip of the local blockchain. During the process, each node can adaptively select a subset from the encoded segments based on the storage resources as step 3. It should be noted that the CF will also be maintained locally.

Furthermore, the distributed ledger can be recovered with the support of the RS Decoder. The process can be described as follows:

1.

Segment collection. With the support of the data request module, the node can collect enough encoded segments from its neighbor nodes. During the process, the integrity of the received segments is checked against the CF, and the forged segments are abandoned, with the nodes returning them removed from the neighbor list as step 4. The process is repeated until the number of received valid segments reaches the upper limit for decoding;

2.

Decoding and validation. With the support of the RS Decoder, the entire block body can be recovered as step 5. The correctness of the recovered block can be checked against the Merkle root in the header and the CF as step 6.

3.2. Data Encoding

For a specific encoding pattern, RS(n, p), a group containing p transactions is the basic unit to be encoded. The encoding result for the block can be obtained after processing all groups. For simplicity, for the block with T transactions ($TX=t{x}_1,t{x}_2,t{x}_3,\cdots ,t{x}_T$), assuming all transactions are L bytes, all T transactions can be divided into g groups exactly ($T=p·g$).

Based on the description in Section 2.2, each transaction, tx, can be represented as a byte array to be processed by Encoder as $tx=d_1,d_2,\dots ,d_L$. Then, a group of p transactions can be represented as matrix A with p rows and L columns as follows:

$$A=\left[\begin{array}{cccc}{d}_1^1& d_2^1& \cdots & d_L^1\\ d_1^2& d_2^2& \cdots & d_L^2\\ ⋮& ⋮& \ddots & ⋮\\ d_1^p& d_2^p& \cdots & d_L^p\end{array}\right],$$

(11)

where $d_j^i$ indicates the j-th byte for the i-th transaction in the block. Then, based on Equation (2), the $A_{enc}^1$ can be obtained by decoding A as follows:

$$A_{enc}^1=\left[\begin{array}{c}A\hfill \\ Q\hfill \end{array}\right],$$

(12)

where $A_{enc}^1$ is called an encoded result and Q is the redundant matrix with q rows and L columns as follows:

$$Q=\left[\begin{array}{cccc}{c}_1^1& c_2^1& \cdots & c_L^1\\ c_1^2& c_2^2& \cdots & c_L^2\\ ⋮& ⋮& \ddots & ⋮\\ c_1^q& c_2^q& \cdots & c_L^q\end{array}\right],$$

(13)

where $c_j^i$ indicates the j-th byte for the i-th redundant segment and $q=n-p$. After performing the same operations above for all g groups, the encoding results, $B_{enc}$, for the block can be obtained as follows:

$$B_{enc}={\left[A_{enc}^1,A_{enc}^2,\dots ,A_{enc}^g\right]}^T.$$

(14)

Each row element of $B_{enc}$ represents a piece of encoded segment $S_j,(j=1,2,\dots ,n·g)$. As described in Section 3.1, during the block synchronization, each node can select l segments in each $A_{enc}^i,(i=1,2,\dots ,g)$, a total of $l·g$ segments are stored locally, and the j values of stored $S_j$ as metadata are added in the block body. The compression ratio can be obtained through the stored $l·g$ segments of a subset of $B_{enc}$ and metadata, which will be analyzed in detail in Section 4.1.

$$S_j=B_{enc}\left[j:\right],(j=1,2,\dots ,n·g).$$

(15)

It should be noted that, in practice, the transactions packed in a block have different sizes, and the number of transactions (T) cannot always be divided exactly by the selected p. To address these issues, the transactions should be regularized by padding zeros before being encoded by RS Encoder. Assuming the longest transaction has L bytes, the transaction regularization process is as follows:

1.

Padding zeros to all transactions to make their length reach L bytes;

2.

Constructing $\lceil T/p\rceil ·p-T$ transactions with L zero bytes and adding them to the transaction list.

Then, the ledger can be processed in the same way as the above descriptions, and $\lceil T/p\rceil ·p$ transaction segments and $\lceil T/p\rceil ·q$ redundant segments are obtained.

3.3. Block Recovery by Decoding

In the blockchain with the CS engine, the distributed ledger can be recovered with the support of the RS Decoder, which guarantees the traceability of the blockchain. As described in Section 2.3, due to the redundant matrix, the original message can be decoded without reliance on all data segments, and RS code can detect and correct errors. In this part, we assume each encoded segment is maintained on multiple nodes, and the received segments are all correct, where the decoding process can be performed successfully as long as the number of the received segments ($S_{rec}$) exceeds the lower limit (denoted by $m\ge \lceil n-t\rceil$). When n different j values of $S_j$ of a group are collected, the collection ends. After all T transactions are recovered, the original block is decoded successfully. The processes of data collection and decoding are described as follows:

1.

Collecting segments. The node collects data segments $S_j$ from its neighbor nodes with the support of the data request module. Specifically, the node initiates a request to collect data segment $S_j$ with the support of the Decode Reader submodule, and after receiving it, the neighbor nodes send the stored segments and metadata through the Local Reader submodule. Typically, Ethereum nodes discover their neighbors and maintain connections with the K-bucket and Kademlia algorithms. In particular, the K-bucket can be modified to maintain the information about the data segments to improve response efficiency. Therefore, like the encoding process, a group of $n-t$ segments is the minimum unit for data collection. The data request finishes when the node has received n different segments;

2.

Decoding segments. Based on the received m segments $S_{rec}$ and constructed $n-m$ segments with L zero bytes, matrix $A_{rec}^1$ is constructed and the first group of p segments can be decoded according to Equation (3). Decoding matrix $A_{dec}^1$ can be obtained by decoding $A_{rec}^1$, which is called a decoded result, and each row of $A_{dec}^1$ can be considered as a decoded segment.

Similarly, the other groups could be decoded as $A_{dec}^2\sim A_{dec}^g$, and the decoded results can be represented as a matrix with $p\times g$ rows and L columns as follows:

$$B_{dec}={\left[A_{dec}^1,A_{dec}^2,\dots ,A_{dec}^g\right]}^T.$$

(16)

Then, the original block body can be obtained by removing all redundant rows and reconstructing all T transactions by byte. The correctness of the recovered block can be checked by computing the Merkle root based on the transactions by decoding and comparing the result with that maintained in the header.

3.4. Commitment Generation for Encoding Results

As a permissionless blockchain network, some malicious nodes exist, which could return the forged segments to interrupt the ledger recovery process and even corrupt the data integrity of the blockchain. Even if a block is recovered by multiple RS decodings under the untrusted network, this is based on repeated data requests and results in huge communication and computational overheads when decoding fails. According to the principles of the blockchain, all data should be committed to the header. However, in the encoding process described above, there is no commitment to the encoding results to guarantee data integrity. Thus, in this part, the Cuckoo filter is constructed to commit the encoding segments before broadcasting the new block. Then, during the ledger recovery process, the CF can detect the forged segments quickly based on the membership checks, which improves the recovery efficiency.

During the block synchronization process, the CF can be constructed as follows:

1.

Setting parameters. We calculate the number of elements to be stored as $N_x=\lceil T/p\rceil ·n$, the false positive rate as FPP, which is usually less than 1%, the number of hash functions as k = 2, and the load factor ($\gamma$) as 95%. According to the parameters, we calculate the length of the fingerprint (fl) and the number of buckets ($N_{buk}$), and then calculate the size of the filter ($S_{CF}$) in accordance with Equation (8). We initialize the CF with all bits set to 0;

2.

Inserting the encoded segments. The hash value $h_1\left(S_j\right)=hash(S_j+r+i)$ is based on Equation (9). Then, $h_1\left(S_j\right)$ is taken as modulo $N_b$ to obtain $b_1$, which determines the bucket where $S_j$ will be inserted. The low fl bits of $h_1\left(S_j\right)$ are stored as the fingerprint in bucket $b_1$. When bucket $b_1$ is full, the fingerprint that was the first element $S_j^{\prime }$ inserted into the bucket is kicked out and the new fingerprint is stored in bucket $b_1$, while the kicked fingerprint of element $S_j^{\prime }$ is stored in another bucket, $b_2$, where $b_2$ is obtained by taking the hash value $h_2\left(S_j^{\prime }\right)=h_1\left(S_j^{\prime }\right)\oplus hash\left(fingerprint\left(h_1\left(S_j^{\prime }\right)\right)\right)$ based on Equation (10) modulo $N_b$, and the low fl bits of $h_2\left(S_j^{\prime }\right)$ are stored as fingerprints in bucket $b_2$;

3.

Writing hash in the header. The hash fingerprints from each bucket in the CF are retrieved. When the entries in buckets are initial values, the initial values are treated as fingerprints. All fingerprints are then concatenated together, hashed, and the resulting hash value represents the CF hash, which is written into the block header.

During the block recovery process, the CF can be used in two cases:

1.

First Check. Before decoding, the CF can detect the received segments based on the membership checks by generated commitment; therefore, a reliable matrix for decoding is constructed quickly;

2.

Second Check. After $\lceil T/p\rceil$ groups of receiving segments for a block are decoded, transactions based on the decoded segments will be organized as a Merkle tree to obtain the root, which will be compared with the Merkle root maintained in the header to determine the correctness of the decoded result. If there are discrepancies between two roots, the CF, which is stored locally, will carry out the second check based on the membership checks to identify the groups where the error segments are located. The segments of these groups will be re-collected, decoded, and then verified repeatedly until the computed Merkle root matches the one maintained in the header.

To further enhance the block recovery efficiency under the untrusted network, it is necessary to modify the requirements for ending segment collection given in Section 3.3. Specifically, we are supposed to ensure the reception of n different j values of $S_j$ of a group, which are able to pass the First Check mechanism.

3.5. A Demonstration of the CS Engine

In this subsection, a demonstration of the CS engine will be shown, which includes regularization, the encoding of a group of transactions, verification, and the decoding of a group of received segments.

As an example shown in Figure 4, in the RS(6,4) code method, a group of four transactions ($t{x}_1\sim t{x}_4$) is transformed into a group of four transaction segments ($S_1\sim S_4$) through regularization. Considering the first set of column data (${[d_1^1,d_1^2,d_1^3,d_1^4]}^T$) from four transaction segments, RS encoding is performed, resulting in encoded column data (${[d_1^1,d_1^2,d_1^3,d_1^4,c_1^1,c_1^2]}^T$), which includes redundant data (${[c_1^1,c_1^2]}^T$). After this group of transactions is complete for RS encoding, we obtain six encoded segments, including four transaction segments ($S_1\sim S_4$) and two redundant segments ($S_5\sim S_6$).

In Figure 5, six received data segments of a group contain a forged segment at location $j=2$, and membership checks are performed according to the generated commitment by CF. The segment at location $j=2$ is detected as a forged segment by checking commitment and constructing a segment with L zero bytes to replace the forged segment. After that, the group of segments is decoded.

3.6. The Features of CS Engine

Based on the design scheme, the features of the blockchain applying the CS engine are drawn as follows:

In the proposed CS engine, all nodes perform independently, which ensures the decentralization and robustness of the system, which means there is no specific difference between the full and light nodes. The storage volume for the ledger on each node is reduced by encoding all transactions into multiple segments and distributing them to multiple nodes. When synchronizing the new block into the local blockchain, each node adaptively selects a subset of segments to be stored locally according to its storage resources. A more precise algorithm (such as the incentive mechanism introduced in FileCoin [1,48]) will help to obtain better storage efficiency, which is outside of the scope of this paper.

The CS engine is consensus-algorithm-independent and slightly modifies the original block structure and workflow, lowering the deployment difficulty. Specifically, (1) an additional 256-bit memory is added to the head of the blockchain to store the hash value of CF, and (2) during the process of block synchronization, the encoding of transactions and the construction of CF are added.

The CS engine can reduce the storage burden on each node without damage to blockchain functions such as traceability. After collecting all necessary segments from neighbor nodes, the original transactions can be recovered by RS decoding, which ensures the traceability of the blockchain. With the support of the Cuckoo filters, the forged responses can be detected efficiently by the membership check in O(1) complexity.

4. Theoretical Analysis

The compression ratio and the recovery availability are analyzed in Section 4.1 and Section 4.2, respectively. Then, the communication and computational overhead is analyzed in Section 4.3. For clarity, the symbols used in this paper and their meanings are listed in

Table 2. Main parameters and meaning of CS.

Symbol	Meaning	Symbol	Meaning
$fl$	length of fingerprint	T	average number of transactions
$\gamma$	the load factor of CF	$\overline{S_H}$	average size of the block headers
$P_F$	the probability of cheating CF	$S_{CF}$	average size of CF
${\beta }_S$	system compression ratio	$S_{CB}$	average compressed blocks size
${\beta }_L$	local compression ratio	$S_{OB}$	average original blocks size
${\beta }_l$	${\beta }_L$ under different l	$S_{sy}$	storage overhead of the system
N	total number of nodes	$L_B$	metadata size for segment
n	number of segments in a group	l	number of selected segments
p	number of txs in a group	$\alpha$	proportion of malicious nodes
q	number of redundant segments in a group	$p_l$	proportion of nodes storing l segments
$\overline{L}$	average size of the regularized txs	${\alpha }^{\prime }$	proportion of forged segments for decoding

.

4.1. Compression Ratio

The benefits introduced by the CS engine to the blockchain storage can be measured in two aspects: local compression ratio, ${\beta }_L$, and system compression ratio, ${\beta }_S$. The local compression ratio indicates the space saved by the CS engine on each node, which is defined as

$${\beta }_L=1-\frac{S_{\mathrm{CB}}}{S_{\mathrm{OB}}}=1-\frac{\left(\overline{L}+L_B\right)·l·⌈\frac{T}{p}⌉+S_{\mathrm{CF}}+\overline{S_H}}{\overline{L}·T+\overline{S_H}},$$

(17)

where $S_{CB}$ and $S_{OB}$ are the average sizes of the compressed and original blocks for a node to store, respectively, and $l(1\le l\le n)$ is the number of selected segments to be maintained locally. T is the average number of transactions packed into the block, and $\overline{S_H}$ is the average size of the block headers. $\overline{L}$ is the average size of the regularized transactions (also the average size of the data segments obtained by RS encoding as described in Section 3.2), and $L_B$ is the size of the metadata required to describe the index of each encoding segment. Considering that there are thousands of transactions (T is very large) and $\overline{S_H}$, and $L_B$ is much smaller than $\overline{L}$, the local compression ratio can be approximated as

$${\beta }_L\approx 1-\frac{l}{p}-\frac{fl}{\overline{L}·\gamma }·\frac{p+q}{p},$$

(18)

where fl and $\gamma$ are the length of the fingerprint and load factor of the Cuckoo filter. Furthermore, since $\overline{L}$ is far larger than fl, Equation (17) can be simplified as

$${\beta }_L\approx 1-\frac{l}{p}.$$

(19)

From the above formula, the local compression ratio is determined by l and the RS pattern, specifically p.

For a group of encoded segments containing n encoded segments, it is assumed that nodes with a proportion of $p_l$ (l = 1, 2, …, n) in the network store l encoded segments, where ${\sum }_{l=1}^n{p}_l=1$. Therefore, the storage overhead of the system can be expressed as

$$S_{sy}=N·⌈\frac{T}{p}⌉·\left(fl·n+\sum _{l=1}^n\left(l·p_l\right)\right),$$

(20)

where the total number of nodes is N. The system compression ratio can be expressed as

$${\beta }_S=1-\frac{S_{Sy}}{N·S_{OB}}=1-\frac{fl·n+{\sum }_{l=1}^n\left(l·p_l\right)}{p}.$$

(21)

Assuming that $p_l=2^{n-l}·p_n$, which means the number of nodes storing $l+1$ encoded segments is reduced by half compared to the number of nodes storing l encoded segments, the system compression ratio can be expressed as

$${\beta }_S=1-\frac{fl·n+{\sum }_{l=1}^n\left(l·2^{n-l}/\left(2^n-1\right)\right)}{p}.$$

(22)

Specifically, when p = 20, ${\beta }_s$ of the CS engine-based scheme is approximately 90%. For the plain blockchain network, when the proportion of full nodes to total nodes is greater than 10%, the scheme proposed in this paper has more advantages in storage.

From the above formula, the system compression ratio is mainly determined by the number of segments maintained locally on each node, which is affected by the proportions of the nodes with limited and ample resources in the network. Considering that most nodes act as light nodes with limited resources in blockchain applications, especially for the blockchain-based IoT scenario, the system storage efficiency could be improved greatly by applying the CS engine.

4.2. Availability of Block Recovery

The Cuckoo filter is constructed to guarantee the data integrity of the encoding segments during the RS encoding process, which can provide error detection via membership checks during the ledger recovery process. The received segments are first checked by the CF, and only valid ones can be used in the RS decoding process. Correspondingly, the nodes providing the forged segments will be removed from the peer list maintained locally. The availability of the ledger recovery for the Byzantine network environment is analyzed in this section.

For simplicity and clarity, we make some assumptions of the Byzantine network: (1) all encoding segments are maintained by multiple nodes; (2) the proportion of the malicious nodes in the network is denoted as $\alpha$, and $\alpha$ does not exceed 0.5; (3) all malicious nodes perform fraudulent behaviors independently; and (4) the mistakes in the communication channel are out of the scope of the analysis. Furthermore, we assume that the malicious nodes can be divided into two types: the ones only modifying the segments at random ($N_S$) and the ones modifying the segments with the support of the ample computing resources ($N_C$). Therefore, the latter could cheat the CF by finding the hash conflicts. The proportions of the two types of malicious nodes within all nodes are denoted by $\mu$ and ($\alpha -\mu$), respectively.

During the ledger recovery process, all receiving segments are first checked by the CF. For the forged segments provided by $N_S$, there is a probability of cheating the CF, as follows:

$$P_F=\frac{1}{2^{fl}·N_{buk}}=\frac{1}{2^{⌈log\left(2·C_{buk}/FPP\right)⌉}·N_{buk}}.$$

(23)

For the forged segments provided by $N_C$, all of them can cheat the CF because they are the colliding pre-images. Furthermore, the proportion of the colliding pre-images provided by $N_S$ in all n segments passing the CF checks can be estimated as

$${\alpha }^{\prime }=\frac{\mu ·P_F+(\alpha -\mu )}{1-\alpha +\left(\mu ·P_F+(\alpha -\mu )\right)}.$$

(24)

After the first check, for receiving n ($n=p+q$) segments of a group, the probability that c correct segments exist can be estimated as

$$P(X=c)=C_n^i·{(1-{\alpha }^{\prime })}^c·{\left({\alpha }^{\prime }\right)}^{n-c},$$

(25)

Suppose $p:q=\xi$; then, $n:p=(1+\xi )/\xi$, and when n increases proportionally with p, $n:p$ is a fixed value. $P(X=c)$ can be considered as

$$P(X=c)=C_{⌈\frac{1+\tilde{\xi }}{\xi }·p⌉}^c·{\left(1-{\alpha }^{\prime }\right)}^c·{\left({\alpha }^{\prime }\right)}^{⌈\frac{1+\tilde{\xi }}{\xi }·p⌉-c}.$$

(26)

Therefore, the probability of successfully recovering a group ($A_g$) is as follows:

$$P_{RecG}=P(X\ge n-t)=\sum _{i=n-t}^{n}P(X=i),$$

(27)

where $n-t$ is the minimum number of segments required to successfully decode a group. When ${\alpha }^{\prime }\le 1/3$, on the one hand, $\xi \le 1$, $P_{RecG}$ increases towards 1 as p increases; on the other hand, when $\xi$ takes a larger value, $P_{RecG}$ decreases in steps as p increases.

The probability of decoding a group correctly complies with the geometric distribution. Thus, the expected value when successfully recovering a group of encoded segments can be calculated as

$$E\left(RecG\right)=\frac{1}{P_{RecG}}.$$

(28)

Considering that the transactions in the block are divided into $\lceil T/p\rceil$ groups, $\lceil T/p\rceil ·E\left(RecG\right)$ attempts for all groups are required. The average number of decoding attempts required to recover the block is as follows:

$$N_{RecB}=\frac{\lceil T/p\rceil ·E\left(RecG\right)}{\lceil T/p\rceil }=E\left(RecG\right).$$

(29)

Thus, we can draw the following conclusion: when ${\alpha }^{\prime }\le 1/3$, on the one hand, $\xi \le 1$, $N_{RecB}$ decreases towards 1 as p increases; on the other hand, $\xi$ takes a larger value, $N_{RecB}$ increases in steps as p increases.

4.3. Implementation Overheads

The CS engine decreases the storage burden on each node with extra communication and computation overheads. The communication overhead is introduced mainly by the data segment collection during the ledger recovery process. The computation overhead is introduced mainly by the RS encoding and decoding processes.

Based on Section 3.4, the communication overhead can be estimated as

$$O_{Comm}=⌈\frac{T}{p}⌉·n·\overline{L}.$$

(30)

As p increases and n increases proportionally with p, the communication overhead for block recovery barely changes. When a node joins the blockchain system or when a historical transaction needs to be viewed, only a small fraction of nodes may be involved in ledger recovery. The communication overhead for the requesting node may be significant, but for the entire network it does not significantly increase the overall system communication overhead.

The computation overhead can be divided into two parts: (1) encoding and calculating the CF for storage and (2) decoding to recover the block. According to the RS(n,p) model, the average computation overhead of encoding a group of p transactions is denoted as EnCom. Therefore, the computation overhead of encoding a block is $[T/p]·EnCom$. In addition, $[T/p]·2nhash$ operations are required to insert $[T/p]·n$ encoded segments into the CF. Thus, the computational overhead for encoding is as follows:

$$O_{Comp}^{En}=⌈\frac{T}{p}⌉·(EnCom+2nhash).$$

(31)

The computational overhead of decoding the received segments of a group based on the RS(n,p) model is denoted by DeCom. The average overhead of decoding computation required to successfully recover a block is denoted by $N_{RecB}·⌈T/p⌉·EnCom$. In addition, each decoding operation requires the computation of 2n hash operations to verify whether the decoded segments exist in the CF.

$$O_{Comp}^{De}=N_{RecB}·⌈\frac{T}{p}⌉·(EnCom+2nhash).$$

(32)

As p increases and n increases proportionally with p, the EnCom and DeCom are proportional to plogp; thus, $O_{Comp}^{En}$ increases continuously. Furthermore, $N_{RecB}$ will increase or approach 1, leading to an increase in $O_{Comp}^{De}$. Therefore, as p increases, the computational overhead will continue to increase.

5. Experimental Evaluation

The simulation platform is established and the compression ratio is evaluated in Section 5.1. Then, the availability of the ledger recovery is evaluated in Section 5.2. Finally, the proposed scheme is compared with the existing scheme in terms of the compression ratio and block recovery delay for ledger recovery in Section 5.3.

5.1. Evaluation of Compression Ratio

An Ethereum-like private blockchain system is formed using a PC with Windows 10 OS, two Ubuntu virtual machines, and two CentOS virtual machines. The PC has an Intel (R) Core(TM) i7-7700HQ CPU @2.8 GHz and 8 GB memory. Each virtual machine is assigned 2 GB memory and 20 GB disk space. The blockchain protocols are implemented with Python 3.7, where only the core functions (such as consensus process, transaction processing, etc.) are realized. Furthermore, all parameters in the private blockchain are referenced from the typical values in Ethereum, and all blocks to be processed are the blocks (from 16,530,547 to 16,537,406) fetched from the real-world Ethereum. These blocks have an average size of 83,126 bytes, with an average of 155.15 transactions per block.

We consider a fixed value of p:q (typically, p:q = 1:2, 2:1, or 1:1 [18]), when we study the relationship between properties and p. The core function of the RS modular is implemented with the support of the ISA-L library.

Furthermore, the CF size can be adjusted dynamically according to p:q and FPP. The error correction ability of RS decoding is related to the number of redundant segments and the q value. The larger the q value, the stronger the error correction ability of RS decoding. When $p:q=\xi$, the larger q is, the smaller the required CF false positive probability (FPP), so we assume $FP{P}^q={10}^{-20}$, which reduces CF storage overhead and ensures block recovery efficiency with an increase in p. Otherwise, we set the load factor, $\gamma$, to 95% and the number of hash functions, k, to 2 for the CF according to [40]. In addition, the proportions of malicious nodes that modify the segments at random in all malicious nodes is set to 0.5, i.e., $\mu =0.5\alpha$.

It should be noted that the conclusion of the simulation is almost the same as that on the PC. According to Equation (17), five nodes in the private blockchain system have the same performance in the compression ratio, which means the system compression ratio can be estimated on a single node. Furthermore, under different parameters, the five nodes in the private chain system have the same trend in encoding and decoding speed, which means that the trend of block recovery performance can be tested on a single node. Therefore, for simplicity, the subsequent analysis is based on the simulation results from the PC with Windows 10 OS.

The CF sizes for different p:q are shown in Figure 6, where the sizes for p:q = 2:1, 1:1, and 1:2 are represented as black, red, and blue lines. As we can see from Figure 6, two conclusions can be obtained. First, the CF size decreases for a larger p. This is because a larger p results in a smaller fingerprint length based on the assumption of $FP{P}^q={10}^{-20}$. Second, for a specific p, the CF size increases for a larger p:q because more redundant segments need to be inserted.

Different from the CF size, the size of the segments maintained locally on each node is determined by p and the segment number (l). Based on the selected blocks, when p:q = 1:1, the local compression ratio for l = 1, 2 is represented by black and blue lines, as shown in Figure 7. It can be concluded that the local compression ratio decreases for a larger l. When l = 1, the local compression ratio for p:q = 2:1, 1:1, and 1:2 is represented by red, blue, and green lines, respectively. It can be concluded that the larger the p:q, the smaller the ${\beta }_L$, and the larger the l, the smaller the ${\beta }_L$, which coincides with Equation (19).

As indicated in Figure 7, the simulation results match the theoretical results very well. Generally speaking, as p increases, the compression rate gradually tends toward 1. When p = 20, the compression ratio for l = 1 can exceed 94%. Considering that most nodes in the blockchain network are light nodes with limited resources, they will select few encoded segments to be stored locally, as simulated. Therefore, the system storage efficiency can be improved greatly.

5.2. Efficiency of Block Recovery

In this section, we introduce the average number of decoding attempts for block recovery and the average decoding time of block recovery.

When $\alpha$ = 1/3, the average number of decoding attempts ($N_{RecB}$) for recovering blocks in our scheme is shown in the following Figure 8, where the $N_{RecB}$ for p:q = 2:1, 1:1, and 1:2 is represented as black, red, and blue lines, respectively. As p increases, $N_{RecB}$ becomes smaller, gradually tending to 1 when p:q = 1:2, and $N_{RecB}$ becomes larger with the number of steps when p:q = 2:1, which is consistent with our analysis in Section 4.2. Since n-t may take the same integers in different p of Equation (25), when p:q = 2:1, $P_{RecG}$ decreases with the number of steps; thus, $N_{RecB}$ increases with the number of steps.

In this paragraph, we discuss the average decoding time delay of the block recovery process, $t_B$, under different p:q, where the $t_B$ for p:q = 2:1, 1:1, and 1:2 is represented as black, red, and blue lines, respectively, as shown in Figure 9. When the proportion of malicious nodes $\alpha$ = 1/3, on the one hand, the curve corresponding to p:q = 2:1 shows that $t_B$ increases significantly with the increase in p, because more attempts are required overall and the decoding becomes more complex. On the other hand, when p:q = 1:1 and p:q= 1:2, as p increases, the $N_{RecB}$ tends to 1, and the increase in $t_B$ is mainly due to the increasing complexity of the decoding process, causing $t_B$ to increase slowly, as shown in Figure 9.

Figure 10 illustrates that when p:q = 1:1, the $t_B$ increases as p increases with different proportions of malicious nodes, $\alpha$, where the $t_B$ for $\alpha$ = 1/2, 2/5, and 1/3 is represented as black, red, and blue lines, respectively. It can be concluded that the larger the $\alpha$, the greater the value of $N_{RecB}$ required to recover the block, and $t_B$ increases significantly with an increase in p. When p = 20 and $\alpha$ = 1/2, the maximum $t_B$ is approximately 1.74 ms, which is still a small value compared to the block time. Therefore, we can claim that the blockchain system based on the CS engine can achieve excellent block recovery efficiency under attacks with different proportions of malicious nodes.

Through the above analysis, due to a block recovery in the order of milliseconds or even less, the blockchain system using the CS engine is friendly to the synchronization of ledgers after new nodes join the blockchain or the recovery of a historical block by nodes.

5.3. Comparison between the Proposed Scheme and the Existing Scheme

The performance in terms of the compression ratio between the proposed and ECLS scheme is compared in Figure 11, where the two schemes under the condition of p:q = 1:1 are represented by black and red lines, respectively. As indicated in Figure 11, the compression ratios in the proposed and the ECLS scheme increase for a larger p, as they have the same RS encoding pattern. Furthermore, for a specific p, the ECLS scheme has a higher compression ratio because there is no storage overhead for the Cuckoo filter and meta data.

The performance in terms of the average decoding time delay of the block recovery process between the proposed and ECLS scheme is compared in Figure 12, where the two schemes under the condition of $\alpha$ = 1/3 and p:q = 1:1 are represented by black and red lines, respectively. On the one hand, since the ECLS scheme makes no guarantee regarding the data integrity of the encoded segments, more forged segments can participate in the decoding process. On the other hand, the ECLS scheme has to verify the correctness of the Merkle root if there is a decoded result of a group that does not match the encoded result of the corresponding group, causing $⌈t/p⌉$ groups of segments to require re-collecting and decoding. The smaller the p, the more the groups, and the greater the probability of error, which results in a larger time delay, as shown by the red line in Figure 12.

However, based on commitment verification, the proposed scheme can effectively reduce the participation of forged segments in the process of decoding and quickly identify the erroneous groups after the verification of the Merkle root fails. Therefore, compared with the ECLS scheme, our scheme requires fewer decoding attempts. Thus, regardless of the changes in p, there is an obvious advantage of our scheme in terms of $t_B$ when compared against the ECLS scheme, as shown in Figure 12. Even if the gap in $t_B$ between the ECLS scheme and our scheme gradually narrows, there is still an improvement of more than 200% in block recovery efficiency when p = 20.

6. Conclusions

This paper proposes a distributed ledger storage solution, the CS engine, based on CF and RS code techniques. The CS engine stores transaction segments and redundant segments corresponding to transactions through data regularization and RS encoding technology. By using the verification ability of CF and the error-correcting ability of RS decoding, the CS engine greatly improves the efficiency of block data recovery under malicious node attacks. This approach reduces the data storage pressure while facilitating the relatively fast query of a specific transaction by recovering the whole block, ensuring the core characteristics of blockchain, including traceability and decentralization. Additionally, in this paper, the feasibility of fast block data recovery and a strong ability to resist malicious node attacks of CS are proven through theoretical analysis and experimental verification. It should be noted that the security of the CS engine is guaranteed under the assumption of permissionless blockchains. There are stricter assumptions, such as those proposed in [49], where the security of the CF should be studied in future work.