Towards Effective Guidance of Smart Contract Fuzz Testing Based on Static Analysis

Abstract

In smart contract fuzz testing, it is crucial to consider the inter-dependencies between the contract functions. To effectively test the business logic of a contract, its functions must be invoked in a meaningful order. In this paper, we propose techniques that utilize static analysis on Ethereum bytecode to tackle this challenge. When compared with the current state-of-the-art, our approach takes Solidity compiler’s variable packing optimization into account and allows more precise analysis of the data-flows between functions. In addition, we devise a novel test case initialization algorithm for fuzz testing, which minimizes the redundancy in the generated seed set. Our algorithm reduces test cases that share similar function call patterns and leads to more effective testing of the contract code during the fuzz testing. Experimental results show that the proposed techniques improve the effectiveness of smart contract fuzz testing for vulnerability detection. Specifically, our techniques enabled the fuzz testing tool to trigger the target bugs in the benchmark 3.0 times faster on average.

1. Introduction

Smart contracts are gaining growing interest from academia and industry as they can transparently enforce the terms of agreements written in code form. The blockchain system underlying the smart contract ecosystem enables these terms to be executed without relying on central authority. Smart contracts are particularly popular in financial applications that manage users’ digital assets, such as cryptocurrencies or digital tokens.

Unfortunately, software vulnerabilities in smart contracts have been a consistent problem, since they can pose serious threats to the contract users. For example, in 2016, an attacker exploited a re-entrancy vulnerability in the DAO [1] smart contract, stealing over USD 50 million. Similarly, in 2018, an integer overflow bug found in certain ERC20 tokens allowed unauthorized attackers to generate an arbitrary number of tokens, which led to a temporary shutdown of the market [2].

Due to such serious impacts, there has been extensive research on the detection of smart contract vulnerabilities. Fuzz testing [3,4] (or fuzzing), which is a popular software testing technique, is also widely used to identify vulnerabilities in smart contracts. In fuzzing, a test case is randomly generated or mutated and then provided as an input to the program under test. Despite its simplicity, fuzzing has proven its effectiveness in bug detection across various software domains [5,6,7,8], including smart contracts [9,10,11,12,13].

Testing smart contracts presents a unique challenge that rises from the inter-dependencies between contract functions. Often, the functions of a contract must be called in an appropriate sequence in order to execute meaningful business logic. In other words, certain functions can be successfully executed only when other relevant functions have been already called. If such proper context is not provided, the function may exit without executing any important logic, leading to incomplete testing of the code.

A recent study on smart contract fuzzing revealed that static data-flow analysis on the bytecode can lead to more effective testing of the contract, even in the absence of the source code [14]. In this work, the data dependencies between the functions are identified with static analysis [15,16] on the Ethereum smart contract bytecode. Then, this information is used to generate an initial test case set that is used for the actual fuzzing phase. These test cases are initialized to include promising function call orders that lead to meaningful exploration of the contract logic. Note that by performing analysis and testing at bytecode level, this work enables the security testing of closed-source smart contracts.

However, according to our observation, previous work on the integration of static analysis with smart contract fuzzing has two problems. First, the test case initialization algorithm of the current state-of-the-art [14] tends to generate similar test cases that share common function call patterns. Such redundancy in the initial test case (a.k.a. seed) set can result in uneven testing of the contract code. Second, our findings reveal that variable packing optimization [17] of smart contract compilers generates bytecode patterns that existing static analysis struggles to handle correctly. This causes the analyzer to produce imprecise data-flow information, which misguides the fuzzing process.

In this paper, we propose two techniques to improve the effectiveness of smart contract fuzzing guided by static data-flow analysis. First, we devise a novel test case initialization algorithm to generate a compact set of test cases. Our algorithm minimizes the redundancy in the test cases while retaining all promising function call sequences needed for effective testing. Next, we propose a fine-grained static taint analysis [18,19] for smart contract bytecode that can precisely trace the variable accesses and data-flows of the contract functions.

Experimental results demonstrate that the proposed techniques can significantly improve the effectiveness of smart contract fuzzing, in terms of bug detection capability. The fuzzer equipped with our techniques could trigger the known target bugs in our benchmark 3.0 times faster than the baseline, on average. Moreover, our implementation outperforms four other smart contract testing tools that are widely used for evaluation in previous work. Specifically, our fuzzer could detect 1.1, 4.2, and 6.8 times more bugs than SmarTest, sFuzz, and Mythril respectively.

To summarize, our work has the following contribution.

• We propose an algorithm to generate a compact initial seed corpus for smart contract fuzzing while retaining meaningful function call orderings in the seeds.
• We propose a static taint analysis for smart contract bytecode that can precisely identify the data-flows between functions, even in the presence of variable packing optimization.
• We empirically show that our techniques can significantly improve the bug-finding capability of smart contract fuzzing. For open science, we make our implementation publicly available at https://doi.org/10.5281/zenodo.14738106.

2. Background

2.1. Ethereum Smart Contract

A smart contract is a programmed contract that consists of code (functions) and data (states). Developers can write a smart contract and deploy it on the Ethereum blockchain. The deployed contract will then operate based on the business logic written in its functions. The contract usually manages its user’s digital assets such as ether, which is the primary currency in Ethereum. The status of digital assets, such as the balance of each user, is recorded as the data of the contract. The integrity of the data is guaranteed by the nature of blockchain.

Ethereum smart contracts are mostly written in Solidity, which is a programming language specifically designed for developing smart contracts. Source code containing business logic is first compiled into machine-readable bytecode. This bytecode can be deployed and executed in Ethereum Virtual Machine (abbreviated as EVM), which is the runtime environment for Ethereum blockchain. Then, users can interact with the deployed contract by calling a function defined in the contract. Such invocation is called a transaction, and this can be thought of as an instance of a function execution. Therefore, a transaction contains concrete information for a function call, such as the argument values and the amount of ether to send to the contract.

In Solidity, a smart contract can use state variables to store its data. These variables are similar to global variables in C language, as their values are maintained even after the execution of a function. From the perspectives of users, sending a transaction to the contract can update the value of state variables. At the implementation level, state variables are stored in a persistent data space called storage, and each contract can use this key-value storage to record its status.

Due to the stateful nature of smart contracts, the functions of a contract often have dependencies between each other. That is, the invocation of one function may only be meaningful after another function has been called. For example, consider a smart contract with (1) a function for user registration, and (2) another function for modifying the information of a registered user. If the second function is called before any user is registered, the transaction will simply return without running any meaningful logic. Note that such dependencies can be captured in the data-flow through the contract’s state variables. In this example, the first function will update a state variable that stores the information of the registered users, while the second function reads that variable.

2.2. Smart Contract Security

Smart contract code can have various kinds of software bugs, which can lead to the permanent loss of digital assets. Inter overflow and re-entrancy are famous types of vulnerabilities that have caused serious accidents in the Ethereum platform [1,2]. In addition, previous studies [9,20,21] have identified various types of vulnerabilities that can exist in smart contracts and discussed their security impacts.

In this paper, we focus particularly on integer overflows and underflows, since arithmetic operations in smart contracts often decide the amount of ether to transfer, which is the most crucial information in smart contract ecosystems. A single overflow or underflow in these computations may cause a devastating loss of assets to users. Hereafter, we use integer bugs as terminology that refers to both integer overflows and underflows.

As a solution to check and prevent integer bugs at runtime, SafeMath [22] library was proposed, offering safe APIs for arithmetic operations. Starting from Solidity 0.8.0, it has been integrated into the language and the compiler automatically translates arithmetic operations to use this library. Nevertheless, older versions of the Solidity compiler are still in use due to compatibility issues with existing projects. Moreover, there are numerous smart contracts that have been already deployed and running on the Ethereum blockchain. Therefore, the detection of integer bugs is still an important research problem in smart contract security.

3. Related Work

3.1. Smart Contract Analysis and Testing

Various static analyzers have been developed to find security bugs from smart contracts. SmartCheck [23] is a static analysis tool that translates Solidity source code into an XML-based intermediate representation and detects vulnerabilities with syntax-based pattern matching. Securify [24] utilizes Datalog [25] to derive semantic facts of the contract from the EVM bytecode. Then, it detects vulnerabilities by checking for pre-defined patterns encoded in domain-specific languages. Slither [26] statically analyzes the contract code to identify the variable accesses, protected functions, and data dependencies. This information is then used to detect bugs, suggest optimizations, or generate a comprehensive summary of the contract. VeriSmart [27] is an analyzer specialized in detecting integer bugs. VeriSmart automatically extracts invariants that hold between the state variables of a contract and uses them to increase the precision of the analysis. While static analysis can be useful for detecting software bugs, it does not provide a test case that can prove the existence of the found bug.

In this paper, we focus on software testing techniques for triggering smart contract vulnerabilities with concrete inputs. ContractFuzzer [9] is a pioneering black-box fuzzer for smart contracts, equipped with various test oracles to detect critical security vulnerabilities. sFuzz [11] adopts a genetic algorithm motivated by AFL in its fuzzing campaign and employs branch distance as fuzzing feedback to maximize branch coverage. Mythril [28] employed concolic testing [29,30,31] to find smart contract bugs, specifically focusing on vulnerabilities that can cause financial loss to users. Maian [20] first performs inter-procedural symbolic analysis to detect buggy behavior of the contract, and then verifies the result by running the contract with concrete inputs obtained from SMT solvers.

Recently, there has been work to apply important advances in traditional fuzz testing to the domain of smart contract fuzzing. For instance, the concept of directed grey-box fuzzing [32,33] has been successfully implemented in Vulseye [34] by considering the stateful property of smart contracts. Also, ItyFuzz [35] adopts the idea of snapshot-based optimization [36] to boost the efficiency of state exploration in smart contracts. Meanwhile, a series of papers have proposed testing techniques that specialize in detecting certain kind of bugs. SliSE [37] was proposed as an effective solution for detecting re-entrancy vulnerabilities. Gritti et al. [38] formalized a novel instance of confused deputy vulnerability in smart contracts and proposed a system to detect those bugs. This research is orthogonal to our study and can be integrated to achieve a synergy in smart contract testing. For instance, our implementation can also benefit from snapshot-based optimization to increase the fuzzing throughput. Similarly, while our technique is agnostic to the kind of target bugs, it can be be extended to support a wider spectrum of bugs.

3.2. Generation of Transaction Sequence

In smart contract testing, a test case (test input) can be thought of as a sequence of transactions. As explained in Section 2.1, smart contract functions often have dependencies on each other. Therefore, deciding the function call order of a transaction sequence is an important problem in test case generation for smart contracts. Most of the early smart contract fuzzing tools [9,11,20] rely on random mutation strategies when deciding the order of transaction sequences.

Recent studies on smart contract testing have proposed techniques that maximize the bug-finding capability by carefully deciding the function call orders. Smartian [14] statically analyzes the EVM bytecode of the smart contract under test and identifies data-flows between functions. When initializing the test cases for fuzzing, Smartian uses this information to include meaningful function call subsequences in each test case. Subsequently, the authors of MuFuzz [39] pointed out that data-flow analysis can be utilized not only for seed initialization, but also for sequence-aware mutation of the test cases. Harvey [40] is a grey-box fuzzer that adopts heuristics to generate transaction sequences. It first tests a single function while forcefully mutating the value of state variables. If the path coverage of that function is increased by such mutation, other transactions are prepended to generate longer sequences. However, this approach may waste the time and resources for fuzzing if the contract contains state variables that cannot be updated by prepending transactions.

Meanwhile, approaches based on machine learning have also been employed to tackle the transaction sequence generation problem. RLF [41] is a vulnerability-guided fuzzer that incorporates reinforcement learning to generate vulnerable transaction sequences. It models the fuzzing process as a Markov Decision Process (MDP), where an agent selects actions to maximize the reward. In this context, an action represents the selection of a function to include and the reward quantifies the possibility that the generated transaction sequence can trigger a vulnerability. Similarly, ILF [10] models the process of fuzzing as an MDP. Unlike RLF, ILF focuses on maximizing code coverage by defining its reward as a coverage-oriented metric. In addition, it trains a neural network-based fuzzer using sequences generated through symbolic execution to further enhance the code coverage. SmarTest [42] is a symbolic executor that prioritizes execution paths using a pre-trained language model, which computes the likelihood of a transaction sequence to be vulnerable. It generates new transaction sequences by appending new transactions to the existing one, which are later pruned out based on the satisfiability of the path’s state conditions.

Transaction sequence generation techniques based on machine learning [10,42] are complementary to the approaches that rely on static analysis [14]. Static analysis directly reasons about the program semantics, whereas machine learning extracts and utilizes various features to make a decision based on the learned model. Although the learned model can make flexible and effective decisions for contracts similar to the training dataset, it may fail to guide the fuzzer effectively if the target contract contains previously unseen code patterns. While static analysis does not require any training dataset to analyze the given code, it often suffers from the innate imprecision issue. In this context, we discuss the limitations of the existing static analysis-guided fuzzing approach [14] in Section 4 and propose solutions in Section 5.

4. Guiding Smart Contract Fuzzing: Challenges

In this section, we introduce the basic ideas for guiding smart contract fuzzing with static analysis. At the same time, we discuss important limitations of Smartian [14], which is the state-of-the-art fuzzer that implements these ideas. Our findings reveal that the initial test case set generated by Smartian can contain inputs that disturb effective testing of the target contract. In Section 4.1 and Section 4.2, we discuss two main reasons for such inefficiency in test case initialization.

4.1. Redundancy of Test Cases

The key intuition of Smartian is to identify a data-flow (i.e., def-use chain [43]) between two functions and include that ordered pair in the initial test cases for fuzzing. To generate initial test cases, Smartian first starts with test cases that consist of only one function. Then, it appends another function to an existing test case if the added function can spawn a previously unseen data-flows between functions. This process is repeated until the extension of test cases does not lead to the gain of function-level data-flows. The resulting set is used as the seeds to start the fuzzing campaign with (following the convention in the literature, we will use the terms initial test case and seed interchangeably throughout this paper).

Unfortunately, this strategy can generate similar test cases that share the prefixes in the function call sequence. Consider the example code in Figure 1. Note that a state variable (e.g., stateA) defined by one function (e.g., f) is used by another function (e.g., g), resulting in an inter-procedural data-flow. Therefore, static analysis can identify the data-flows that can occur between the function pair <f, g> or <g, h_i> (where h_i=h1, h2, or h3). Based on this information, Smartian starts with a single function, [f], and extends this until three sequences, [f-g-h1], [f-g-h1], and [f-g-h3], are obtained as initial test cases for fuzzing (the arguments of functions are initialized with dummy values and mutated later during the actual fuzzing). While a single function call sequence, [f-g-h1-h2-h3], is sufficient to explore all the interesting behaviors of the example contract, Smartian generates similar test cases with a common prefix, [f-g].

Such redundancy of test cases becomes a serious problem when the size of contract code increases. For example, assume that the contract contains many more functions than just f, g, and h_i. If too many test cases are focusing on executing specific functions, it may lead to insufficient exploration of other functions, as the time budget for the fuzzing campaign is limited. In the worst case, such bias of the initial test cases may prevent the fuzzer from detecting a vulnerability in the contract.

4.2. Imprecision of Static Analysis

According to our observation, Smartian fails to precisely analyze certain behaviors of EVM bytecode, which in turn produces wrong data-flow information. As explained in Section 2.1, state variables of a smart contract are placed in key-value storage at runtime. In other words, the Solidity compiler generates EVM instructions that read a state variable from the storage (SLOAD) or write a state variable to the storage (SSTORE). To enable such accesses, each state variable is assigned a unique position in the storage.

For efficient use of storage space, the Solidity compiler often packs multiple state variables into a single slot of the storage [17]. The EVM storage is a mapping from the integer address to a 32-byte slot, which is large enough to hold multiple state variables. Figure 2 illustrates the layout of the state variables of the previous example in Figure 1. Two state variables (stateA and stateB) of the contract are packed into the same slot of the storage. To read stateB, EVM bytecode has to first load the 32-byte slot stored in storage address 0 and then extract the value of stateB with bit-masking operations.

Unfortunately, the static analysis of Smartian focuses on tracing the storage address used as the operand of SLOAD and SSTORE instructions. Therefore, it cannot distinguish the access to stateA and stateB, since both variables are packed in the same slot at storage address 0. As a result, the data-flow analysis output of Smartian will contain spurious def-use chains. For instance, Smartian will incorrectly analyze that f and h1 in Figure 1 are accessing the same storage variable, if they are packed in the same storage slot. This makes Smartian believe that there is a data-flow between the two functions, which is not true.

5. Methodology

This section introduces our approaches to address the challenges discussed in Section 4. In Section 5.1, we propose a novel algorithm to generate a compact initial test case set for smart contract fuzzing. In Section 5.2, we introduce our static taint analysis strategy for EVM bytecode containing variable packing.

5.1. Algorithm for Compact Seed Generation

Algorithm 1 presents our algorithm for deciding the function call orders of the seed corpus, the initial test case set for fuzzing. While the algorithm takes in the set of contract functions ($\mathcal{F}$) as an explicit argument, it also implicitly utilizes the result of static data-flow analysis. The outermost loop iterates through each function that defines state variables and initializes a function call sequence in Line 7. Then, this sequence is extended as long as a new data-flow is observed by appending one of the functions at the end of the current sequence (Line 11), where ⧺ denotes the concatenation of two sequences. Here, the gain of the data-flow is decided based on the function-level def-use chains obtained by static data-flow analysis. That is, DataFlowOf in the algorithm computes the set of def-use chains found in the provided function sequence. When the extension of the sequence does not increase the data-flow any more, it is added to the accumulative seed set $\mathcal{S}$. Also, note that in Line 5 we check whether the current function to work with (f) is already subsumed in one of the generated seeds, to avoid redundancy in the generated seeds.

When compared with the algorithm of Smartian [14], the current state-of-the-art fuzzer, our algorithm generates a smaller number of seeds, while the average sequence length of each seed is longer. This difference is attributed to the worklist algorithm used by Smartian. In its algorithm, a LIFO (last in, first out) worklist is maintained to search for meaningful function call sequences. If appending one function to an existing sequence spawns a previously unseen def-use chain, Smartian’s algorithm immediately adds that sequence to the worklist without trying to extend it further. The items in the worklist are popped and processed one by one, and the algorithm stops when the worklist becomes empty.

When Algorithm 1 is applied to the previous example in Figure 1, it will start with sequence [f] and extend it as much as possible. This results in a single function sequence, [f-g-h1-h2-h3], where the positions of $h_i$ are interchangeable. In contrast, the algorithm of Smartian generates three sequences, [f-g-h1], [f-g-h1], and [f-g-h3], as the seed set, where the three def-use chains between f and $h_i$ are contained in separate test cases. This is because each sequence, [f-g-$h_i$], is separately evaluated and added to the worklist.

Algorithm 1: Initialization of function call sequences in fuzzing seeds.

5.2. Taint Analysis to Handle Variable Packing

To precisely analyze the storage variable accesses of EVM bytecode, we devise a fine-grained taint analysis strategy that tracks the byte offsets of a value loaded from the EVM storage. Recall from Section 4.2 that the static analysis employed by Smartian focuses on identifying the storage address used in SLOAD instructions. If a value v is loaded by instruction $v=\mathtt{SLOAD}\left(a\right)$ (although EVM is a stack-based machine, the instruction is presented in three-address-code for readability), Smartian relies solely on the value of address a to determine which storage variable has been accessed. In contrast, our static analysis further traces how the loaded value (v) is used in the remaining part of the code. Specifically, we perform taint analysis to observe which byte offset of this value is extracted and subsequently propagated to other instructions in the bytecode. Our analysis follows the standard workflow of static data-flow analysis framework [15,43] described below:

Step 1.

Construct the control-flow graph of the provided contract code.

Step 2.

Collect data-flow facts at each program point by applying the abstract semantics for taint analysis. The abstract semantics of each EVM instruction capture how the storage variables are accessed, extracted, and propagated in the program.

Step 3.

Examine the taint sink instructions to identify the data-flows propagated from the taint sources. At this step, we can observe which byte offsets of a storage variable are actually used in the program.

Now, we present the details of abstract semantics and the definition of taint sources and sinks. Our static taint analysis implements the abstract semantics for EVM instructions that can be used to extract specific offsets of an integer value. For instance, a series of EVM instructions $x=\mathtt{AND}(v,\mathtt{0}\mathtt{xFFFFFFFF}\mathtt{0000});y=\mathtt{SHR}(x,\mathtt{16})$ extract the four consecutive bytes at offset 2–5 of the 32-byte word v. Our analyzer recognizes the semantics of such bitwise operations and identifies the byte offsets extracted. While the constant operands are directly written in the above example (e.g., 0xFFFFFFFF0000), actual EVM bytecode computes these values with a series of arithmetic instructions, such as multiplication and exponent operations. Therefore, we must also statically analyze the status of the EVM stack, which is used for arithmetic operations. To identify the constant integer values, we employ the standard flat integer domain [15,16]. While analyzing the operands of the AND instruction, for example, we check whether one of the two operands is a constant value consisting of consecutive 0xFF and 0x00 bytes. If these conditions are not satisfied, the static analyzer does not have to trace this computation result further, as this instruction is not intended to extract specific offsets of an integer loaded from the storage.

Once the analyzer identifies the bytes extracted from a slot of the storage, the extracted value is considered as the source of our taint analysis. Then, we perform a standard taint analysis that propagates the taint information through the control-flow graph of the program. In our analysis, EVM instructions related to ether transfer (e.g., CALL, SELFDESTRUCT) or conditional jumps (JUMPI) are treated as taint sinks. Only if a tainted value is used as an operand of these sink instructions, we record it as a use of storage variables. By deciding the list of sink instructions carefully, we can precisely identify meaningful data-flows of the contract. For instance, smart contracts often provide getter functions that simply return the value of storage variables to the caller (user), without performing any important logic. In Smartian, such functions are analyzed to form def-use chains, as their bytecode contains SLOAD instructions that access the corresponding storage address. In contrast, our taint analysis effectively filters out such data-flows by excluding the RETURN instruction from taint sinks. Then, the value loaded from the storage will not flow into any sink instruction and will not be recognized as a used variable.

5.3. Implementation

We implemented our techniques on top of the open-sourced Smartian [44]. This allowed us to reuse the state-of-the-art fuzzing module for smart contracts and focus on improving the initial seed generation phase. We added 397 lines of F# code to implement our seed generation algorithm and static taint analysis for precise data-flow tracking. We publicly open-source our implementation and relevant artifacts at https://doi.org/10.5281/zenodo.14738106.

6. Evaluation

In this section, we present the experimental result of our methodology to answer the following evaluation questions.

Q1.

How do our techniques affect the size of the initial seed set generated for fuzzing?

Q2.

Can our techniques improve the bug-finding effectiveness of smart contract fuzzing?

Q3.

Does our system outperform other smart contract testing tools?

6.1. Experimental Setup

Machine setting. We performed experiments on a server equipped with two Intel Xeon Gold 6426Y CPUs (2.5 GHz) and 512 GB RAM. We used Ubuntu 22.04 and Docker 25.0.0 to run the experiments in Docker containers. Each container was assigned with one logical CPU core and 6 GB of RAM, and we ran 60 containers in parallel.

Benchmark. We used the benchmark from VeriSmart [27] that consists of 57 realistic smart contracts. Each contract contains a known integer overflow (or underflow) vulnerability. Each vulnerability was exploited or reported in the wild and assigned a CVE (common vulnerability exposure) ID, which is a unique identifier for a vulnerability found in real-world software. We chose these contracts as our benchmark since these known vulnerabilities can be used as ground-truth data to evaluate the effectiveness of fuzzing in finding realistic bugs. For this reason, this benchmark has also been used in the evaluation of Smartian [14]. Also, the code size of this benchmark is significantly larger than that of other popular benchmarks, such as SmartBug [45]. Specifically, the average LoC (line of code) of our benchmark is 167, while the average LoC of SmartBug is only 51. Such code size poses non-trivial challenges to fuzzers and makes the benchmark suitable for evaluating the effectiveness of testing.

Tools for comparison. We used two fuzzers and two symbolic executors as our comparison target tools. We first compared our tool against Smartian (commit eca1ed1), since it is the baseline implementation that our system is built on top of. In addition, we included the smart contract fuzzer sFuzz and symbolic executor Mythril in comparison targets, as they are widely used in the evaluation of other papers [14,42]. Note that we excluded Manticore and ILF as these tools are not maintained any more and do not run on Ubuntu 22.04 at the time of writing. Instead, we included SmarTest (commit 36d191e) [42] in the experiment, as it is the state-of-the-art symbolic execution tool that addresses the transaction sequence generation problem with machine learning, as discussed in Section 3. Lastly, note that we could not include ILF and RLF in comparison targets since they do not support the detection of integer overflow bugs in our benchmark.

6.2. Impact on the Initial Seed Set

Recall that the techniques presented in Section 5 aim to reduce redundancy in the initial test cases generated for fuzzing. To evaluate the effectiveness of our techniques, we counted the number of seeds generated for each contract in our benchmark. Also, we measured the average sequence length of the seeds generated for each contract.

Table 1. Impact of our techniques on initial seeds generated.

Modes for Seed Generation	Num. of Generated Seeds			Average Length of Seeds
	Min	Max	Avg	Min	Max	Avg *
Baseline (original Smartian)	9	234	48.3	2.8	7.6	4.2
Ours (with SeedGen algorithm only)	1	9	3.5	4.5	64.0	15.3
Ours (with both SeedGen algorithm and taint analysis)	1	11	3.7	3.5	58.0	11.5

* Note that this column presents the average of average.

summarizes the status of seeds generated by different modes. The baseline implementation, Smartian, generated 48.3 seeds on average for 57 contracts in the benchmark. In contrast, our improved seed initialization algorithm (SeedGen in Section 5.1) generated only 3.5 seeds on average. At the same time, the average sequence length of the generated seeds increased from 4.2 to 15.3 in the improved algorithm. In other words, our algorithm generated fewer seeds while the sequence length of each seed tended to be longer, when compared with the baseline. Note that the reduction in the number of seeds generated is more drastic than the increase of the sequence length, indicating that our algorithm produces a more compact set of seeds overall.

The last row of the table shows that applying the static taint analysis for variable packing (Section 5.2) on top of the SeedGen algorithm reduced the average sequence length of seeds from 15.3 to 11.5. This is attributed to more precise data-flow information obtained in the static analysis phase. Specifically, the total number of def-use chains statically identified in all of the contracts decreased from 3080 to 2186. We manually investigated the 894 def-use chains that disappeared in our enhanced static analysis and confirmed that all of them were either (1) spurious data-flows that cannot actually occur at runtime, or (2) data-flows in getter functions that do not trigger any meaningful contract logic. Unlike Smartian, our taint analysis can exclude these def-use chains from the result of data-flow analysis. Provided with a smaller number of statically identified def-use chains, our seed initialization algorithm can generate shorter function call sequences.

Lastly, we discuss the overhead incurred by the proposed static taint analysis and seed initialization algorithm. The time consumed by our static taint analysis is proportional to the number of nodes in the control-flow graph, which is determined by the size and structure of the contract code. In our evaluation, the analysis took 120 ms on average and 221 ms in the worst case. Meanwhile, the worst-case time complexity of our SeedGen algorithm was $O\left(n^{2}m\right)$, where n is the number of functions in the contract and m is the number of def-use chains computed from the data-flow analysis phase. In practice, the algorithm requires less time in most of the cases, since the loops in Algorithm 1 can terminate before iterating through all the functions. In our dataset, the algorithm finished in 70 ms on average and 1626 ms in the worst case. Overall, we can conclude that the computational overhead incurred by our approach is acceptable.

To summarize, we can answer the first evaluation question and conclude that the techniques proposed in this paper can effectively reduce the redundancy in the seed set generated for fuzzing.

6.3. Impact on the Fuzzing Effectiveness

While our techniques can generate more compact seed sets for fuzzing, it is another question whether they eventually lead to more effective fuzzing of smart contracts. To answer this question, we ran fuzzing on each contract for two hours and measured the elapsed time until the labeled bug was triggered. This metric, known as time-to-exposure (or TTE), is widely used as the primary evaluation metric for fuzzing when the benchmark contains known bugs that serve as ground truths [32,33,46]. If one testing tool exhibits a shorter TTE than others, it indicates that this tool can detect the targeted bug more promptly and effectively.

We measured and compared the TTEs of three different implementations: (1) the baseline Smartian, (2) our fuzzer that uses SeedGen only, and (3) our fuzzer incorporating both SeedGen and static taint analysis. Note that the three fuzzers share the same implementation in the fuzzing phase and they only differ in the generation of the initial seed set for fuzzing. We repeatedly fuzzed each contract 40 times and computed the median of TTE over the 40 iterations, to mitigate the fluctuation that comes from the non-deterministic nature of fuzzing.

Table 2. Comparison of the median TTE (time-to-exposure) of each target bug in our benchmark.

CVE	Baseline (Smartian)	Ours (SeedGen Only)		Ours (SeedGen + Taint Analysis)
	TTE	TTE	Rate	TTE	Rate
2018-10299	6	5	83%	3	50%
2018-10376	261	413	158%	158	61%
2018-10468	217	98	45%	91	42%
2018-10706	20	42	210%	34	170%
2018-11239	12	24	200%	17	142%
2018-11411	22	19	86%	17	77%
2018-11429	50	37	74%	25	50%
2018-11446	1088	270	25%	292	27%
2018-11561	2	2	100%	2	100%
2018-11687	43	76	179%	50	118%
2018-12070	75	33	44%	34	45%
2018-12230	11	10	91%	8	73%
2018-13126	109	25	23%	32	29%
2018-13127	332	59	18%	56	17%
2018-13128	240	28	12%	29	12%
2018-13129	728	61	8%	63	9%
2018-13131	448	64	14%	60	13%
2018-13132	724	66	9%	60	8%
2018-13189	52	10	19%	11	21%
2018-13202	29	27	95%	26	91%
2018-13208	81	51	63%	61	75%
2018-13220	33	23	71%	22	68%
2018-13221	22	30	136%	29	132%
2018-13225	35	26	75%	22	64%
2018-13227	88	52	59%	47	54%
2018-13228	84	70	84%	53	63%
2018-13230	57	44	78%	35	62%
2018-13327	586	390	67%	447	76%
2018-13493	29	17	59%	15	52%
2018-13533	5	10	200%	8	160%
2018-13625	48	32	67%	24	50%
2018-13670	24	7	30%	6	26%
2018-13698	4	2	50%	2	50%
2018-13703	9	6	71%	6	71%
2018-13777	27	9	33%	7	26%
2018-13778	23	8	35%	7	30%
2018-13779	41	14	34%	12	29%
2018-13782	42	15	36%	9	21%
2018-13783	152	33	22%	30	20%
2018-13836	25	29	116%	27	108%
2018-14001	9	10	111%	11	122%
2018-14002	19	19	100%	17	89%
2018-14003	6	3	55%	3	55%
2018-14004	31	32	105%	27	89%
2018-14005	20	16	82%	12	62%
2018-14006	25	32	128%	29	116%
2018-14063	32	29	91%	27	84%
2018-14084	93	42	45%	40	43%
2018-14086	15	14	93%	12	80%
2018-14087	455	34	7%	34	7%
2018-14089	76	55	72%	43	57%
2018-14576	37	24	66%	34	93%
2018-17050	24	24	100%	20	83%
2018-18665	8	4	50%	3	38%
Total	6725	2563	38%	2240	33%

presents the TTEs of each fuzzer for the target bugs in our benchmark. Since every contract in our benchmark contains one known vulnerability (CVE), each row of the table can be considered as one target contract. Out of the 57 bugs in the benchmark, we excluded three bugs that none of the three modes could detect within the two-hour time limit. The table indicates that, while the baseline Smartian took 6725 s in total to trigger the 54 bugs listed, the SeedGen algorithm greatly reduced the sum of TTEs to 2563 s. In addition, employing static taint analysis for precise data-flow analysis further reduced the total TTE to 2240 s. This means that applying both of the proposed techniques allows the smart contract fuzzer to detect the target bugs in our benchmark 3.0 times faster. The improvement in the TTE is more prominent for non-trivial bugs that take a relatively longer time to detect. For example, in CVE-2018-11446, which took the longest time for Smartian to trigger, our proposed techniques reduced the TTE by more than 3.7 times.

Based on the experimental results provided, we can answer the second evaluation question that our proposed techniques can significantly improve the bug-finding effectiveness of smart contract fuzzing.

6.4. Comparison with Other Tools

Lastly, we compared the bug-finding effectiveness of our system against other testing tools for smart contracts. In particular, we compared our fuzzer with sFuzz, Mythril, and SmarTest. Note that comparison against the baseline Smartian has been already performed in Section 6.3. Again, we ran the tools on each contract for two hours and repeated the experiment 40 times.

Table 3. Number of bugs (CVEs) found by each tool.

Tools	Ours	SmarTest	sFuzz	Mythril
# of found bugs	54	50	13	8

compares the average number of bugs found at the end of the two-hour experiment. Our fuzzer successfully triggered 95% of the 57 bugs contained in the benchmark, which is 1.1, 4.2, and 6.8 times more bugs than SmarTest, sFuzz, and Mythril, respectively. In addition, our system could detect all the bugs found by sFuzz and Mythril. While our fuzzer missed two bugs that SmarTest could find, it detected six bugs that SmarTest was unable to trigger. The result implies that, although our fuzzing methodology is more effective than other tools overall, different tools may still complement each other to detect a more comprehensive set of bugs.

Figure 3 presents the number of bugs found over time, obtained by computing the average over the 40 iterations. The figure indicates that our system not only found more bugs than other tools, but it also triggered the bugs much more promptly than others. Our tool triggered more than 50 bugs after performing fuzzing for 10 min, whereas SmarTest found less than 40 bugs after running symbolic execution for the same amount of time.

From these experimental results, we can answer the last evaluation question that our tool equipped with the proposed techniques to improve smart contract fuzzing can significantly outperform other state-of-the-art testing tools for smart contracts.

7. Conclusions

In this paper, we proposed two methodologies to effectively guide smart contract fuzzing based on static analysis of EVM bytecode. We studied critical limitations in current state-of-the-art fuzzers for smart contracts and addressed the identified challenges systematically. First, we designed and implemented static taint analysis that can cope with the EVM bytecode that contains variable packing patterns. Our taint analysis can distinguish the variables extracted from the same slot of the EVM storage, and trace the exact use of each variable in the contract code. This allows us to compute data-flows between functions more precisely than the existing static analyzer. Second, we proposed a novel algorithm for fuzzing seed generation based on statically analyzed data-flow information. The proposed algorithm avoids generating similar seeds that contain the same function call sequences. By minimizing such a redundancy, we can generate a more compact set of fuzzing seeds that leads to more effective testing of the contract code. Our evaluation demonstrates that the proposed techniques significantly improve the effectiveness of smart contract fuzzing. By employing the two techniques, our smart contract fuzzer triggers the target bugs in our benchmark 3.0 times faster than the baseline. Moreover, our tool significantly outperforms existing testing tools for smart contracts, in terms of bug-finding capability.

In this study, we focused on improving the generation of initial seed corpus for smart contract fuzzing. We leave it as future work to employ the proposed static analysis on EVM bytecode for improving other components of smart contract fuzzing. For instance, optimizing the mutate operations of fuzzing with static data-flow analysis can be a promising direction for future work. In addition, we plan to extend our implementation to support the detection of other types of bugs that exist in smart contracts. We believe our methodology is general enough and applicable to enhancing the effectiveness of fuzzing for other bug types.