Ethereum Smart Contract Vulnerability Detection and Machine Learning-Driven Solutions: A Systematic Literature Review

Abstract

In recent years, emerging trends like smart contracts (SCs) and blockchain have promised to bolster data security. However, SCs deployed on Ethereum are vulnerable to malicious attacks. Adopting machine learning methods is proving to be a satisfactory alternative to conventional vulnerability detection techniques. Nevertheless, most current machine learning techniques depend on sufficient expert knowledge and solely focus on addressing well-known vulnerabilities. This paper puts forward a systematic literature review (SLR) of existing machine learning-based frameworks to address the problem of vulnerability detection. This SLR follows the PRISMA statement, involving a detailed review of 55 papers. In this context, we classify recently published algorithms under three different machine learning perspectives. We explore state-of-the-art machine learning-driven solutions that deal with the class imbalance issue and unknown vulnerabilities. We believe that algorithmic-level approaches have the potential to provide a clear edge over data-level methods in addressing the class imbalance issue. By emphasizing the importance of the positive class and correcting the bias towards the negative class, these approaches offer a unique advantage. This unique feature can improve the efficiency of machine learning-based solutions in identifying various vulnerabilities in SCs. We argue that the detection of unknown vulnerabilities suffers from the absence of a unique definition. Moreover, current frameworks for detecting unknown vulnerabilities are structured to tackle vulnerabilities that exist objectively.

1. Introduction

A growing and widespread interest in the research community has arisen due to the emergence of smart contracts (SCs). A trusted environment for SCs has been created by decentralized blockchain technology in recent years without the need for third-party intervention [1]. A smart contract is a digital agreement that operates on a blockchain network and is automatically executed once specific terms and conditions are met. In other words, SCs work similarly to If-Then statements in different computer programs, and in this way, real-world assets may be affected by changes in the smart contract [2,3]. Compared with traditional contracts, they have the following significant merits: (i) reducing transaction risks; (ii) reducing administrative and service costs; and (iii) improving the efficiency of business processes [4].

Despite the unique benefits provided by SC technology, it is in the early stages of development. Moreover, there are still many problems that need to be addressed [4]. The exploitation of vulnerabilities in SCs to target the contracts themselves has witnessed a moderate increase recently. In this context, the core question is: What is the reason for SCs being targeted by hackers? Liao et al. [1] answered the question as follows: (i) SCs are valuable but vulnerable; (ii) SCs are challenging to patch due to the immutability nature of the blockchain; and (iii) there is a lack of assessment standards for ensuring.

The classification of traditional vulnerability detection tools by researchers involves five core methods, namely, static analysis, symbolic execution, dynamic analysis, formal verification, and fuzzy testing [5,6,7]. It should be noted that symbolic execution and formal verification are typically considered part of static analysis, whereas fuzzy testing falls under dynamic analysis. However, conventional strategies heavily lean on vulnerability rules that are manually created, a method that is time-consuming and challenging to address in all possible cases [3,5,7,8,9]. Moreover, it is still a challenge to discover unknown vulnerabilities [10].

Machine learning models can provide the possibility of finding vulnerabilities in substantially less time while retaining reasonable accuracy. Moreover, adopting more analyzers in the machine learning model can significantly increase the probability of finding the intersected vulnerabilities. Furthermore, applying a machine learning model to predict the security vulnerabilities in a SC can lead to a substantial enhancement in efficiency when compared with conventional vulnerability detection tools [11,12].

To date, many systematic reviews have assessed traditional vulnerability detection tools, applications of SCs, and future research opportunities [1,2,13,14]. To the best of our knowledge, we identify a research gap: a lack of a systematic review that covers machine learning-based solutions to cope with vulnerability detection problems in SCs. Our primary focus was developing a systematic literature review (SLR) to bridge this gap. Moreover, we observed that the class imbalance issue has been neglected by a significant majority of machine learning-driven solutions. The detection of unknown vulnerabilities in SCs has also been ignored in many proposed machine-learning-based frameworks. Motivated by these limitations and to resolve them, we arranged an SLR about vulnerability detection tools and machine learning-driven solutions.

In summary, the novel and significant contributions of this paper are enumerated as follows:

• The focus is on state-of-the-art machine learning-driven solutions that have been applied for vulnerability detection in SCs. In this respect, 55 journal articles and conference papers published between 2019 and 18 May 2024, will be explored. Moreover, we classify these methods under three different machine learning perspectives (i.e., classical models, deep learning, and ensemble models).
• We underline cutting-edge machine learning-driven frameworks that address the class imbalance issue and unknown vulnerabilities.
• We propose some contemporary open challenges to cover significant gaps when machine learning methods are applied for vulnerability detection.

The remainder of this paper is organized as follows: Preliminaries are introduced in Section 2. Section 3 briefly describes related work. In Section 4, the research methodology covers our paper selection procedure. The classification of the selected papers is represented in Section 5. Section 6 provides an analytical comparison. Section 7 and Section 8 ultimately lead to discussions and conclusions.

2. Preliminaries

This section introduces terminology, such as blockchain, Ethereum, SCs, vulnerability detection tools, machine learning techniques, and the class imbalance necessary for the paper.

2.1. Blockchain

Blockchain is a chain structure formed by the orderly concatenation of data blocks according to the generation time. The development of blockchain technology can be divided into three stages. It has passed the 1.0 stage and is in the process of moving to blockchain 2.0, that is, the smart contract stage [4,12,15,16].

2.2. Ethereum

In essence, Ethereum is a distributed (also decentralized) ledger that uses blockchain as its basic support technology, like Bitcoin. It supports the execution and invocation environments of SCs through a Turing-complete machine that is called the Ethereum Virtual Machine [5,7].

2.3. Smart Contracts

SCs were first proposed by Szabo (1997) [17], emphasizing that they facilitate the execution of contracts using protocols and user interfaces. When predetermined conditions are satisfied, it is triggered to execute and update the blockchain. Once a contract related to the transaction is executed, the transaction result cannot be changed or reversed. It includes a set of executable functions and state variables. There are deterministic and non-deterministic SCs. A deterministic SC does not need any information from an external party (outside the blockchain). A non-deterministic SC depends on oracles or data flows from an external party. SC technology is based on three things: the platform, the programming language, and the execution environment [4,18].

2.4. Vulnerability Detection Tools

SCs written in a high-level language such as Solidity are compiled into bytecodes and executed by the transaction drivers. Therefore, the security threats of SCs mainly come from potential vulnerabilities at the three levels of virtual machines, high-level languages, and blockchains. Table 1 summarizes the most discussed Ethereum SC vulnerabilities [6,14,19,20].

There are five distinct categories of vulnerability detection tools: (1) static analysis, (2) symbolic analysis, (3) dynamic analysis, (4) formal verification methods, and (5) fuzzy testing [7,8,18]. Table 2 presents a brief explanation of these analysis tools and their well-known samples.

2.5. Machine Learning Techniques

Machine learning is a subset of artificial intelligence and enables computers to learn from data and make decisions or predictions without being explicitly programmed to do so [11].

2.5.1. Classical Machine Learning Models

Classical machine learning techniques are classified into four types based on the nature of the learning regime and the data available, including (1) supervised, (2) semi-supervised, (3) unsupervised, and (4) reinforcement learning. Table 3 briefly describes common machine learning algorithms [11].

2.5.2. Deep Learning Models

Deep learning, a specialized subset of machine learning, is inspired by the information processing patterns found in the human brain and distinguished by its use of neural networks with three or more layers. Three types of deep neural networks include: (i) multi-layer perceptron (MLP), (ii) convolutional neural networks (CNN), and (iii) recurrent neural networks (RNN) [21,22].

2.5.3. Ensemble Learning

The concept driving this idea is that the combined intellectual capacity of two minds surpasses that of a single mind. Ensemble learning’s basic principle is to train multiple base learners as ensemble members and integrate their predictions into a single output that should have superior performance on average than any other ensemble member with uncorrelated error on the target datasets. Ensemble methods can improve resilience to noise. The most popular advanced ensemble methods are boosting, bagging, and stacking [23].

Table 1. Twelve types of security vulnerabilities in SCs.

Vulnerability	Description
Timestamp Dependency	When a contract employs block variables as a call condition to execute important operations or as a seed for generating random numbers [24,25].
Reentrancy	It allows attackers to invoke a specific contract function multiple times while the contract execution remains incomplete, enabling them to manipulate the contract’s behavior or pilfer funds [1,24,25,26].
Transaction Ordering Dependency	The attacker can manipulate the order in which transactions occur, thus potentially exploiting the logic of a contract [24,25].
tx.origin	A transaction origin attack is a phishing attack that can drain a contract of all funds [1,25,26].
Block-hashBlock Number	When the block has a block number, it is utilized to generate randomness by generating random numbers [24,25].
Gas Related Issues	The dependency of contract codes on data items that are either provided or manipulated by contract users is what causes these vulnerabilities [24,25].
Delegate Call	Using Ethereum virtual machine opcodes maliciously by the callee contract to bring up to date the state variables of the caller’s contract [24,26].
Arithmetic UnderflowOverflow	When the result of a mathematical operation exceeds the maximum value that the program can store, an arithmetic underflow is the opposite of an arithmetic overflow [1,24,26].
Unchecked Call	When an exception is thrown by the called contract and the return value is not validated, there is a risk of the execution continuing, which could result in unexpected behavior within the contract [1].
Self Destruct	The attacker can use the “self destruct” method to kill a contract because of poor authentication in the contract [1,24,26].
Access Control	Inadequate authorization or authentication enables the attacker to maliciously access the critical functions [1].
Denial of Service	Acting in a malicious manner while receiving a transaction. Enhancing the gas artificially to calculate a function. Exploiting access controls to gain unauthorized entry into private components of SCs [1].

Table 1. Twelve types of security vulnerabilities in SCs.

Vulnerability	Description
Timestamp Dependency	When a contract employs block variables as a call condition to execute important operations or as a seed for generating random numbers [24,25].
Reentrancy	It allows attackers to invoke a specific contract function multiple times while the contract execution remains incomplete, enabling them to manipulate the contract’s behavior or pilfer funds [1,24,25,26].
Transaction Ordering Dependency	The attacker can manipulate the order in which transactions occur, thus potentially exploiting the logic of a contract [24,25].
tx.origin	A transaction origin attack is a phishing attack that can drain a contract of all funds [1,25,26].
Block-hashBlock Number	When the block has a block number, it is utilized to generate randomness by generating random numbers [24,25].
Gas Related Issues	The dependency of contract codes on data items that are either provided or manipulated by contract users is what causes these vulnerabilities [24,25].
Delegate Call	Using Ethereum virtual machine opcodes maliciously by the callee contract to bring up to date the state variables of the caller’s contract [24,26].
Arithmetic UnderflowOverflow	When the result of a mathematical operation exceeds the maximum value that the program can store, an arithmetic underflow is the opposite of an arithmetic overflow [1,24,26].
Unchecked Call	When an exception is thrown by the called contract and the return value is not validated, there is a risk of the execution continuing, which could result in unexpected behavior within the contract [1].
Self Destruct	The attacker can use the “self destruct” method to kill a contract because of poor authentication in the contract [1,24,26].
Access Control	Inadequate authorization or authentication enables the attacker to maliciously access the critical functions [1].
Denial of Service	Acting in a malicious manner while receiving a transaction. Enhancing the gas artificially to calculate a function. Exploiting access controls to gain unauthorized entry into private components of SCs [1].

Table 2. Classification of vulnerability detection (VD) tools.

VD Tools	Description	Samples
Static	These tools use a compliance pattern to check the vulnerability of a contract [4,13].	Smartcheck, Securify, and Slither
Symbolic	These tools use symbolic execution to check for the transformation of the memory state [4,13].	Oyenty and Mythril
Dynamic	By employing a dynamic tool, the contract’s behavior is continuously monitored during execution to identify and mitigate any potential vulnerabilities or security breaches proactively [4,13].	Maian and ECFChecker
Formal Verification	These methods produce proofs based on an abstract “mathematical model” of a system [4,13].	F* Framework, Isabelle/HOL, and FEther
Fuzzy Testing	These methods run the SC on the blockchain and fuzz some input to the contract [4,13].	ContractFuzzer, Echidna, and ILF

Table 2. Classification of vulnerability detection (VD) tools.

VD Tools	Description	Samples
Static	These tools use a compliance pattern to check the vulnerability of a contract [4,13].	Smartcheck, Securify, and Slither
Symbolic	These tools use symbolic execution to check for the transformation of the memory state [4,13].	Oyenty and Mythril
Dynamic	By employing a dynamic tool, the contract’s behavior is continuously monitored during execution to identify and mitigate any potential vulnerabilities or security breaches proactively [4,13].	Maian and ECFChecker
Formal Verification	These methods produce proofs based on an abstract “mathematical model” of a system [4,13].	F* Framework, Isabelle/HOL, and FEther
Fuzzy Testing	These methods run the SC on the blockchain and fuzz some input to the contract [4,13].	ContractFuzzer, Echidna, and ILF

Table 3. Classical machine learning algorithms.

Learning Algorithm	Description
Supervised	It is a type of labeling learning technology that uses a labeled training dataset to construct a model representing the learned relationship between the input and output values.
Semi-supervised	It is a type of learning technology in which most of the training samples are unlabeled and a few are labeled.
Unsupervised	It is a machine learning algorithm that uses training datasets without labels.
Reinforcement	Training algorithms using a system of reward and punishment mechanisms.

Table 3. Classical machine learning algorithms.

Learning Algorithm	Description
Supervised	It is a type of labeling learning technology that uses a labeled training dataset to construct a model representing the learned relationship between the input and output values.
Semi-supervised	It is a type of learning technology in which most of the training samples are unlabeled and a few are labeled.
Unsupervised	It is a machine learning algorithm that uses training datasets without labels.
Reinforcement	Training algorithms using a system of reward and punishment mechanisms.

2.6. Class Imbalance

Classification is one of the core tasks in the field of machine learning. The class imbalance issue can make learning difficult since learning tends to be biased towards the majority class. That is to say, learners will overclassify the majority group when there is a class imbalance in training data due to its increased prior probability. As a result, instances belonging to the minority group are misclassified more often than those belonging to the majority group. Current solutions for class imbalance can be classified into three categories: data-level methods, algorithm-level methods, and hybrid approaches [27].

2.6.1. Data-Level Methods

Data-level methods for addressing class imbalances include over-sampling and under-sampling. These techniques modify the training distributions to decrease the level of imbalance or reduce noise, e.g., mislabeled samples or anomalies. In their simplest forms, random under-sampling discards random samples from the majority group, while random over-sampling duplicates random samples from the minority group [27].

2.6.2. Algorithm-Level Methods

Unlike data sampling methods, algorithmic methods for handling class imbalances do not alter the training data distribution. Instead, the learning or decision-making process is adjusted in a way that increases the importance of the positive class. Most commonly, algorithms are modified to take a class penalty or weight into consideration, or the decision threshold is shifted in a way that reduces bias towards the negative class [27].

2.6.3. Hybrid Approaches

Data-level and algorithm-level methods have been combined and applied to class imbalance problems. One strategy includes performing data sampling to reduce class noise and imbalance and then applying cost-sensitive learning or thresholding to further reduce the bias towards the majority group [27].

3. Related Work

This section reviews 14 surveys and SLRs published from 2019 to 2023 in blockchain and SCs, then discusses the advantages and limitations of each study. Table 4 details review papers related to SCs. Table 5 describes the method of their article selection. Table 6 refers to research questions, followed by recently published survey papers.

Table 4. Overview of related reviews and their coverage based on SCs.

Reference	Environment	Review Type	Year of Pub	Selection Process	Taxonomy	Year Covered
[28]	Security, performance, and applications	SLR	2019	clear 1	✓	2016–2018
[29]	Security	survey	2019	clear 1	✓	Until Deccember 2018
[30]	Applications	SLR	2019	clear 1	✓	2014–April 2018
[31]	Security	SLR	2020	semi clear 2	✓	2015–July 2019
[32]	Cyber security	SLR	2020	clear 1		April 2018
[33]	Development, test, and security	SLR	2021	clear 1	✓	2016–2020
[34]	Technical aspect, future directions	survey	2021	not clear 3		Until 2020
[4]	Difficulties and future trends	review	2022	not clear 3		Until 2022
[13]	Analysis tools	SLR	2022	semi clear 4	✓	2016–December 2021
[35]	Vulnerability detection and machine learning	survey	2022	not clear 3		Until 2021
[36]	Vulnerability detection tools within Web 3.0	survey	2023	not clear 3	✓	2011–2023
[14]	Security threats, collaborative defense	review	2023	not clear 3		2018–2023
[37]	Vulnerability detection	survey	2023	not clear 3		Until 2023
[38]	Platforms, applications, and challenges	review	2023	clear 1	✓	2015–May 2020

¹ The identification, exclusion, eligibility, and inclusion methodology are clear. ² The number of selected papers is not clear. ³ The identification, exclusion, eligibility, and inclusion methodology are not clear. ⁴ Keywords are not mentioned in the methodology.

Table 4. Overview of related reviews and their coverage based on SCs.

Reference	Environment	Review Type	Year of Pub	Selection Process	Taxonomy	Year Covered
[28]	Security, performance, and applications	SLR	2019	clear 1	✓	2016–2018
[29]	Security	survey	2019	clear 1	✓	Until Deccember 2018
[30]	Applications	SLR	2019	clear 1	✓	2014–April 2018
[31]	Security	SLR	2020	semi clear 2	✓	2015–July 2019
[32]	Cyber security	SLR	2020	clear 1		April 2018
[33]	Development, test, and security	SLR	2021	clear 1	✓	2016–2020
[34]	Technical aspect, future directions	survey	2021	not clear 3		Until 2020
[4]	Difficulties and future trends	review	2022	not clear 3		Until 2022
[13]	Analysis tools	SLR	2022	semi clear 4	✓	2016–December 2021
[35]	Vulnerability detection and machine learning	survey	2022	not clear 3		Until 2021
[36]	Vulnerability detection tools within Web 3.0	survey	2023	not clear 3	✓	2011–2023
[14]	Security threats, collaborative defense	review	2023	not clear 3		2018–2023
[37]	Vulnerability detection	survey	2023	not clear 3		Until 2023
[38]	Platforms, applications, and challenges	review	2023	clear 1	✓	2015–May 2020

¹ The identification, exclusion, eligibility, and inclusion methodology are clear. ² The number of selected papers is not clear. ³ The identification, exclusion, eligibility, and inclusion methodology are not clear. ⁴ Keywords are not mentioned in the methodology.

Table 5. The method of article selection in review papers.

Reference	Database								Keywords	# Papers
	Google Scholar	IEEE	ScienceDirect	Springer	Wiley	Taylor & Francis	ACM	Other
[28]		✓	✓	✓	✓		✓	✓	Smart contract, contract, chaincode	90
[29]		✓	✓	✓			✓	✓	Smart contract, security, verification	53
[30]								✓ 1	Blockchain, application	260
[31]	✓								Smart contract, Dapps, token, EVM, security	-
[32]	✓	✓	✓	✓			✓		Blockchain, block-chain, distributed ledger, security	30
[33]	✓	✓	✓				✓		Blockchain, smart contract, Ethereum	96
[13]		✓	✓	✓	✓				-	132
[38]		✓	✓	✓			✓		Smart contract, smart contract applications, smart contract tools, smart contract platforms, smart contract challenges, blockchain	183

¹ Scopus database.

Table 5. The method of article selection in review papers.

Reference	Database								Keywords	# Papers
	Google Scholar	IEEE	ScienceDirect	Springer	Wiley	Taylor & Francis	ACM	Other
[28]		✓	✓	✓	✓		✓	✓	Smart contract, contract, chaincode	90
[29]		✓	✓	✓			✓	✓	Smart contract, security, verification	53
[30]								✓ 1	Blockchain, application	260
[31]	✓								Smart contract, Dapps, token, EVM, security	-
[32]	✓	✓	✓	✓			✓		Blockchain, block-chain, distributed ledger, security	30
[33]	✓	✓	✓				✓		Blockchain, smart contract, Ethereum	96
[13]		✓	✓	✓	✓				-	132
[38]		✓	✓	✓			✓		Smart contract, smart contract applications, smart contract tools, smart contract platforms, smart contract challenges, blockchain	183

¹ Scopus database.

Table 6. Research questions followed by survey papers.

Reference	Research Questions
[28]	1: What are the security problems with SCs, and how can we address them? 2: What methods are applied to improve the performance of SCs?
[31]	1: How is the existing security state-of-play for SCs? 2: How do you design countermeasures to the identified security challenges? 3: When and where are these studies about SC security published? 4: What are potential future research challenges?
[32]	1: What are the latest blockchain applications focused on security? 2: How is blockchain used to improve cyber security? 3: What methods are available for blockchain solutions to manage security without requiring a cryptocurrency token?
[13]	1: What are the static analysis tools available for Ethereum blockchain SCs? 2: Which dynamic analysis tools are available for the Ethereum blockchain SC? 3: For Ethereum blockchain SCs, what kind of analysis approaches are employed by static/dynamic analysis tools? 4: What are the five most common vulnerabilities detected by analysis tools?
[38]	1: How can available platforms support SC development? 2: What are the presented decentralized applications, which mainly discuss SC design and implementation? 3: What are the challenges that relate to the application of SCs? 4: What are the solutions to the identified challenges?

Table 6. Research questions followed by survey papers.

Reference	Research Questions
[28]	1: What are the security problems with SCs, and how can we address them? 2: What methods are applied to improve the performance of SCs?
[31]	1: How is the existing security state-of-play for SCs? 2: How do you design countermeasures to the identified security challenges? 3: When and where are these studies about SC security published? 4: What are potential future research challenges?
[32]	1: What are the latest blockchain applications focused on security? 2: How is blockchain used to improve cyber security? 3: What methods are available for blockchain solutions to manage security without requiring a cryptocurrency token?
[13]	1: What are the static analysis tools available for Ethereum blockchain SCs? 2: Which dynamic analysis tools are available for the Ethereum blockchain SC? 3: For Ethereum blockchain SCs, what kind of analysis approaches are employed by static/dynamic analysis tools? 4: What are the five most common vulnerabilities detected by analysis tools?
[38]	1: How can available platforms support SC development? 2: What are the presented decentralized applications, which mainly discuss SC design and implementation? 3: What are the challenges that relate to the application of SCs? 4: What are the solutions to the identified challenges?

A systematic survey [28] reviews the security, performance, and applications of SCs. The method of selecting articles is explicitly mentioned in this study. However, the search domain for the reviewed articles is not clear. The scholars introduced a taxonomy that categorizes recent studies and developments regarding SCs based on three main categories: (i) security methods and tools, (ii) performance improvement approaches, and (iii) decentralized applications.

In another study, a survey [29] reviews the security verification of blockchain SCs. The method of article selection has not been well demonstrated in this research. Their article search is limited to December 2018. They proposed a taxonomy for the topic of security verification of blockchain SCs. The security assurance aspects are classified into three categories, including environment security, vulnerability scanning, and performance impacts. In contrast, the correctness verification aspect is classified into two categories, including programming correctness and formal verification.

Likewise, a systematic literature review [30] reviews blockchain-based applications, their current status, classification, and open issues. The method of article selection in their study has not been clearly indicated. Moreover, the search scope of their article is not clear. They presented a comprehensive classification of blockchain-enabled applications across diverse sectors such as supply chain, business, healthcare, IoT, privacy, and data management.

Furthermore, a survey [31] explores Ethereum SC security and future research. The method of article selection in their study has not been clearly revealed. Their article search domain spans from 2015 to 2019. They focused on how SCs can be maliciously exploited and targeted. Specifically, they concentrated on abnormal contracts, program vulnerabilities, and unsafe external data issues in SCs.

Another systematic literature review [32] is dedicated to utilizing blockchain for cyber security. The researchers demonstrated their article selection method. The scope of their article search is limited to April 2018. Their study presented a systematic analysis of the most frequently adopted blockchain security applications. Moreover, they highlighted opportunities for future research via three potential research agendas. However, the scholars do not introduce a taxonomy regarding cyber security.

In another study, a survey [33] reviews SC development, techniques, tools, and open challenges. The researchers effectively indicated their article selection method. The scope of their article searches spans from 2016 to 2020. They reviewed papers related to six specific topics: SC testing, SC code analysis, SC metrics, SC security, decentralized application (DApp) performance, and blockchain applications. Moreover, this paper identifies open challenges for each of the six topics above.

The survey [34] reviews the technical aspects and future research directions of blockchain-based SCs. The method of article selection is not described in their research. Their article search domain is not clear. They reviewed security attacks, vulnerabilities, and possible solutions. Moreover, they elaborated on future research topics, including SCs and game theory, SCs and artificial intelligence, and SCs in data science.

The review [4] summarizes the recent progress of SCs in blockchain. The method of article selection is not described in their research. Their article search domain is not clear. They investigated the difficulties faced by SCs and the corresponding solutions. Moreover, this review examines and judges the future challenges and development trends of SCs.

The systematic review [13] reviews the analysis tools associated with the Ethereum blockchain SC. They described their method of article selection. The scope of their article search covers the years from 2016 to December 2021. They investigated a detailed review of 86 analysis tools, covering all the analysis tools present in the literature or on the web, irrespective of their type and analysis approach.

The survey [35] reveals vulnerability detection using machine learning. The method of selecting articles is not explicitly mentioned in this study. Moreover, the search domain for the reviewed articles is not clear. They provided insights on the limitations and advancements of machine learning-driven solutions.

In another study, a survey [36] reviews the approaches for vulnerability detection in SCs within Web 3.0 applications. The method of article selection has not been demonstrated in this research. Moreover, their article search is not clear. They proposed a taxonomy of vulnerability detection approaches into four categories: formal verification, symbolic execution, fuzzing, and taint analysis.

The literature review [14] investigates the research progress on blockchain security threats and collaborative defense. The method of article selection in their study has not been indicated. Moreover, the search scope of their article is not clear. They discussed the challenges of blockchain security and future research directions, such as parallel detection and federated learning.

Furthermore, the survey [37] explores vulnerability detection and security enhancement. The method of article selection in their study has not been clearly revealed. Their article search domain is not clear. They provided a taxonomy of SC vulnerability analysis relevant to security.

Finally, the review [38] focuses on SC-based platforms, applications, and challenges. The researchers indicated their article selection method. The scope of their article search is not clear. They provided a table that shows the distribution of academic papers according to category. Moreover, they categorized large SC implementations by domain, including insurance, supply chain management, the Internet of Things, healthcare, multimedia, cloud computing, identity management, and record management.

Table 5 presents the databases, keywords, and the number of articles associated with the references mentioned in Table 4. The selection process for these references is either “clear” or “semi-clear”.

In summary, reviewing previous articles, the following weaknesses were found: (1) The survey articles did not include the newly proposed machine learning-drive solutions for vulnerability detection in SCs. (2) The problem of class imbalance and unknown vulnerabilities has been ignored by many research papers that rely on machine learning-driven solutions. Considering the cases mentioned above, this SLR is presented to solve the problems of previous research.

4. Methodology

Our systematic literature review on Ethereum smart contract vulnerability detection has been carried out following the guidelines of Moher et al. [39]. In this section, we will introduce how the examined papers have been selected, prepared for analysis, and an overview of the produced research (how the papers are distributed over time and publication types).

4.1. Research Questions (RQ)

By studying review articles and SLRs, challenges in different machine learning-driven vulnerability detection tools in the SC area remain; thus, several questions have been raised regarding machine learning-based approaches to vulnerability detection. The existing studies are entirely valid and relevant to the field in question. Therefore, three questions were prepared, which will be answered analytically and graphically in the following sections.

• RQ1: What machine learning-driven techniques are used in vulnerability detection tools?
• RQ2: What machine learning-driven vulnerability detection tools have addressed the class imbalance issue?
• RQ3: What machine learning-driven vulnerability detection tools have addressed unknown vulnerabilities?

4.2. Selection of Primary Studies

On 23 February 2024 the search was conducted, and it was later updated on May 18, 2024. The platforms used for the search were IEEE, ScienceDirect, Springer, Wiley, Taylor & Francis, and ACM. The search covers title, abstract, and keywords. To search for primary studies, we have identified the following keywords: “smart contract” AND “vulnerability” AND (“machine learning” OR “deep learning” OR “ensemble learning”) AND “vulnerability detection”.

For each paper found, the title was analyzed, followed by an abstract, an introduction, and conclusions to determine whether it pertained to the established inclusion criteria. Then we analyze the rest of each document to find significant contributions and open issues.

4.3. Inclusion and Exclusion Criteria

This SLR targets journal articles, international conference proceedings, and symposiums published between 2019 and 18 May 2024. Papers on topics not belonging to the Ethereum smart contract vulnerability detection and machine learning-driven solutions have been excluded. Duplicated articles and all other kinds of work like books, surveys, review articles, technical reports, or master theses were omitted. Moreover, any articles not written in English were excluded from the study.

4.4. Selection Results

Considering the keywords, the following results have been obtained, divided by platform:

• IEEE: 57 results.
• ScienceDirect: 12 results.
• Springer: 29 results.
• ACM: 7 results.
• Wiley: 3 results.
• Taylor & Francis: 3 results.

Subsequently, the duplicated studies were removed. Then the papers concerning the exclusion criteria were removed. Finally, 55 papers remained on which analysis was conducted. All the phases of the methodology for the inclusion and exclusion of research articles are depicted in Figure 1.

In addition, we plot the number of papers published per year to reveal temporal trends (see Figure 2).

5. Taxonomy of Machine Learning-Driven Solutions for Vulnerability Detection

In this section, cutting-edge algorithms are classified under three different machine learning perspectives (i.e., classical, deep learning, and ensemble models). Furthermore, this SLR explores state-of-the-art machine learning-driven solutions that address the class imbalance issue and unknown vulnerabilities. In addition, we investigate the development of contemporary solutions in line with standardized quality metrics.

5.1. Machine Learning Models

5.1.1. Classical Machine Learning Models

Momeni et al. [26] and Li et al. [40] propose classical machine learning frameworks to detect security vulnerabilities of SCs on the Ethereum platform. The K-Nearest Neighbor and Decision Tree are common classifiers among their models. Momeni et al. [26] utilize Random Forest and Support Vector Machine, while Li et al. [40] employ Linear Regression and Stochastic Gradient Descent in their method. The model developed by Momeni et al. [26] not only identifies security vulnerabilities but can also be applied to other languages and platforms. However, Li et al.’s [40] model detects unknown vulnerabilities based on the similarity principle. In contrast to the model proposed by Momeni et al. [26] that relies on manual feature extraction, the Eth2Vec [41] scheme utilizes a neural network to automatically learn features of vulnerable Ethereum Virtual Machine (EVM) bytecodes.

Xu et al. [42] and Yang et al. [43] propose vulnerability detection techniques for SCs, which extract SC features based on the abstract syntax tree (AST) and train the models to detect smart contract vulnerabilities. Xu et al. [42] use K-Nearest Neighbor and Stochastic Gradient Descent classifiers, while Yang et al.’s [43] model is developed based on Random Forest classifier. Xing et al. [21] propose Random Forest based on opcode features (RFBOOF) model. This model was developed based on classical and deep learning algorithms. In comparison with classical machine learning-based approaches, the method proposed by Eshghie et al. [44] (Dynamit) eliminates the need for any domain knowledge, code instrumentation, or exceptional execution environment.

5.1.2. Deep Learning Models

The methods proposed by Tang et al. [45], Jain and Tripathi [46], Shen and Li [47], Demir et al. [48], Han et al. [49], Zhu et al. [50], and Chen et al. [51] utilize a combination of machine learning algorithms to detect SC vulnerabilities. The Lightning Cat method [45] trains three deep learning models: Optimized-CodeBERT, Optimized-LSTM, and Optimized-CNN. This framework extracts problem code segments functions, which not only considers the key features of SC vulnerability code but also solves the length limitation problem of deep learning for processing long texts. The method proposed by Jain and Tripathi [46] focuses on a feature extraction mechanism based on a two-step hierarchical model using deep learning techniques: Bidirectional Gated Recurrent Unit (Bi-GRU) and Text-CNN. The framework proposed by Shen and Li [47] follows a parallel feature extraction strategy based on CNN-LSTM and RNN-GRU. Demir et al. [48] focus on detecting Reentrancy vulnerabilities using a Bidirectional LSTM (Bi-LSTM) model by employing assembly data to detect flaws. Han et al. [49] propose a model based on GNN and CNN to obtain the semantic and structural information of the source code, unlike methods that mostly view SC source code as natural language for processing. The GraBit method [50] is developed based on GraphCodeBERT and Bidirectional LSTM, which embeds both the source code and concise key data flow graphs extracted from the code. The SCVSN paradigm [51] utilizes the Siamese network and LSTM to complete the task of vulnerability detection. The Siamese network consists of two subnetworks that share the same parameters in a low-dimensional and easily separable feature space. The SCVSN scheme can identify the presence of vulnerabilities within a SC but cannot detect what kinds of vulnerabilities there are.

The models proposed by Zhang et al. [52], Hwang et al. [53], Zhou et al. [54], Mittal et al. [55], Feng et al. [56], Liang and Zhai [57], Zhou et al. [58], and Zeng et al. [59] are developed based on convolutional neural networks (CNNs). The Multi-Objective Detection Neural Network (MODNN) method [52] can validate twelve types of vulnerabilities, including ten recognized threats and more unknown types, without the need for predefined knowledge. CodeNet [53] includes two main steps: (1) the SC source code is compiled into bytecodes; bytecodes are transformed into an SC-based input image for CNN architectures while keeping the semantics and context of an SC; and (2) it analyzes the SC-based input images to detect vulnerable SCs. The Tree-based Machine Learning Vulnerability Detection (TMLVD) method [54] feeds the intermediate representations of SCs derived from AST into a tree-based CNN for building the prediction model. Like TMLVD, the technique introduced by Mittal et al. [55] and the SmartEmbet-TextCNN-TMP (SEET) framework [56] vectorize the Solidity code into AST, while SCGRU [57] transforms the standardized data into word vector representations of SCs with semantic information through the Word2Vec word embedding module. The CS-VDM scheme [58] automatically detects the vulnerabilities in the SC without expert knowledge and in very little time. Like the CS-VDM method, the SolGPT technique [59] focuses on runtime. Moreover, it boosts feature extraction capabilities while reducing reliance on labeled data. Nonetheless, this strategy may have constraints when it comes to the diversity of types and samples.

Graph Neural Networks (GNNs) are the central architecture of vulnerability detection models proposed by Cai et al. [60], Chen et al. [61], Liu et al. [62], Liu et al. [63], Liu et al. [29], Zhen et al. [64], Nguyen et al. [65], Xiong et al. [66], Cai et al. [60], Wang et al. [67], Wang et al. [68], Lin et al. [69], Wei et al. [70], and Wu et al. [71]. The method proposed by Cai et al. [60] constructs a graph representation for a SC function with syntactic and semantic features by combining AST, control flow graph (CFG), and program dependency graph (PDG). In contrast, Chen et al. [61] construct a novel semantic graph (SG) for each function and learn the SGs using graph convolutional networks (GCNs) with residual blocks and edge attention. The Multi-Relational Nested Contract Graph (MRNG) scheme [62] characterizes the rich syntactic and semantic information in the SC code, including the relationships between data and instructions while Combining the graph feature and the expert patterns (CGE) method [63] is a fully automated approach for SC vulnerability detection at the function level. The paradigm followed by Liu et al. [72] blocks risky transactions during the operation phase. In this method, the code violates the rules, the interrupt module is triggered, the code transaction is terminated, and an error report is issued. The Dual Attention Graph Neural Network (DA-GNN) scheme [64] combines attention mechanisms and GNN. This method starts by generating attributes of the SC from its bytecode, including the control flow graph and opcode sequence. It extracts attribute features from both parts and combines them as input to the dual attention module to obtain graph features for identifying vulnerabilities in SCs.

The MANDO-HGT framework [65] adapts heterogeneous graph transformers (HGTs) with customized meta relations for graph nodes and edges to learn their embeddings and train classifiers for detecting various vulnerabilities. In contrast, the FBB-VD scheme [66] uses a graph neural network to combine multiple code representations and detect vulnerabilities in SCs. The method proposed by Wang et al. [67] chooses the appropriate network depth and does not blindly increase the number of hidden layers. The GVD-net method [68] uses a distinct strategy in the preprocessing phase based on a weight matrix and generates the corresponding relationships of nodes based on a non-Euclidean graph. The Smart Contract Vulnerabilities by Fusing Semantic Features with Expert Features (SmartFuSE) model [69] conducts static analysis to extract vulnerability-specific expert patterns and joint graph structures at the function level. This model allows for the representation of the comprehensive program semantics of vulnerable code. Additionally, it utilizes a unique graph neural network with a hybrid attention pooling layer to emphasize critical vulnerability features. In contrast to other graph-based approaches, the technique suggested by Wei et al. [70] capitalizes on structural information and artificial rules. The Self-Attention Graph Pooling SAGDP framework [71] focuses more on essential features than unimportant features to capture the graph’s structural and feature information jointly.

LSTM is another deep learning method used by Ren et al. [73], Colin et al. [25], Xu et al. [74], Zhang et al. [52], Qian et al. [75], Zhou et al. [76], and Hu et al. [77] to detect vulnerabilities in SCs. The Blass framework [73] constructs program slices with complete semantic structure information (CPSs). It uses AST and a depth-first traversal algorithm to convert CPSs into code chains during CPS vectorization. The paradigm proposed by Colin et al. [25] not only standardizes the pre-processing method for SC training data but also introduces bugs to create a balanced dataset of flawed files across Solidity versions using AST. The HAM-BiLSTM model [74] uses a hierarchical attention mechanism. This framework takes the code segment and account information of an SC as input and divides the input samples into three levels of documents: word level, sentence level, and document level, whereas Zhang et al.’s method [52] first vectorizes the SC code, then inputs the vectorized data into the LSTM network to generate a model, and finally uses the model to detect defects. Wang et al. [78] propose a different vulnerability detection method based on LSTM that uses a code representation fusion strategy. The bidirectional LSTM with attention mechanism (BLSTM-ATT) method was developed by Qian et al. [75] as a distinct technique to detect Reentrancy vulnerabilities. This work stands out due to the implementation of a representation method that contributes to capturing essential semantic information and control flow dependencies. Although most of the methods reviewed in this paper are focused on the detection of Reentrancy vulnerabilities, the HuntFlow strategy [76] detects arithmetic vulnerabilities such as Integer Overflow vulnerabilities. The framework proposed by [77] stands out from other LSTM-based strategies due to its incorporation of two parsing methods: a combination of opcode sequences and AST.

The methods proposed by Vu et al. [79] and Zeng et al. [80] are focused on Bidirectional Encoder Representations from Transformers (BERT). SecBERT [79] is a pre-trained model to extract the implicit features and analyze them using the Multi-Layer Perceptron algorithm, whereas SCVulBERT [80] utilizes SCVulTokenizer for code tokenization and rich prior knowledge for training.

Furthermore, researchers also paid attention to other deep learning models that were found less frequently in the analyzed works. Narayana and Sathiyamurthy [81] develop a model based on Autoencoders. The Link-DC model [82] uses expert knowledge as the original data input to capture richer feature information. The Clear method [83] employs a contrastive learning (CL) model to capture the fine-grained correlation information among contracts. The S-BiGRU scheme [84] combines a Bidirectional Gated Recurrent Unit and SMOTE for smart contract vulnerability detection. This paradigm processes and parses Solidity source code into a series of tokens embedded into feature vectors using Word2Vec. These word vectors are fed into the model for training to obtain a model capable of detecting smart contract vulnerabilities for subsequent detection work. The method proposed by Qin et al. [85] focuses on the problem of poor detection caused by excessive noise. Other researchers have primarily overlooked this issue. The bytecode for smart contract vulnerability detection is given special attention in the cross-modality framework proposed by Qian et al. [86]. This scheme is developed based on a large-scale labeled benchmark dataset that consists of four different types of vulnerabilities.

5.1.3. Ensemble Learning Models

Wang et al. [24], JJ et al. [87], Ma et al. [88], and Huang et al. [89] develop ensemble learning models to detect vulnerabilities in SCs. ContractWard [24] is a model for effectively and efficiently detecting six types of vulnerabilities based on extracted static characteristics. It can be applied to detect vulnerabilities in SCs written in all high-level languages, such as Solidity, Serpent, and LLL. JJ et al.’s [87] model includes Bagging, AdaBoost, and Gradient Boost classifiers, while the HGAT scheme [88] uses functions based on AST and CFG. The CDRF [89] is a time-saving vulnerability detection method that processes the opcode fragments by word2vec and PCA to obtain one-dimensional binary features.

5.2. Class Imbalance Issue

This subsection reviews machine learning-driven vulnerability detection tools that have considered the class imbalance issue in their framework.

The Synthetic Minority Oversampling Technique (SMOTE) is applied by [24,46,84,87]. SMOTE is an oversampling technique, interpolating between the minority classes to generate extra ones [24]. ContractWard, proposed by Wang et al. [24], follows two sampling algorithms, namely, SMOTE and SMOTETomek, to balance the training datasets. That is to say, these methods are adopted to extend the number of minority classes to be similar to that of the majority classes. Hence, it can eliminate unnecessary samples while conducting the sampling process. Jain and Thripathi [46] are convinced that unless the level of imbalance is substantial, it will not have a significant impact on the performance of the model. In this context, they calculate the class imbalance ratio. Then their approach aligns with the policy outlined by Wang et al. [24], except for the substitution of SMOTETomek with TomeLinks. Finally, Jain and Thripathi [46] verify that the balanced dataset’s ratio of vulnerable to non-vulnerable contracts is approximately 1:1. In contrast to the approaches suggested in [24,46], Song et al. [84] and JJ et al. [87] exclusively employ the SMOTE model to avoid any biasing effect and hence possess a uniform sample distribution. It concentrates on the feature space to produce new examples by interpolating between positive samples that are close together.

Zhang et al. [52] and Cai et al. [60] adopt entirely distinct approaches from previous methodologies to address the class imbalance issue. Zhang et al. [52] utilize the focal loss algorithm to deal with the class imbalance problem. Their scheme (MODNN) uses k-fold cross-validation (KCV) to validate each randomly generated subset. Each sample of data for KCV can be used for testing and training to avoid overfitting and underfitting and to obtain more credible results. When the loss of MODNN is within the acceptable range, the stop condition control parameter ‘‘early_stop’’ value is reset to 1. Cai et al. [60] believe that a sample imbalance may threaten the validity of their approach. With the size of the dataset increasing, more factors need to be controlled in the experiment. In this context, they randomly shuffle the dataset in the same proportion and utilize a validation set to test the model’s generalization performance to alleviate this potential threat.

5.3. Unknown Vulnerabilities

The present machine learning-based vulnerability detection techniques mainly focus on known vulnerabilities, while unknown vulnerabilities in SCs have been rarely investigated. This subsection delves into examining recently published articles [40,52,65,71] that have tackled this particular issue.

Li et al. [40] propose a classical machine learning-based method for detecting unknown vulnerabilities in SCs using opcode sequences. It combines an n-gram model and a vector weight penalty mechanism to enhance the detection of unknown vulnerabilities. They consider unknown vulnerabilities to be those that already exist but cannot be discovered using existing vulnerability detection methods. Moreover, researchers emphasize that such vulnerabilities are objectively present, even if they have not been found. Their scheme analyzes the similarity of the behavior of unknown and known vulnerabilities from two aspects: (1) vulnerability and (2) attack.

The methods proposed by Zhang et al. [52], Nguyen et al. [65], and Zou et al. [71] utilize deep learning models to detect unknown vulnerabilities in SCs. MODNN [52] uses a CNN-based model to identify more unknown vulnerabilities without needing a specialist or predefined knowledge. This method learns vulnerabilities in unknown patterns from explicit features. By contrast, Nguyen et al. [65] and Zou et al. [71] use a GNN-driven strategy to deal with unknown vulnerabilities. The framework proposed by Nguyen et al. [65] can be re-trained for new types of vulnerabilities and detect vulnerabilities in large sets of contracts efficiently and accurately. In contrast, the SAGP approach [71] is equipped with the self-attention mechanism to detect vulnerabilities that may not be identifiable by expert knowledge, thereby minimizing the possibility of errors that can arise from manually established detection patterns. This method is implemented using a hierarchical pooling structure.

6. Comparison

An analytical discussion and comparison between the final articles are provided in Section 5. Section 4.1 deals with research questions. The present section answers the questions.

6.1. RQ1: What Machine Learning-Driven Techniques Are Used in Vulnerability Detection Tools?

Table 7. A taxonomy of vulnerability detection tools based on machine learning models.

Model Type	References
Supervised Models	[26,40,41,42,43,44]
Deep Learning Models	[45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,89,90]
Ensemble Models	[24,87,88]
Supervised and Deep Learning	[21]
All Models	[25]

illustrates a taxonomy of vulnerability detection tools based on machine learning models to develop methods for detecting vulnerabilities in SCs. Researchers in [21,25] use a hybrid method to build their framework. Moreover, supervised learning is the most found learning method among classical models, while unsupervised learning is neglected.

Table 8. The most found classifiers for detecting vulnerabilities in SCs.

Models	References
Classical Models
RF	[21,24,25,26,41,43,44,87,89]
SVM	[24,25,26,43,44,89]
KNN	[24,26,40,42,44]
DT	[26,40,44,87]
LR	[40,44]
SGD	[52]
NB	[44]
Deep Learning Models
ANN	[21,25,26,44,61,79,81]
CNN	[21,45,46,47,49,52,53,54,55,56,57,58,59]
GNN	[29,49,51,60,61,62,63,64,65,66,67,68,70,71,88]
Autoencoder	[81]
LSTM	[25,45,47,48,50,69,73,74,75,76,83,89,90]
Bi-GRU	[46]
BERT	[45,50,79]
Other	[51,82,83,84,85,86] 1−2−3−4−5−6
Ensemble Models
Bagging	[87]
GBoost	[87]
XGBoost	[24,25,87,89] 7
AdaBoost	[24,89]

¹ [82] DCN: deep forward propagation of a fully connected neural network in parallel. ² [83] Contrastive Learning (CL). ³ [51] Siamese network. ⁴ [84] Bidirectional Gated Recurrent Unit (BiGRU). ⁵ [85] ResNet50. ⁶ [86] Cross-modal mutual learning. ⁷ [89] AdaBoost and Light Gradient Boosting Machine (LightGBM).

depicts the most found classifiers for detecting vulnerabilities in SCs using machine learning-driven solutions.

Turning to Figure 3, the pie chart details the percentage of machine-learning-driven solutions for vulnerability detection in SCs. A significant majority of this chart is accounted for deep learning models, and the remaining 21% is used for classical models (14%) and ensemble models (7%). With respect to Figure 4, RF, GNN, and XGBoost are the most commonly used classifiers among classical, deep learning, and ensemble models, respectively.

6.2. What Machine Learning-Driven Vulnerability Detection Tools Have Addressed the Class Imbalance Issue or Unknown Vulnerabilitis?

RQ2 and RQ3 are addressed together in this subsection.

Table 9. More details on machine learning-based vulnerability detection (VD) tools and datasets.

Reference	Method Name	# VD	Unknown VD	Dataset	Balanced Dataset
[26]	-	46		Etherscan 1
[21]	RFBOOF	3		--
[24]	CotractWard	6		Etherscan 1	✓
[42]	-	8		SolidiFI 2, SmartBugs 3, SmartBugs-wilds 4
[52]	MODNN	12	✓	Etherscan 1, SolidiFI 2, SmartBugs-wilds 4, EDAUB 5, Xblock 6	✓
[53]	CodeNet	4		SolidiFI 2, SmartBugs 3
[54]	TMLVD	5		Etherscan 1
[60]	GNN	9		SolidiFI 2, Smartbugs-wilds 4, smartbugd-cuarted 7, HuanGai 8	✓
[61]	SG and EA-RGCN	4		SmartBugs 3
[62]	MRNG	3		SmartBugs 3
[81]	-	3		Known_Attacks 9
[73]	Blass	4		Data are confidential
[63]	CGE	3		Etherscan 1, VSC 10
[72]	-	4		Etherscan 1
[46]	-	13		Smartbugs-wilds 4	✓
[87]		6		Etherscan 1, SmartBugs 3, Blockchain 11, Positive 12	✓
[88]	HGAT	6		SmartBugs-wilds 4
[45]	Lightning Cat	4		SolidiFI 2
[64]	DA-GNN	3		*, **
[25]	-	7		SolidiFI 2, ***	✓
[40]	-	7	✓	Etherscan 1
[65]	MANDO-HGT	7	✓	SolidiFI 2, SmartBugs 3, SmartBugs-wilds 4
[43]	-	3		Etherscan 1
[55]	-	1		SmartBugs-wilds 4
[66]	FBB-VD	-		Etherscan 1, SolidiFI2, SmartBugs 3
[47]	-	1		Etherscan 1, SmartBugs 3
[74]	HAM-BiLSTM	1		Etherscan 1
[90]	-	4		Not mentioned
[79]	-	4		Etherscan 1
[48]	-	1		Etherscan 1
[82]	Link-DC	2		Etherscan 1
[56]	SETT	2		[91]
[80]	SCVulBERT	3		Smart-Contract-Dataset
[57]	SCGRU	6		SmartBugs 3
[83]	Clear	3		[86]
[78]	AFS	1		Github, Gitter chat room 13, Karl 14
[49]	-	4		Eth2Vec [41]
[50]	GraBit	1		Smartbugs-wilds 4
[67]	-	1		Etherscan 1
[75]	BLSTM_ATT	1		RSC, RCS
[68]	GVD-net	10		Smartbugs-wilds 4
[76]	HuntFlow	1		[92]
[77]	-	4		Etherscan1, Eth2Vec [41]
[69]	SmartFuSE	3		ESC and VSC [63,93]
[51]	SCVSN	1		SolidiFI 2, Smartbugs-wilds 4
[89]	CDRF	4		Etherscan 1
[70]	-	4		ESC and VSC [63,93]
[71]	SAGP	2	✓	SmartBugs 3, [63]
[84]	BiGRU	5		Etherscan 1, +, ++	✓
[58]	SC-VDM	4		Etherscan 1
[59]	SolGPT	4		+++
[41]	Eth2Vec	7		Etherscan1
[44]	Dynamit	1		Etherscan1
[85]	-	4		#
[86]	-	4		##

¹ https://goto.etherscan.com; ² https://github.com/DependableSystemsLab/SolidiFI-benchmark; ³ https://smartbugs.github.io; ⁴ https://github.com/smartbugs/smartbugs-wild; ⁵ https://library.dedaub.com; ⁶ https://xblock.pro/#/datasets; ⁷ https://github.com/smartbugs/smartbugs-curated; ⁸ https://github.com/xf97/HuangGai/tree/master/manualCheckDataset; ⁹ https://consensys.github.io/smart-contract-bestpractices/known_attacks/; ¹⁰ https://github.com/vntchain/go-vnt; ¹¹ https://blockchain.unica.it/; ¹² https://blog.positive.com/; ¹³ https://gitter.im/orgs/ethereum/rooms/; ¹⁴ https://github.com/cleanunicorn/karl; * https://github.com/thec00n/etherscan_verified_contracts; ** https://github.com/ZZXLX/contract_hex/tree/master; *** https://github.com/tintinweb/smart-contract-sanctuary; + https://github.com/enzymefinance/oyente; ++ https://github.com/ethereum/remix-project; +++ https://github.com/Messi-Q/Smart-Contract-Dataset; # https://github.com/sec-bit/awesome-buggy-erc20-tokens; ## https://github.com/Messi-Q/Cross-Modality-Bug-Detection.

provides more details regarding vulnerability detection machine learning-driven approaches, including their method’s name, number of detected vulnerabilities, which method supports unknown vulnerabilities, their benchmark datasets, and the class imbalance issue. According to our research findings, a substantial majority of frameworks identify Reentrancy as a widely recognized vulnerability in SCs. Moreover, many techniques concentrate solely on identifying one particular vulnerability.

With regards to Figure 5, looking from an overall perspective, a large proportion of published articles have not addressed unknown vulnerabilities (UV) in their proposed frameworks. Moreover, roughly a tiny minority of researchers use balanced datasets in their research. In addition, Etherscan and SmartBugs are the most favored datasets.

7. Discussion

In this section, some challenges and key findings from our experiments are presented to provide insights into using machine learning-based methods for vulnerability detection.

Numerous investigations have been conducted to identify vulnerabilities, relying on traditional methods including static analysis, symbolic execution, dynamic analysis, formal verification, and fuzzy testing. These approaches frequently require tailored implementation of testing, analysis, and verification algorithms to cater to the unique SC language and vulnerability categories. Notably, their analysis algorithms may vary significantly when dealing with source code and bytecode, which restricts their adaptability to emerging languages or vulnerability types. Thus, developing vulnerability detection methods compatible with multiple platforms or programming languages can increase the generalizability of machine learning-driven solutions. Furthermore, employing single code representations like AST and control flow graphs in deep learning vulnerability detection methods can result in the occurrence of false vulnerability detection and the absence of semantic information. Moreover, the familiar deep learning method fails to consider the running logic and data flow of the program, as it treats the SC source code solely as a text sequence. As a result, it cannot extract the constructed features. In addition, these methods are limited to a particular range of vulnerability categories and do not consider the complex relationships among nodes.

Moreover, data-level methods are used by researchers to cope with the class imbalance issue, while these techniques suffer from reducing the total amount of information, increasing training time, and overfitting. Unlike data-level methods, algorithmic-level methods increase the importance of the positive class, and they can shift in a way that decreases bias towards the negative class. These characteristics can enhance the effectiveness of machine learning-based solutions in detecting multiple types of vulnerabilities in SCs.

Furthermore, the proposed unknown vulnerability techniques are focused on unknown vulnerabilities that are objectively present but cannot be discovered by existing vulnerability detection tools. In other words, unknown vulnerabilities have similarities with known vulnerabilities. Additionally, there might be particular unknown vulnerabilities that are not constrained by any limitations and may exhibit no similarities to recognized vulnerabilities. In this context, our proposed strategy is to use novelty and anomaly detection techniques since anomaly detection refers to non-typical events that only happen under exceptional circumstances. In contrast, novelty detection can identify unknown vulnerabilities that a machine learning system is not aware of during training. Moreover, we believe that the detection of unknown vulnerabilities suffers from the lack of a unique definition. Thus, the detection of unknown vulnerabilities assumes a crucial role in the development of a robust and efficient framework.

8. Conclusions and Future Work

In recent times, there has been a notable surge in interest in the application of machine learning-driven solutions for detecting vulnerabilities in smart contracts (SCs). Nevertheless, to the best of our knowledge, this is the first SLR that has focused on machine learning-based vulnerability detection approaches. This paper utilized a search query to find articles published from 2019 to 18 May 2024. Finally, we reviewed 55 articles published in peer-reviewed scientific research databases like IEEE, ScienceDirect, Springer, Wiley, Taylor & Francis, and ACM. The current study did not analyze all available studies. Non-English articles, editorials, book chapters, and books were removed from the review. However, given the growing number of studies in this area, it was impossible to cover all studies.

We proposed a taxonomy for machine learning-driven strategies under three different machine learning perspectives (i.e., classical, deep learning, and ensemble models). Moreover, we explored what strategies were implemented to deal with the class imbalance issue in this field. Furthermore, we highlighted machine learning-driven vulnerability detection methods that focused on detecting unknown vulnerabilities in SCs.

The significant observations are: (1) The graph neural network methods present a superior ability to detect vulnerabilities in SCs. (2) Ensemble learning-based methods can improve the reliability of vulnerability detection. (3) Methods for detecting unknown vulnerabilities need more attention since the existing machine learning-based vulnerability detection methods mainly focus on known vulnerabilities. The challenge lies in the fact that unknown and known vulnerabilities share certain similarities. (4) The development and reliability of vulnerability detection methods may be negatively affected by imbalanced datasets. Thus, analyzing vulnerabilities can pose a challenge when dealing with imbalanced data.

According to RQs, (1) deep learning techniques, specifically those based on graphs, have been observed to be more prevalent compared with other machine learning approaches; (2) attention to unknown vulnerabilities is essential to improving the Limited Coverage downside; and (3) the importance of the class imbalance issue has been almost ignored in machine learning-based vulnerability detection tools.

To conclude, this study can be followed by researchers who tend to develop ML-driven solutions for vulnerability detection in SCs. Alternatively, we assert that several challenges remain open and may be worth absorbing the attention of researchers.

We suggest that researchers shift frameworks onto Curriculum Learning (CL) [94] to reduce reliance on adequate expert knowledge and automatically extract key information about smart contract vulnerabilities. CL is a training strategy that trains a machine learning model from easier to more complex data. It benefits from hard and soft regularization terms in which they can not only detect noisy data and outlier samples but also reflect the importance of training samples and improve performance. In this context, an intuitive question is: Could the curriculum-like training strategy benefit machine learning-driven vulnerability detection tools?