Applying Artificial Intelligence in Cryptocurrency Markets: A Survey

Abstract

The total capital in cryptocurrency markets is around two trillion dollars in 2022, which is almost the same as Apple’s market capitalisation at the same time. Increasingly, cryptocurrencies have become established in financial markets with an enormous number of transactions and trades happening every day. Similar to other financial systems, price prediction is one of the main challenges in cryptocurrency trading. Therefore, the application of artificial intelligence, as one of the tools of prediction, has emerged as a recently popular subject of investigation in the cryptocurrency domain. Since machine learning models, as opposed to traditional financial models, demonstrate satisfactory performance in quantitative finance, they seem ideal for coping with the price prediction problem in the complex and volatile cryptocurrency market. There have been several studies that have focused on applying machine learning for price and movement prediction and portfolio management in cryptocurrency markets, though these methods and models are in their early stages. This survey paper aims to review the current research trends in applications of supervised and reinforcement learning models in cryptocurrency price prediction. This study also highlights potential research gaps and possible areas for improvement. In addition, it emphasises potential challenges and research directions that will be of interest in the artificial intelligence and machine learning communities focusing on cryptocurrencies.

1. Introduction

Cryptocurrency markets have experienced a remarkable transformation recently, both in statistical terms and public acknowledgement. The total market capitalisation of cryptocurrencies rose from 267.8 billion to 1.664 trillion between November 2017 and March 2022 (based on data on www.tradingview.com, accessed on 1 November 2022). Excluding Bitcoin as the most recognised cryptocurrency with the highest market capital, the capitalisation of “alternate coins”, or altcoins, grew from 86.31 billion to 1.007 trillion in the same period. Moreover, there is an increasing trend among international companies that have invested in cryptocurrency-based business solutions. For example, the possibility of purchasing Tesla merchandise using Dogecoin is a revealing example of the phenomenon in the cryptocurrency world. Thus, cryptocurrency markets have become more popular, established, and their integration with other financial assets appears inevitable [1,2].

Cryptocurrency markets have been a prominent subject for investors. Many small investors have made a small fortune by speculating on cryptocurrencies. They follow trending news on social media or waves of excitement in cryptocurrencies. On the other hand, there is also a risk, and investors are prone to losing money because of the volatile nature of cryptocurrency markets. For example, based on the analysis of data on (www.coinmarketcap.com, accessed on 1 November 2022), the Bitcoin market capitalisation dropped from 1.18 trillion to 935 billion USD in only ten days in April 2021, and it almost halved to 602 billion USD in the following three months up to July 2021 (www.coinmarketcap.com). Thus, this provided a trading system to predict market directions, and eventually mitigating investment risks is an open problem in cryptocurrency markets. To this end, there is an increasing number of promising methods to deal with cryptocurrency analysis and predict inherent trends. However, high volatility and the fact that cryptocurrencies do not behave like fiat currencies make them uncertain for investors [3] and require them to have their own specific trading strategies. The uncertainty in predictability suggests that several price formation elements of cryptocurrencies still have not been thoroughly examined. Hence, these challenges have led to the necessity of further investigation to gain a better understanding of cryptocurrency markets [4].

Asset trading has been experiencing substantial changes with the rapid enhancements in computing and telecommunication infrastructures that support a highly productive improvement in quantitative trading [5]. Besides, the growing number of investors and, consequently, transactions, in addition to multiple sources of alternative data such as social media hashtags, tweets, and feeds, have built massive blocks of big data around cryptocurrency markets. As a result, market participants look beyond traditional approaches to build automated-profitable trading models dealing with this data. In contrast to quantitative trading approaches utilised for decades, financial technology (FinTech) companies have significantly changed these trading strategies to handle big data appropriately. FinTechs are mainly established around integrating artificial intelligence (AI) and finance [6]. As one of the primary tools of FinTechs, AI has become a popular research subject in quantitative trading and presents excellent efficiency in discovering profitable trading rules [7,8]. In contrast to traditional statistical models, AI techniques as a ubiquitous analysis tool can deal with high-dimensional data and complex environments, such as cryptocurrency markets. Further references on FinTech companies and their approaches to dealing with the challenges and complexities of cryptocurrencies can be found in [9,10,11].

The number of academic publications is rapidly growing in the field of cryptocurrencies in connection to various domains, such as economics, finance, and AI. To further support this claim, a python package called “academic-keyword-occurrence” [12] is used to extract the historic word occurrences of a set of keywords in academic papers from Google Scholar. Figure 1 demonstrates the exponential increase in the number of publications based on four different keywords, namely altcoin, altcoins, cryptocurrency, and cryptocurrencies, from 2015 until 2021, based on the package results.

Alternatively, another possible categorisation of literature in the realm of cryptocurrency trading is about trading software systems, systematic trading, emergent trading technologies, crypto-asset portfolio research, market condition, and so forth [13]. In this paper, we mainly focus on the third category (emergent trading technologies), including econometric methods, machine learning technology, etc. As evidence of growing literature on the application of AI models in predicting cryptocurrency prices, several surveys and reviews of recent publications on this topic have been published recently. For example, Mosavi et al. [14] review deep learning methods in various finance and economic sectors such as insurance, auction mechanisms, and banking. Additionally, Sabry et al. [15] present a survey on current challenges and opportunities of AI applications in several cryptocurrency domains, such as volatility prediction, cryptocurrency mining, and fraud detection. Murat Ozbayoglu et al. [16] provide a state-of-the-art snapshot of deep learning models for a range of financial applications, including algorithmic trading, risk management, fraud detection, and behavioural finance. The primary purpose of this paper is to review recent studies on AI applications in cryptocurrencies as well. However, unlike other studies, it only focuses on cryptocurrency price prediction by machine learning (ML) models and provides an in-depth review of the challenge of predicting the price of cryptocurrencies. Furthermore, this work investigates the financial aspects underlying the price prediction presented in recent studies, which are examples of studies conducted for the first time on this topic. Therefore, to achieve the main purpose of our study, this survey aims to answer the following questions to fill the gap in the existing surveys of recent publications in AI and cryptocurrency price prediction.

• What factors influence the price of cryptocurrencies?
• What is the state-of-the-art in AI research in the domain of cryptocurrency price prediction?
• What are current gaps in the literature that may be addressed by conducting future research?

The rest of the paper is structured as follows. Section 2 primarily focuses on the domain knowledge of the cryptocurrency sector from a financial and economic perspective. Then, ML approaches used in predicting cryptocurrency prices are divided into two subsections, namely supervised and reinforcement learning, and analysed in Section 3. Subsequently, Section 4 discusses current research gaps and recommends potential future research. Finally, Section 5 concludes the paper.

2. Cryptocurrency Markets

In this section, a short introduction to cryptocurrencies and blockchain technology, as the digital infrastructure underpinning the financial transactions of cryptocurrencies, is provided. Then, the fundamental drivers of price formation on cryptocurrencies are discussed from the perspective of a famous framework for the life cycle of data science projects, namely, the Cross-Industry Standard Process for Data Mining (CRISP-DM). In addition to supply-demand equilibrium in deciding cryptocurrency prices, it is observed that several other drivers influence price fluctuations in cryptocurrency markets.

2.1. A Short Background on Cryptocurrencies

Traditionally, economic systems typically use third-party financial institutions such as banks to process payments. These entities operate as mediators among parties for exchanging funds, and they have complete control over all transactions. Although traditional financial systems have effectively executed financial transactions, there are a few drawbacks to such systems. For example, limits on the amount of money transacted, transaction costs, a lack of trust, security problems, transparency, and flexibility are among them. There were unsuccessful attempts to develop decentralised unregulated virtual currencies to tackle these problems. However, it was the invention of blockchain by Stuart Haber and W. Scott Stornett in 1991 that addressed some of these difficulties [17].

As a distributed database solution, blockchain technology maintains an expanding list of data records confirmed by participant nodes. Each transaction information is recorded in a public ledger that is available to all nodes. Thus, a blockchain network is more transparent than a centralised transaction network involving a third party. Furthermore, as participant nodes are all anonymous in a blockchain network, it is more secure for other nodes to verify the transactions [18]. The term blockchain is derived from the fact that it is incrementally built up by blocks of data or chains of data where each of those chains contains three main elements [19] as follows:

• Block-data is a set of messages or transactions;
• Chaining-hash is a copy of the hash value of the immediately preceding block; and
• Block-hash is the calculated value of the hash of the data block.

Since the invention of the technology, various blockchain projects have been developed in different sectors, from international payments and healthcare to music royalties tracking. Moreover, attempts have been made to create a decentralised virtual currency, such as Bit Gold in 1998. However, the advent of Nakamoto’s Bitcoin paper [20] in 2008 was a historic milestone for both blockchain and digital currencies.

Cryptocurrencies rely on a decentralised peer-to-peer network, using blockchain technology to store all transactions in a decentralised public ledger. In addition, blockchain technology is responsible for verifying transactions and synchronising nodes in a network of participants. It provides a digital platform for fast, efficient, and secure cryptocurrency transactions.

In a blockchain system for cryptocurrencies, a block header consists of the main metadata in cryptocurrencies, such as the previous block, block version, hash, timestamp, nonce, and transaction details. It is used to identify a particular block in a blockchain system, and each block has a unique header.

Table 1. Information attributes of a blockchain system for cryptocurrencies.

Headers	Attributes and Definitions
	Total circulation of crypto: the total number of mined cryptocurrency coins
Crypto statistical info	Crypto price: the price of the coin
	Market capitalisation: the total value of cryptocurrency in circulation
	Blockchain size: total size of the blockchain
	Avg. block size: average block size for the past 24 h
Block info	Avg. trans per block: average number of transactions per block for the last 24 h
	Avg. payments per block: the average number of payments per block for last 24 h
	Total no. of trans: the total number of transactions on blockchain
	Median (avg.) confirmation time: the median (avg.) time for a mined block to be added to the public ledger
	Total hash rate: the estimated number of terahashes per second
	Hash rate distribution: an estimation of hash rate distribution amongst the largest mining pools
Mining info	Network difficulty: the difficulty of mining a new block
	Miners revenue: total value of cryptocurrency block rewards and transaction fees paid to miners
	Total transaction fee
	Fees per transaction: average transaction fees per transaction

presents information attributes included in a cryptocurrency blockchain, and a short description of each item is presented.

Traditional financial systems and cryptocurrencies can be compared from various perspectives, such as their monetary, regulatory, and decentralised functionality. From a monetary function view, three conventional money functions are the medium of exchange, store of value, and unit of account. In economics, a medium of exchange is a transitional instrument widely accepted for exchanging products and services, such as fiat money. In the monetary economy, a store of value means any form of financial assets that can be used to save and eventually be exchanged in the future. For example, gold and silver are two popular commodities that have been saved throughout history due to their ability to store value. A unit of account is a monetary unit that can measure the value of other assets or products. Although cryptocurrencies can technically fulfil the monetary role of medium of exchange, they are not currently capable of performing two other monetary roles [21]. From a regulatory standpoint, the regulation should not target cryptocurrencies, as there are several practical constraints for regulating this decentralised open-source ecosystem. In other words, the centralised command-and-control approach of traditional regulation may not succeed in decentralised cryptocurrencies. Instead, regulations seem to aim at the interface between financial associations and cryptocurrencies [22]. Regarding the decentralised function, financial institutions are the essential intermediaries that control and facilitate financial transactions in a centralised economic system. However, these institutions can dominate financial activities when they grow and impose disproportionate market power. On the contrary, decentralised platforms reduce transaction costs and eliminate the monopoly power of any entity by creating effective decentralised peer-to-peer networks [23].

Moreover, the uniqueness of cryptocurrencies is based on three distinct features, namely, anonymity, decentralisation (no central authority), and double-spending attack protection [24]. However, even with these different characteristics, there is still ambiguity between electronic money and cryptocurrency. The European Central Bank (ECB) has provided a clear distinction between virtual and electronic currencies, presented in

Table 2. Comparison of electronic money and virtual currency based on the definition of the European Central Bank.

	Money	Electronic	Virtual
Attributes
Money format		Digital	Digital
Acceptance		By undertakings other than the issuer	Usually within a specific virtual community
Legal status		Regulated	Unregulated
Issuer		Legally established electronic money institution	Non-financial private company
Supply of money		Fixed	Not fixed (depends on issuer’s decisions)
Supervision		Yes	No
Type of risk		Mainly operational	Legal, credit, liquidity, and operational

based on [25]. In particular, virtual money is different from electronic money considering its format, acceptance, legal status, issuer, and so forth.

Bitcoin is the most popular and well-established cryptocurrency and a de facto standard in cryptocurrency markets. However, there are nearly 17,000 other cryptocurrencies called altcoins, which have around 50% of the total market capitalisation. Figure 2 shows the market capitalisation of the top ten cryptocurrencies in January 2022 based on CoinMarketCap (www.coinmarketcap.com). This figure shows BTC, ETH, USDT, BNB, USDC, SOL, ADA, XRP, LUNA, and DOT, which stand for Bitcoin, Ethereum, Tether, Binance coin, USD coin, Solana, Cardano, Ripple, Terra, and Polkadot.

At a technical level, altcoins use almost the same or similar blockchain technology used by Bitcoin. Nevertheless, altcoins fundamentally seek to introduce some new alternative features compared to Bitcoin to increase their market share. For instance, Ethereum incorporates almost all of Bitcoin’s attributes with additional features such as a fee limit or a digital platform to run smart contracts (SC) [26]. Litecoin, as another example, is designed to cope with the computational requirements for mining cryptocurrency coins. Furthermore, the aim of creating Dash coin is to have a faster transactions process and extend privacy protection [27]. Aside from the technical differences, cryptocurrencies can generally be categorised as follows.

• Mining-based altcoins: they have similar characteristics to Bitcoin, and as the name implies, they use the typical mining process for generating new coins. One of the most famous leading altcoins belonging to this category is Ethereum.
• Stablecoins: One of the main issues of mining-based cryptocurrencies is high volatility and fluctuation in their prices, making their trading complicated. Hence, stablecoins were introduced to address this challenge, which is valued based on stable existing currencies such as fiat currencies. Additionally, stable assets behind stablecoins secure and support their value. For example, Diem (previously Libra), developed by Facebook, and Tether, with the highest capital among stablecoins, are two famous coins in this cryptocurrency category.
• Utility tokens: this type of cryptocurrency can give value to its investors by providing access to a future product or service. For example, Filecoin is a famous open-source cryptocurrency that aims to store data on hard drive storage spaces compared to cloud storage companies such as Amazon.

Cryptocurrencies can also be categorised based on the currency domain, platform category facilities, and the domain of application [28].

The cryptocurrency ecosystem mainly relies on many exchanges as they provide trading tools for investors. Exchanges facilitate cryptocurrency trades and allow traders to sell or buy digital assets for other assets, primarily fiat currencies. Centralised exchanges (CEX), decentralised exchanges (DEX), and hybrid exchanges are three types of cryptocurrency exchanges. CEX is governed by a corporation, such as Binance and Coinbase. DEX provides an automated process for peer-to-peer trades. For example, DODO and Uniswap are part of this group. Finally, hybrid exchanges, as the name implies, are a combination of both centralised and decentralised exchanges [29]. There are numerous cryptocurrency exchanges worldwide. For example, Bitcoin can be traded in various currency cross pairs in at least 387 markets [30]. Payment methods, supported coins, transaction fees, transaction speed, and trading volume are a few factors of an exchange’s popularity.

Table 3. Information about a number of most known cryptocurrency exchanges.

Exchange	No. Supported Coins	Transaction Fee (%)	Headquarter Location	Founded
Binance	320+	0.100	Malta	2017
Coinbase	40+	0.500	San Francisco, US	2012
BitMex	160+	0.075	Eden Island, Seychelles	2014
Okex	230	0.150	Malta	2017
Huobi	310+	0.200	Seychelles	2013
Bitfinex	30+	0.200	Hong Hong	2012
Kraken	60	0.260	San Fransisco, US	2011
Bitterx	320+	0.350	Seattle, US	2014
BitStamp	10+	0.500	Luxembourg	2011
KuCoin	270+	0.100	Mahe, Seychelles	2017

presents a number of the most known exchanges along with their specifications. The number of supported coins in the second column of the table shows the number of cryptocurrencies available in the exchanges for trade. Furthermore, their transaction fees, the location of the headquarters, and the year in which the exchange was founded are represented in columns three, four, and five in Table 3, respectively. It is observed that transaction fees can vary for each exchange and are between 0% and 4% per trade.

Cryptocurrencies, similar to most new technologies, come with their disadvantages. Decentralisation supports cryptocurrencies to be less prone to government interventions, and blockchain technology provides a medium with the semi-anonymity of owners. Therefore, these lead to involvement in illegal activities as one of the main drawbacks [15]. For instance, the famous story of the Silk Road website portrayed a negative image of Bitcoin, which was a dark web black market for trading illegal substances [31]. It is worth mentioning that illicit internet activities are not only limited to cryptocurrencies and blockchain technologies. For example, it is estimated that 10 million Americans are victims of online identity theft each year [32].

2.2. Price Determinants of Cryptocurrencies

The growing importance of cryptocurrency markets in changing the worldwide economy is not negligible. It is necessary to understand the underlying features of financial aspects of cryptocurrencies before taking further action in devising an ML model with high prediction capability in forecasting their behaviours. Moreover, there are motives for both governmental bodies and individual investors to expand their knowledge about the factors behind the price formation of cryptocurrencies. For instance, cryptocurrency markets can be a potential source of instability in economic systems. Consequently, they may influence policymakers’ decisions and limit the authority of governments all over the world. Moreover, an investor’s asset portfolio allocation and risk management strategy can be influenced by fluctuations in cryptocurrencies [2]. To manage the consequences, the econometrics of cryptocurrencies seek interdisciplinary research to explain how traders’ actions inform price discovery [33].

For the purpose of interdisciplinary research, the Cross-Industry Standard Process for Data Mining is considered a framework for analysing the problem of price prediction. It is one of the most common methodologies used to describe data science projects (Figure 3) [34]. CRISP-DM is a process model that naturally describes the data science life cycle and contains six phases. First, the business understanding phase is about the objectives and requirements of a particular project. Next, the data understanding phase focuses on identifying, collecting, and analysing datasets to achieve the project goals. Then, the data preparation phase processes the final dataset to be used in the modelling phase, which builds and assesses several models to find the best model(s). Finally, the evaluation and deployment phases are self-descriptive, which are about evaluating results and deploying the model into action. A significant observation and motivation behind incorporating CRISP-DM are that the study of current literature in the cross-disciplinary field of AI and cryptocurrencies demonstrates that AI community researchers mainly focus on the modelling phase of CRISP-DM, and the business understanding phase is mainly neglected. Thus, this section provides information to understand the factors behind the price formation in cryptocurrency markets by focusing on business, finance, and economic perspectives by means of CRISP-DM.

Factors that drive the price of cryptocurrencies can be divided into internal factors and external drivers, including marco-financial and attractiveness for investors [35]. The internal factors are about the generation of the data blocks via the mining process in a blockchain network, and they directly impact the supply and demand of each cryptocurrency [36]. Furthermore, similar to a fiat currency, which is valued based on the trust in which it will be accepted as an exchange medium, cryptocurrencies establish their trustworthiness among market participants by creating trust and credibility [1]. In other words, the attractiveness of cryptocurrencies for investors is considered an external factor driving price fluctuations.

Despite the general supply-demand equilibrium concept that stabilises the price of commodities, the price of Bitcoin seems to follow different paradigms. Ciaian et al. [1] expand a gold standard model to investigate factors in the formation of Bitcoin price variation. This study considers three main factors including Bitcoin supply and demand market forces, its attractiveness for investors, and global macro-financial development. The results demonstrate that the impact of attractiveness has changed over time, which means there are periods when investors are interested in investing in Bitcoin regardless of the supply-demand equilibrium, and the macro-financial development factor has a minor influence on Bitcoin price in the short run. The key finding is that the demand-side pressure substantially affects the Bitcoin price in comparison to supply-side factors.

Another fundamental question in the price prediction of a cryptocurrency is whether other financial assets impact the change in price value or price fluctuations. Therefore, the degree to which cryptocurrency markets integrate with other financial assets and their own interconnections is important to be investigated by investors for building better price-prediction strategies. Ji et al. [2] use a directed acyclic graph (DAG) and Granger causality test to uncover the causal structure between three categories of assets, including international equities and currency assets. An energy index is also incorporated as a commodity component due to the fact that electricity price is important in Bitcoin mining. The results demonstrate that Bitcoin is an isolated market in the contemporary causal structure. However, there are time-variant causal relationships, especially in bearish market trends, according to the time-lagged causality structure. In another study, Zeng et al. [37] use a vector autoregression (VAR) model to examine the connectedness and volatility spillover effects relationship between Bitcoin and conventional financial assets. There is a weak relationship between Bitcoin and financial assets; however, the correlation among cryptocurrencies is comparably stronger. Moreover, there is a stronger connectedness via negative returns than positive returns. The study also demonstrates that the connectedness between Bitcoin and other financial assets varies over time. Corbet et al. [38] also use a spillover approach to explore relationships between Bitcoin, Ripple, and Litecoin, as well as relationships with a combination of financial assets. Volatility spillover results are categorised based on low and high frequencies. There is little evidence of volatility spillovers between cryptocurrencies and financial assets at short frequencies. However, there are obvious connections between cryptocurrencies in high and low frequencies.

Furthermore, other studies investigate the correlation between the prices of cryptocurrencies to build better price prediction models. The study by Gkillas et al. [39] does a pairwise comparison of ten different cryptocurrencies considering ninety combinations of them. They apply multivariate extreme value theory and estimate a bias-corrected extreme correlation coefficient. They find that the extreme correlation coefficient is more observable in bear markets than bull markets among different pairs. In a negative return state, the extreme correlation coefficient generally has the highest value where one side of the cryptocurrency pair is either Bitcoin or Litecoin. On the other hand, for positive return exceeding, either Bitcoin or Ethereum is one side of the cryptocurrency pair. The results also demonstrates pairs including Dash on one side, which has a weak dependency level with other cryptocurrencies. Stosic et al. [40] also analyse cross-correlations between 119 cryptocurrencies. This study utilised Random matrix theory and minimum spanning trees (MST) to develop their methods. In contrast to the common belief that Bitcoin has a global influence on all cryptocurrencies, five communities of cryptocurrencies are discovered by MST, indicating the existence of diverse collective behaviour between cryptocurrencies. Bitcoin and Ethereum communities are extremely close to each other among all communities. The Synereo community, including cryptocurrencies such as Tether, has a strong anti-correlation. This result is aligned with the findings of [39].

Additionally, due to the high volatility in cryptocurrency prices, it is reasonable to investigate whether volatilities are just the results of bubbles. A study by [41] examines possible pricing bubbles in Bitcoin and Ethereum as two leading significant cryptocurrencies. Their findings reveal that the mining difficulty, hash rate, and cryptocurrency liquidity of the coins are fundamental drivers of their price structures. In their results, there is no obvious indication of bubble-based volatility in cryptocurrency prices. However, there are bubbles related to incidents such as a Bitcoin seizure from the Silk Road website or the declaration from the US court that Bitcoin meets an investment contract definition.

3. Artificial Intelligence and Cryptocurrencies

The term “artificial intelligence” was first used in a summer research project at Dartmouth College in 1956 [42]. It was originally founded as a research discipline for building a machine to simulate every aspect of learning or any other feature of intelligence that can be described in principle [43]. Although there is still not a universally accepted definition for AI, as a general definition, AI leverages computers to solve problems and make decisions by mimicking the human brain’s thinking ability and intelligence. AI empowers machines to exhibit human-like behaviours, such as driving a car autonomously, improving corporate productivity, or completing dangerous tasks [44]. Despite having several winters as a seasonal metaphor, when technology, business, and the media paid less attention to AI, and recent predictions about another possible winter [45], major tech companies still prioritise AI over other IT initiatives. As a consequence, the implementation of AI systems is expanding rapidly in a wide spectrum of domains, from health, criminal justice, welfare, and stream history-influenced video viewing suggestions to real-time evaluation of enormous data sets (big data) and fraud detection [46,47].

There is no clear border to distinguish different topics in AI and ML. However, it is essential to differentiate these relevant concepts from each other for an adequate understanding of AI. We follow the frequently used framework in categorising AI and ML models, which considers ML as a subset of AI. Therefore, we have adopted a plausible categorization from [48,49] to assist in explaining the models used for cryptocurrency price prediction as shown in Figure 4. As one of the subsets of AI, ML is an umbrella term for methods and algorithms enabling machines to discover patterns without explicit programming instructions [8]. ML methods perform the experiential “learning” associated with human intelligence and have the capability to improve their analyses via computational algorithms [47]. Different ML models learn from data in different ways including supervised, unsupervised, semi-supervised, reinforcement learning, and so forth. Supervised learning aims to use labelled data to train algorithms for prediction or classification. On the other hand, the purpose of unsupervised learning is to organise datasets into similar groups or clusters. There are no labels associated with data points in unsupervised learning models. A combination of supervised and unsupervised learning when there are only partially labeled data, which leads to semi-supervised learning. As the next category of ML models, RL algorithms choose an action based on previously received rewards in response to past actions and the environmental setting, intending to find decision policies to maximise the total reward.

3.1. Application of Machine Learning in Cryptocurrency

Computer scientists, mathematicians, statisticians, and data scientists have been developing and refining various ML algorithms to extract high-quality knowledge from data to develop trustable and accurate price and movement predictors to perform profitable trading in cryptocurrencies. In this section, we focus on the literature on supervised and unsupervised learning applications in cryptocurrencies. RL in cryptocurrencies will be discussed later in Section 3.2.2. Portfolio performance in financial markets is affected by various factors, including the quality of input data, the data granularity, market maturity, and forecasting models [50]. Thus, in the following, the papers reviewed in supervised and unsupervised learning models are catalogued and summarised based on these metrics in addition to features including baseline methods, trade frequencies, coin types, and data sources (eight attributes in total) in

Table 4. Key to accessing the information about each publication in Table 5.

No	Reference	No	Reference	No	Reference
1	Mittal et al. [4]	6	Chowdhury et al. [51]	11	Chen et al. [52]
2	Poongodi et al. [53]	7	McNally [54]	12	Kim et al. [55]
3	Patel et al. [28]	8	McNally [54]	13	Derbentsev et al. [34]
4	Alessandretti et al. [56]	9	Peng et al. [57]	14	Lamon et al. [58]
5	Sun et al. [59]	10	Jang and Lee [36]	15	Lahmiri and Bekiros [60]

and

Table 5. Detailed information on supervised and unsupervised ML models used in the literature covered in this survey. For each study, the method used for the price prediction task, the baseline model for comparing approaches (if exists), in addition to the target feature, and the specification of data is provided.

No	Method	Baseline(s)	Prediction Feature	Frequency Prediction	Performance Metric(s)	Data Period	Crypto(s)	Data Source(s)
1	Multivariate LR	-	Highest price	1D	F-score	-	10 coins	kaggle.com
2	LR and SVM	-	Price	1H	Cost function accuracy score	-	Ethereum	etherchain.org
3	LSTM and GRU	LSTM	Price	1D, 3D, 7D	MAE, MSE, MAPE, and RMSE	2016–2020	Litecoin Monero	investing.com
4	LSTM, regression	Simple moving average strategy	Cumulative Return	3D, 5D, 7D, and 10D	Geometric mean return Sharpe ratio	2015–2018	10 coins	coinmarketcap.com
5	LightGBM	SVM and RF	Price	2D, 2W, and 2M	AUC indicator	2018	42 coins	investing.com
6	Gboosted trees, NNs, and K-NN	LSTM, RNN, and ARIMA	Closing price	1D	RMSE and squared correlation	2016–2019	9 coins and cci30	coinmarketcap.com cci30.com
7	LDA, LR, RF, XGB, QDA, SVM, and LSTM	-	Price	1D and 5 min	Precision, accuracy, recall, and F1-score	2017–2019	Bitcoin	coinmarketcap.com Bitcoinity.org blockchain.com
8	RF, LSTM, and RNN	ARIMA	Price	1D	Sensitivity, specificity, precision, accuracy, and RMSE	2013–2016	Bitcoin	Coindesk Blockchain.info
9	SVR-GARCH, and SVR		Price	1H and 1D	RMSE and MAE	2016–2017	Bitcoin, Ethereum, and DashCoin	alt19.com fxhistoricaldata.com
10	BNN	SVR, and LR	Price	1D	RMSE and MAPE	2011–2017	Bitcoin	Bitcoincharts.com
11	VADER	-	Price	1D	Pearson R and p-value	2018	Bitcoin and Ethereum	Twitter’s API, Google Trends
12	AODE	-	Price	-	Accuracy rate, F-measure, and Matthews correlation coefficient	2013–2015	Bitcoin, Ethereum, and Ripple	Coindesk CoinMarketCap Etherscan RippleCharts
13	BART	ARIMA and ARFIMA	Price	5D, 10D, 14D, 21D, and 30D	RMSE	2017–2019	Bitcoin, Ethereum, and Ripple	Yahoo Finance
14	Logistic Regression, SVM, and Naive Bayes	-	Price	1D	Confusion matrix accuracy	2017	Bitcoin, Ethereum, and Litecoin	Kaggle.com, Twitter’s API
15	SVR, GRP, RT, kNN, FFNN, BRNN, and RBFNN	-	Price	5 min	RMSE	2016–2018	Bitcoin	-

. In addition, Table 4 has been created for the purpose of quick access to the references of the papers in Table 5.

Hybrid ML models are widely used in the cryptocurrency domain. Hybrid ML models are constructed by combining ML methods that make use of the relative advantages of each model on its own to enhance the overall model’s performance [61]. Patel et al. [28] propose a hybrid model of long short-term memory (LSTM) and gated recurrent unit (GRU) networks to predict the price of two less common altcoins, namely, Litecoin and Monero. Although the proposed model effectively predicts daily prices, it is not effective for longer periods, for example, a seven-day prediction. They also demonstrate that LSTM-based methods are more suitable for short-term prediction. Furthermore, ref. [54] uses a combination of recurrent neural networks (RNN) and LSTM methods to forecast the price of Bitcoin, and a random forest (RF) for feature engineering. Since different exchanges provide different prices of cryptocurrencies based on their supply and demand factors, the study considers the average closing price of five major exchanges instead of one specific exchange. Thus, the data are less noisy, and they lead to a more generalised model. As a result, the LSTM performance is considerably better for long periods, such as 100 days, while it is not the case for RNN. In another study incorporating LSTM, Alessandretti et al. [56] build an investment portfolio system to predict the return on investment by using three supervised learning methods, two gradient-based boosting decision trees, and one LSTM. First, two regression models relying on XGBoost are applied to all cryptocurrencies, and then different models separately are checked on each currency. LSTM performs best; however, gradient-boosting decision trees allow better interpretation. Interestingly, based on their findings, the models have a better performance when the coins’ prices are fed based on Bitcoin price rather than their worth converted into the US dollar value.

Although cryptocurrencies are theoretically recognised as tools for economic decentralisation, they rely on fiat currencies such as the US dollar to be traded. Thus, whether traditional financial assets affect the price of cryptocurrencies is an open question. Jang and Lee [36] implement Bayesian networks (BNs) to predict the price of Bitcoin. As an advantage of the proposed model, blockchain information and microeconomic factors are incorporated to forecast the price. Blockchain-related variables, ten global macroeconomic indices, and five international exchange rates are the inputs of the BN model. Log price and volatility of Bitcoin are compared to linear regression (LR) and support vector regression (SVR) as baseline models. The Bayesian network model demonstrates better predictive performance. Furthermore, their investigation shows that the price of Bitcoin is correlated to factors including macroeconomic variables such as stock indices, exchange rates, and oil prices. In addition, another study by Sun et al. [59] obtains forty features from around ten indices to explain the changes in cryptocurrency markets. In the paper, 42 different types of cryptocurrencies are tested by a novel algorithm called a light gradient boosting machine (LightGBM). The results demonstrate that the model is more suitable for a two-week prediction. Furthermore, the algorithm demonstrates improved forecasting for the top 10 cryptocurrencies in more mature markets. However, they analyse a relatively small amount of data, accounting for seven months, though a dataset with a more extended period is required for generalising the model.

Linear regression models are widely used as an approximation for real-world models and have commonly been the most popular in the past. However, real-world phenomena tend to have quite a lot of nonlinear components, and these methods fail to efficiently fit complex datasets where other models tend to be more effective. Mittal et al. [4] implement a multivariate linear regression to forecast the prices of ten different cryptocurrencies. The high price on a given day is predicted based on the open price, close price, and low price values of previous days. In the same stream of research, Poongodi et al [53] obtain the prediction of the price of Ethereum by combining Linear regression and support vector machine (SVM) models. The results reveal that SVM without additional features has about ten percent higher accuracy than linear regression.

Lahmiri and Bekiros [60] examine seven ML models for Bitcoin intra-day price prediction. The models are categorised into three distinct groups of algorithms, including a statistical ML category such as support vector regression, algorithmic techniques (regression trees), and AI-based neural network topologies. Aside from radial basis function networks (RBFNN), all neural networks outperform other algorithms based on the RMSE metric. Due to the nature of cryptocurrencies, deciding the best approach to model their behaviour is an open question. Thus, in another SVR-based study, Peng et al. [57] compare models in two distinct financial markets of cryptocurrencies and three major fiat currencies. The results are presented as model-based comparisons and financial market-based comparisons. It is understood from the findings that forecasting errors of traditional GARCH family models and SVR models are higher for cryptocurrencies than fiat currencies. Moreover, low-frequency windows have higher error metrics than high-frequency windows in both markets.

Since a large part of the behavioural finance research emphasises that investors’ emotions have significant effects on financial decisions, researchers pay more attention to analysing investor sentiments [62]. Like traditional financial markets, there appears to be a correlation between media sentiment and cryptocurrency price movements. Therefore, one research trend is to employ sentiment analysis in cryptocurrency price prediction. Abraham et al. [3] examine three sources of data to perform sentiment analysis for predicting the price of Bitcoin and Ethereum. Based on the results, although Google trends and tweet volumes are highly correlated with the price of cryptocurrencies, the sentiment of tweets is determined as an unreliable source of prediction. The sentiment of tweets is inclined to keep staying positive even when the market trend is falling. In another study on sentiment analysis, Amon et al. [58] use daily news headlines and tweets for price fluctuations and implement four classification algorithms for a binary classification problem. While logistic regression is the best classifier for Bitcoin and Litecoin, a version of the Naive Bayes classifier performs the best prediction for Ethereum. Although the proposed model identifies the general trends, it cannot anticipate the price fluctuations when they are not in line with the general market trend. Kim et al. [55] study the transaction number and information of online communities of cryptocurrencies by using averaged one-dependence (AODE) and user comments and replies for price predictions. The results generally vary across different cryptocurrencies. Both the Bitcoin price and the transaction number demonstrate a significant positive association with user replies. It is consistent with the outcomes of [52], where there are tweets with positive sentiment. However, unlike Bitcoin, Ethereum, and Ripple have a negative correlation with very negative comments.

A unique aspect of financial assets is that several external factors may seriously influence their price movements. These factors may include macroeconomic factors, a trader’s psychology, fundamental elements, market sentiment, and so forth [63,64]. It is hence a considerable challenge to discover all influencing factors and extract intelligent trading rules from the dynamic financial markets environment [65,66]. Therefore, features containing valuable information are necessary to develop decision-support trading systems for cryptocurrency price prediction in financial markets. This highlights the vital role of feature engineering and selection in cryptocurrency markets [63]. There is a trade-off between a higher number of features, which increases the training time, and having a lower number of features, which decreases model performance. Chen et al. [52] introduce a different perspective regarding data granularity and feature engineering. Features of low and high frequencies are separately selected, and distinct methods are implemented on low and high frequencies data. The results suggest statistical methods such as logistic regression perform better on low-frequency data while other ML approaches including linear discriminant analysis and SVM accomplish better outcomes for high-frequency data. In addition, the study introduces two sentimental features, Baidu media search volume, and Google trend search, combined with other features as another novelty.

Further ML models have been used to investigate different aspects of cryptocurrencies. Chowdhury et al. [51] apply four ML models for forecasting the closing price of nine different cryptocurrencies. The prediction of cryptocurrencies index 30 (cci30) (www.cci30.com, accessed on 1 November 2022), which measures the overall growth of cryptocurrency markets, and the implementation of models on the RapidMiner platform, are two distinct characteristics of this research. A comparison of algorithms shows that a K-nearest neighbours (K-NN) model performs poorly in prediction. In another study, Erbentsev et al. [34] adopt a modified model of a binary auto-regressive tree (BART) from standard models of regression trees. The proposed model combines different components of classification, regression trees, and autoregressive models ARIMA. For three major cryptocurrency leaders, the algorithm is tested based on the dynamics of the cryptocurrencies, namely, stable period, falling trend, transition dynamics (change of trend), and rising trend. The model demonstrates a better performance than the baseline models in both falling and rising trends; however, its forecast is worse in periods of rapid trend changes.

3.2. Reinforcement Learning

One initial goal of AI is to create a fully autonomous agent capable of interacting with an environment for learning optimal behaviours that improve over time considering a particular objective [67]. To respond to this goal, an RL agent interacts with an environment over time by following a procedure. At each time step, the agent is in a state (from a state space) and then selects an action (from an action space belonging to the state). The agent follows a policy (the agent’s behaviour) for all these state-action-state procedures and receives a scalar reward [68]. This configuration of moving from one state to another by selecting actions and receiving rewards over time makes RL particularly effective in achieving the initial goal of the AI field. Figure 5 shows a generic framework for an agent–environment interactions. At any time t, an agent is in a state $S_t$, chooses an action $A_t$, and receives an reward of $R_t$. This action takes the agent to another state $S_{t+1}$. All these states, actions, and rewards are defined inside an environment.

RL follows a different paradigm in terms of learning in comparison to supervised and unsupervised learning in ML. Basically, it is a principled mathematical framework of experience-driven autonomous learning [69]. RL cannot be classified as a supervised learning approach since labeled data are not provided to an agent. Furthermore, RL methods are suitable for problems that have sequential dynamics and involve optimisation of a scalar performance objective, while supervised learning methods are usually applied to problems that involve static input-output mappings and minimise a mismatch between the data and the model [70]. On the other hand, it is not also an unsupervised learning approach, since the agent is provided with information about a rewarding scheme that guides the algorithm through each state-action-state iteration [71].

RL has been around for more than two decades; however, two recent breakthroughs in the last seven years have significantly changed the course of this approach by going beyond theoretical concepts via successful implementation of RL algorithms across a range of problems [72]. First, in 2015, an agent on classic Atari 2600 games surpassed a professional human game tester and all previous algorithms [73]. Later, in 2016, AlphaGo achieved a 99.8% winning rate against all other computer programs in the game Go and could defeat a human professional player for the first time [74]. Despite these successful instances of incorporating RL in problem-solving, there still are several unexplored areas that warrant investigation in using this framework to solve challenging real-world problems.

3.2.1. An Overview on Reinforcement Learning Overview

The fundamental nature of RL is to learn through a process of trial and error. This means that an RL agent has continuous interactions with its environment, and it learns to modify its behaviour based on rewards received after taking actions [67]. RL can be considered as creating an environment for a learner to learn by interacting with the environment and choosing actions without being told. The learner chooses actions in an environment based on the consequences of prior actions referred to as rewards. The agent modifies its behaviour to choose better actions in order to maximise cumulative rewards. To this end, RL has two distinctive features. One is the trial-and-error search to optimise cumulative rewards. The second is the delayed reward which implies that actions may not only have an immediate reward, but they may affect all subsequent rewards or may have rewards in the later stages of learning [14,75].

An RL problem can be modeled and described as a Markov decision process (MDP) by a five-tuple $(\mathcal{S},\mathcal{A},\mathsf{\pi },\mathcal{R},\lambda )$. An RL algorithm is composed of the following:

• States and observations $\mathcal{S}$: A state is a complete description of the circumstances of an environment, and an agent obtains all the information regarding the environment through a state. Additionally, observation is considered in case some information might be omitted due to a partial representation of a state.
• Action spaces $\mathcal{A}$: Each environment allows performing several actions. The set of all legitimate actions in a given environment is called an action space. Unlike a continuous action space, an agent has a finite number of available actions in a discrete action space.
• Policy $\mathsf{\pi }$: A policy $\pi$ is the way an agent behaves at a given time. It determines the action that has to be chosen by the agent when it is in a particular state. In a probabilistic setting, it maps the current state of the environment into a set of probabilities for taking actions from the action space.
• Rewards $\mathcal{R}$: A reward component is an important concept in RL. It is an immediate or instantaneous gain that an agent receives when choosing an action in the current state to move to the next state.
• Discount factor $\lambda$: The quantity $\lambda \in [0,1)$ is the discount factor and generates discounted rewards to prevent infinite cumulative rewards when running for a long period. As a general intuition, discounted rewards mean rewards today are worth more than rewards tomorrow. If it is zero, an agent considers only immediate rewards; while $\lambda$ closer to one means that the agent evaluates its actions based on cumulative rewards in the future.

Figure 6 is one possible taxonomy of RL algorithms produced partially based on work by Zhang and Yu [76]. A short description of each component is provided in the following.

Model-Free and Model-Based Approaches

Based on the proposed taxonomy in Figure 6, two computational strategies constitute reinforcement learning, namely, model-free (MF) and model-based (MB) approaches. In a model-based approach, rewards for taking actions come from internal models of the environment, whereas in a model-free approach a value is associated with actions to maximise the rewards. Additionally, model-free approaches require access to a large number of interactions for the purposes of training, while model-based methods are capable of achieving an optimal policy quickly based on what they have learned about the environment [77]. A model of environment allows an agent to predict future states, and a model-based strategy helps to enhance the efficiency of RL substantially [78]. Additionally, model-free approaches are more suitable for learning complex tasks, such as games.

Value-Based Methods

In an RL setting, an agent receives a reward signal based on choosing an action to move from one state to another. The agent maximises the total received reward (or discounted total reward in an RL setting with an infinite horizon) [75]. The discounted total reward, also called the value, in a state that is gathered through state-action-state iterations in RL, plays a central role in guiding the algorithm towards an optimal policy. These values are obtained dynamically using the Bellman equation. Moreover, a key concept in RL is the trade-off between exploration and exploitation performed by an agent in searching among states and actions considering a non-optimal policy learned so far (on-policy scheme) or without a particular policy (off-policy scheme). An agent should not only learn from previous experiences in earning rewards, but it has to also explore new unattended states and actions to ensure improvements in selecting actions in the future. To this end, on-policy methods seek to improve the policy previously used in decision making while off-policy methods evaluate a policy different from what was used to generate data [75]. In addition, off-policy methods are more sample efficient, meaning that they need comparably lees samples to reach a certain level of performance than on-policy methods because they can learn from any trajectory sampled from the same environment. However, not having stable interactions with the value functions or their approximations is a fundamental shortcoming [79]. Moreover, Off-policy methods learn from previously collected data with no interaction. Thus, it is possible in principle to leverage massive data. Nevertheless, Off-policy RL algorithms have significant technical obstacles in practice, arising from the distributional shift between the policy deducted from collected data and the learned policy [80]. SARSA and Q-Learning are the two most common value-based methods, that use off-policy and on-policy schemes, respectively.

Policy-Based Methods

In a value-based approach, an agent learns an optimal policy based on its interactions with the environment in terms of the estimates of an optimal action-value function. In contrast, a policy-based method learns an optimal policy without using a value function and is only based on its interactions with the environment. In particular, policy-based methods incorporating gradients (PG) compete with value-based techniques in a discrete environment and are successfully implemented in continuous control (as opposed to a discrete action space, in which agents decide which distinct action to select from a set of actions, in continuous action space, actions are defined as real-valued vectors [81]). The efficiency of value-based methods is limited in continuous control settings due to the dimensionality problem; however, policy-based methods are more effective in continuous space by directly learning a policy distribution [82,83]. Instability and requiring small learning rates for training are two problems of standard PG methods, given that the strategy of policy update is simple. Trust-region policy-based (TRPO) schemes address the problems by bounding policy updates to a trust region to optimise standard PG algorithms [83,84]. Deep deterministic policy gradients (DDPG) and proximal policy optimisation (PPO) are other popular PG algorithms.

3.2.2. Reinforcement Learning Applications in Cryptocurrency Markets

There are a wide range of applications of RL in different industries from games [85] and robotics [86] to natural language processing [87] and computer vision [88]. RL is also a natural solution for several problems in finance and economics such as option pricing, multi-period portfolio optimisation, and risk management due to their dynamic nature. The focus of this section is on the application of RL in dealing with better investment policies in cryptocurrency markets.

There are two main streams of literature for using RL models in cryptocurrency markets. Firstly, there is growing interest in explaining and predicting cryptocurrency price behaviours and trading strategies [89]. This stream describes that cryptocurrencies consist of distinct governing factors which distinguish them from traditional financial assets. However, it focuses on the price volatility of the cryptocurrency assets, and there is no systematic analysis of cryptocurrency trading [90]. The second line of research covers economic applications for developing automated trading systems [89]. Since it is in its early stages of development, most studies may not provide practical solutions and a competitive edge for investors [90,91]. In addition, creating better market simulations by including transaction fees and applying feature engineering to cryptocurrency trading data are investigated in the literature.

To mimic a real cryptocurrency market, Sadighian [92] designs a new framework, called deep RL applied to market-making, that uses advantage actor-critic and proximal policy optimisation RL algorithms. These two policy-based methods constructed upon gradient are trained by Bitcoin, Ether, and Litecoin data. Positional profit-and-loss and the trade completion ratio are defined as the reward function, and the average daily returns are compared between currencies considering total accumulated rewards. BTC has the highest return on investment among all cryptocurrencies investigated in the study. In this work, limit order books, order flow imbalances, and trade flow imbalances are used as the environment state space, which is the novelty of the study. However, the short time-frame, two days, could be increased to achieve improved generalisation of the proposed approach. In addition, Koker and Koutmos [89] employ an active trading strategy using deep RL to achieve the greatest risk-adjusted returns. To this end, a gradient ascent algorithm optimises the Sortino ratio as the reward function. Five portfolio performance metrics, including cumulative returns, Sharpe and Sortino ratios, maximum drawdown, and value-at-risk are compared with the buy-and-hold trading strategy as the baseline model. The results indicate that the proposed model outperforms the buy-hold strategy in all tested five cryptocurrencies. On a similar topic of extracting trade ideas, Sattarov et al. [93] propose a four-layer architecture deep RL for recommending trading suggestions to maximise short-term profit. The reward function is based on the difference between the selling price and the purchase price. Additionally, as its unique feature, the agent receives negative rewards if the number of sequential purchases is more than a limited number (20 in the study). Thus, it prevents the agent from many open positions to increase the reward. Double-cross strategy, swing trading, and scalping trading strategies are compared with the proposed model based on the number of trading actions and trading quality. Deep RL finds more trading opportunities and hence performs a higher number of trades than other trading strategies. In addition, the net return is 14.4% for Bitcoin within one month of simulated trading, while the scalping strategy has only 6.1% growth as the best performer of traditional strategies.

The study by Lee et al. [94] proposes the integration of inverse RL and agent-based modelling for Bitcoin market movements. Inverse RL assumes that the reward of an agent is unknown and tries to infer the reward function in the presence of optimal policy or observed behaviour. The RL model proposed by the study has an environment whose states are defined by price, value possessed, the value realised of Bitcoin, and the difference between its price and a moving average of the price. A main advantage of the proposed model is that it considers interactions between market participants. Another variant of deep RL, called deep double Q-networks, and proximal policy optimisation are utilised to solve the problem of optimising the placement of limit orders by [95]. Designing a relatively comprehensive state-space RL environment, by including a wide range of financial aspects, is the main advantage of the study. To accomplish the task, market states are divided into four main sections, including transaction imbalances, best-order volumes and imbalances, volatility and current price drift, and current cost of liquidity. The results indicate that the agent guided by the proximal policy optimisation for order placement is more aggressive against unfavourable price moves. The study also concludes that queue imbalances and volumes are the most important features impacting the behaviour of the agents.

A multi-agent RL method is proposed by Lucarelli and Borrotti [96] to develop a deep Q-learning portfolio management framework. A local agent guided by three learning techniques, namely, deep Q-Networks, double deep Q-Networks, and duelling double deep Q-Networks is evaluated based on two global reward functions. The reward functions are the sum of the local rewards, the weighted sum of the Sharpe ratio of the portfolio, and the net portfolio return. As a result, the framework outperforms weighted portfolio and portfolio selection using a genetic algorithm as the baseline. In another study, a state-augmented RL framework for managing portfolios on two well-known datasets, namely, the Bitcoin market and the HighTech stock market, is proposed by Ye et al. [97]. The proposed portfolio management algorithm performs better in terms of accumulated and risk-adjusted profits in comparison to the standard RL-based portfolio management. Jiang and Liang [98] implement a deterministic deep RL method and convolutional neural network (CNN) for portfolio management. Weight allocation and adjustment are performed by using a Monte Carlo policy gradient method for CNN training. The CNN agent outperforms three benchmark strategies and two out of three portfolio management algorithms.

One unrealistic assumption in many trading algorithms is that they neglect trading fees. In particular, in automated systems that perform many trades in a short amount of time, the accumulated fees, even negligible for a single trade, are considerable and can affect the optimal policy over a long time horizon. Hegazy and Mumford [99] examine seven algorithms, including an altered recurrent RL (RRL), for studying Bitcoin trading strategies. To evaluate the performance of algorithms, they define several metrics including cumulative weighted confidence without a trading fee, profit considering only 0.25% of transaction fees, and finally, incorporating the correct transaction rate for the task of predicting the Bitcoin price. RRL generates the best results for both with or without fee considerations. It likely demonstrates the way RL algorithms incorporate fees into the process of optimising trading strategies rather than adding only during trading. In addition, Betancourt and Chen [100] propose an RL model that takes into account the same assumptions as [99], which are similar to real markets. Their RL model utilises proximal policy optimisation that considers several assets in cryptocurrency markets and adopts the number of assets that may vary over time. In this study, the two agents evaluate the total return and the differential Sharpe ratio metrics. The model performs competitively in comparison to temporal-difference learning, convolutional neural network, and double Q-networks.

The majority of existing RL studies either use all the features in historical trade data, including open price, high price, low price, close price, volume, and so forth, or a row combination of them, while proper feature engineering is not often emphasised in those studies. Feature engineering focuses on obtaining significant factors by using feature selection methods [101]. In some studies, feature engineering is treated separately from the then task of learning. For example, in [95], a neural network architecture is used to directly learn significant features for cybersecurity attack detection, instead of handcrafted approaches. In another study for dealing with feature engineering in the realm of cryptocurrencies, Weng et al. [102] propose a portfolio trading system with a deep RL model using a multidimensional attention-gating mechanism for twenty cryptocurrencies. In this study, XGBoost performs for quantifying the importance of features before feeding data into the RL network. Among all available features, close, high, and low prices have the highest importance, respectively, as determined by XGBoost.

4. Discussion and Potential Future Research

As evidenced in the discussion so far, AI studies in cryptocurrency markets have been growing, especially during the last few years. However, with all advancements in both science and technology, several unexplored topics still warrant further investigation in the cryptocurrency domain. For this reason, this section discusses potential research directions based on the literature reviews in Section 3.1 and Section 3.2.2.

4.1. Integration within Cryptocurrencies or with Other Financial Assets

Recent fundamental changes in cryptocurrency markets indicate that they are increasingly integrated with other traditional financial and economic systems. For instance, Coinbase, one of the most popular cryptocurrency exchanges, went public on the NASDAQ exchange, and there are an increasing number of ways in which cryptocurrencies can be used to pay for services and products. This evidence strongly indicates the acceptance of cryptocurrencies by existing financial bodies as well as the general public. Since financial and economic systems continuously and dynamically evolve, investigating the integration of cryptocurrency markets with other financial assets can be of significant benefit. Internationally, it is important to study the financial impacts of the price of traditional assets and commodities on the price of cryptocurrencies, and vice versa. In a similar fashion, the question of whether or not the prices of cryptocurrencies can be predicted based on changes in the prices of other financial assets (e.g., oil and the US dollar). Investigating the influence of financial markets and political sentiments on the price of cryptocurrencies and whether or not cryptocurrencies can influence traditional assets such as oil prices, is of significant interest.

In addition, the influence of different cryptocurrencies on each other warrants further attention and investigation. In particular, the emergence of government-based cryptocurrencies, governmental restrictions on trading and using cryptocurrencies in some countries (e.g., China), and preferences regarding certain coins based on their liquidity ease, may create an influence from one coin to another. Furthermore, creating a profitable portfolio of different cryptocurrencies needs a better understanding of the interactions and influences between the coins. The necessity of attending to these gaps is evident in the recent changes in the cryptocurrency markets. For example, the market capitalisation of altcoins has increased substantially in the last three years, where it has surged from 500 million dollars in January 2019 to 1.5 trillion dollars in December 2021 (www.coinmarketcap.com). Therefore, the question of whether leading cryptocurrencies such as Bitcoin and Ethereum influence the prices of other altcoins needs to be studied. Modelling approaches such as Bayesian networks [103] can be used to gain insights by modelling causal relationships between cryptocurrencies and other financial systems, and among cryptocurrencies themselves.

Although studies have been undertaken to investigate integrating cryptocurrencies within other markets, they are mainly based on financial and economic models, such as a gold standard model applied in [1] to study the price of Bitcoin, or in another study GARCH models (statistical models) are widely used in the financial sector. Moreover, there are other gaps in the literature that need further attention. One impending issue is that of dealing with big data, where a sound approach from AI can be of great benefit. Additionally, statistical models used to date place a strong emphasis on explainability rather than accuracy. Therefore, selecting an appropriate AI framework that is accurate and explainable considering feature engineering, fitting, resampling, and testing is crucial. Furthermore, designing trustable AI tools that are capable of predicting market movements, price changes, and profitable trade strategies is vitally important. In particular, such a tool should be able to take as input different types of data including price histories, information from social media, news items from international political interactions, governmental decisions from economically influential countries, and expert opinion.

4.2. Macroeconomic Factors as the States in RL

Reinforcement learning appears to be an appropriate tool founded within AI to deal with open problems within cryptocurrency and its price prediction. In particular, RL can be a useful tool in dealing with unknown factors of cryptocurrency price fluctuations. This capability of RL eventually leads to detecting optimal policy for designing automated trading systems. The tool needs to be able to deal with feature engineering, analysing correlational and causal relationships among historical price data of different assets, reliable price prediction capability, social media analysis, and expert opinion. The main challenge in developing such a tool is the definition of the fundamental elements of an RL system (discussed in Section 3.2.1). For example, the connection between macroeconomic factors and cryptocurrencies is studied in supervised and unsupervised learning techniques used in prediction tasks. However, it is not straightforward to use these factors in an RL framework. Thus, designing an RL environment whose states contain macroeconomic information can be beneficial as a potential future research direction. In addition, the definition of action space, reward function, and the structure of a policy, and the discounting factor in dealing with challenges in cryptocurrency markets needs further attention and investigation.

4.3. Minor Challenges Related to Cryptocurrencies

There are further research directions that can be considered in the area of using AI techniques in predicting the behaviour of cryptocurrencies. For example, the COVID-19 pandemic has influenced our daily lives in many ways, from working habits to public transport, and the economy. There are studies that have been conducted concerning the effects of the COVID-19 outbreak on the cryptocurrency market, such as [104,105], which mainly focus on financial models. It is worthwhile to investigate the changes in the parameters of AI models that are fitted using the data before and after the pandemic due to the different characteristics in data including changes in the mean and variance. This comparison and analysis can be used to develop strategies and create policies is situations in the future, when events similar to the COVID-19 pandemic may occur. Moreover, even though cryptocurrencies are designed as a tool of decentralisation, their price historically has demonstrated sensitivity toward governmental laws such as banning cryptocurrency mining in some countries such as China. On the other hand, ML techniques are increasingly being used across different fields, for example, in law and the legal domain in general [106]. Thus, the application of ML models to investigate the effects of laws and regulations on cryptocurrency price prediction can be an area of potential future research. In addition, studying the time granularity of time series data used in training models can impact the accuracy of predictions, and hence, methods for data prepossessing need to be considered carefully to determine an appropriate time granularity.

4.4. Sentiment Analysis

Behavioural finance research increasingly emphasises the point that investors’ emotions significantly affect financial decisions. A number of researchers have devoted their attention to investor sentiment [62] by considering their activities in social media. There are several supervised and unsupervised ML studies that combine sentiment indicators with some financial features. However, RL studies mainly focus on sentiment analysis as a standalone market input to predict prices. Hence, it is worthwhile studying sentimental factors along with other financial inputs in an RL environment. As an example of the impact of sentiment analysis and social influencers, Figure 7 shows the volatility in Bitcoin prices due to two instances of Elon Musk’s comments regarding Bitcoin. It is clear in the figure that large green (increase in price) and large red (decrease in price) candlesticks, for positive and negative comments of Elon Musk on Twitter, respectively, are due to the influence traders accept based on the comments of influencers. Similar diagrams can be extracted after major geo-political events and the emotional influence they can create on the trading behaviour of investors. Thus, investigating the effects of social media on overall market sentiment combined with other data sources can reveal important drivers in cryptocurrency price formation and prediction.

4.5. Further Attention to Altcoins

Bitcoin, as the first cryptocurrency, is studied and referenced more than other coins. For example, Figure 8 presents Google books Ngram viewer for comparing Bitcoin (in blue) and altcoins (in green) between 2010 and 2019. In addition, the red curve shows the number of appearances of the word cryptocurrencies in the Google books Ngram viewer to provide a better context for comparison. As observed in the figure, Bitcoin has generally been referenced more than altcoins, demonstrating the potential unexplored areas in altcoins. Additionally, altcoins with enhanced features might be good candidates to diversify investment portfolios and minimise the total risk of investment [107]. Moreover, with the increasing share of the market by altcoins, there is an opportunity to investigate influential factors in the price prediction of the cryptocurrencies among themselves by removing the volatility pertaining to Bitcoin. It might assist in understanding the whole cryptocurrency ecosystem.

4.6. Extreme Condition Detection in Cryptocurrencies

The past financial market turmoil, such as the 2007–2008 Global Financial Crisis or March 2020 market turmoil caused by a global health crisis, has shown that rare extreme market events can have severe consequences and spillover effects for world economies [108]. Therefore, it is important for financial risk management portfolio diversification to control and monitor extreme downside market risk [109]. As a relatively new financial sector, the cryptocurrency market witnessed various extreme market movements, leading to substantial capital losses for investors. Hence, detecting extreme market conditions in the cryptocurrency market can prevent participants from investment loss. Although considerable research has been conducted to deal with these events, anomaly detection in economic data has been widely ignored [110]. Therefore, detecting anomalies in the cryptocurrency market by implementing AI methods is another area of future research that will eventually assist investors in making informed decisions that minimise risk. The methods, such as the local outlier factor, autoencoders, and Bayesian networks, can serve as anomaly detection techniques for this purpose.

5. Conclusions

This paper first uses a cross-disciplinary approach to discuss the price determinants of cryptocurrencies from a financial and economic perspective. Then, recent studies on the use of various AI models in cryptocurrency price prediction are reviewed through a comparative survey. Recent studies on the use of various AI models in cryptocurrency price prediction are reviewed through a comparative survey. As a relatively new field in finance and economics, many open issues still warrant further investigation in the area of cryptocurrencies, and in particular the use of advanced AI methods to achieve accurate prediction of their prices. More specifically, further attention is needed concerning integrating cryptocurrencies with traditional markets and analysing two-way influence in terms of correlational and causal relationships. Meanwhile, AI is capable of addressing these open problems, and future work in this field has to consider different components and tools to cope with the data engineering aspects of a prediction tool for cryptocurrencies. It is expected that this survey serves as a guideline for researchers to comprehend the current state of the literature, and we discuss open challenges that may be considered as part of future research. The potential of AI in the cryptocurrency domain is evident, and research in this direction can lead to great benefits for cryptocurrency market participants and policy- and decision-makers.