Price Prediction of Bitcoin Based on Adaptive Feature Selection and Model Optimization

Abstract

Bitcoin is one of the most successful cryptocurrencies, and research on price predictions is receiving more attention. To predict Bitcoin price fluctuations better and more effectively, it is necessary to establish a more abundant index system and prediction model with a better prediction effect. In this study, a combined prediction model with twin support vector regression was used as the main model. Twenty-seven factors related to Bitcoin prices were collected. Some of the factors that have the greatest impact on Bitcoin prices were selected by using the XGBoost algorithm and random forest algorithm. The combined prediction model with support vector regression (SVR), least-squares support vector regression (LSSVR), and twin support vector regression (TWSVR) was used to predict the Bitcoin price. Since the model’s hyperparameters have a great impact on prediction accuracy and algorithm performance, we used the whale optimization algorithm (WOA) and particle swarm optimization algorithm (PSO) to optimize the hyperparameters of the model. The experimental results show that the combined model, XGBoost-WOA-TWSVR, has the best prediction effect, and the EVS score of this model is significantly better than that of the traditional statistical model. In addition, our study verifies that twin support vector regression has advantages in both prediction effect and computation speed.

1. Introduction

Since the official birth of Bitcoin in 2009, it has attracted worldwide attention. As a cryptocurrency market representative, Bitcoin price fluctuation affects the stability and development of the world economy. The effective price prediction of Bitcoin at a low time cost is an important issue that investors and policy-makers urgently need to solve (e.g., [1,2]). Researchers have used traditional statistical methods and machine learning models to predict and analyze the price of Bitcoin. For example, Khedr et al. [3] summarized the research progress in the cryptocurrency price-prediction field from 2010 to 2020 and proposed gaps in existing research and prospects for related future research. Traditional statistical methods include Bayesian regression (e.g., [1,2]), logistic regression [4,5], linear regression [6], and autoregressive comprehensive moving average (ARIMA) (e.g., [7,8]). Machine learning models include support vector machines (SVMs) [9], random forest (RF) classifiers [10], artificial neural networks (ANNs) (e.g., [11,12,13]), and long- and short-term memory (LSTM) networks (e.g., [14,15]).

Further research has divided Bitcoin prices into separate periods of time. Chen Z et al. [16] divided Bitcoin price prediction into 5 min price prediction and daily price prediction models according to data characteristics. In the 5 min price prediction model, the accuracy of the machine learning algorithm was 67.2%, which is better than that of the traditional statistical methods. In daily price prediction, traditional statistical methods such as logistic regression and linear discriminant analysis have accuracy rates of about 66%, which is higher than those of machine learning models. Similarly, Mudassir M. et al. [17] used classification and prediction models, i.e., ANN, SVM, and LSTM, to study Bitcoin prices for 1 day, 7 days, 30 days, and 90 days. The results show that the accuracy of the classification model is higher than 60%, and the error percentage of the prediction model is lower than $5\%$. Mallqui D.C. et al. [18] used machine learning technology to predict the direction, maximum value, minimum value, and closing price of the daily Bitcoin price. Compared to the original model, the accuracy of Mallqui’s final model was improved by more than $10\%$.

Current Bitcoin price prediction studies have the following two drawbacks. First, the selection of appropriate influencing factors and prediction models for Bitcoin prices is of great significance. How to choose the factors that affect the price of Bitcoin is a central issue. Yechen Zhu et al. [19] verified the impacts of various economic factors on the price of Bitcoin, and the research results show that the dollar index has the greatest impact on the price of Bitcoin, and the price of gold has the least impact. In addition to economic factors, user sentiment can also play a role in the price of Bitcoin. Based on a study of the impact of macroeconomic factors on the price of Bitcoin, Aggarwal A. et al. [20] creatively considered Twitter emotion as an influencing factor and found that Twitter emotion had a positive correlation with the price of Bitcoin. Based on the study of traditional determinants, Ciaian P. et al. [21] found that the attractiveness of Bitcoin represented by Wikipedia to investors and users significantly affected the change in the Bitcoin price. Kim Y.B. et al. [22] extracted all comments and responses posted in crypto-related online communities and rated them based on the positive or negative sentiments of the comments and responses. Based on the averaged one-dependence estimators, a prediction was made by a machine learning model. Three cryptocurrencies (Bitcoin, Ethereum, Ripple) were tested to verify the scientific nature and feasibility of the model.

Furthermore, Wei C. et al. [23] developed a two-stage machine learning model for predicting Bitcoin prices. In the first stage, an artificial neural network and random forest method were used to extract important economic and technical factors affecting the price of Bitcoin. In the second stage, LSTM, ARIMA, the SVR machine model, and the adaptive network fuzzy inference system (ANFIS) model were used to predict the price of Bitcoin. Finally, it is concluded that the prediction effect of LSTM was better than those of other prediction models. However, in it, the change in Bitcoin price is limited to the impacts of a particular category of factors, rather than a comprehensive consideration of all the influencing factors. At the same time, the multicollinearity of selection influencing factors is not considered in the establishment of the model, and a feature-selection stage is missing.

In this study, the XGBoost algorithm and random forest algorithm were used to select the influencing variables. Previous studies have shown that both the XGBoost and random forest algorithms are effective feature selection models (e.g., [24,25,26,27]). The comparison between the selected variable group and the original full variable group is shown in Section 4. The XGBoost algorithm is a type of ensemble learning boosting system. The idea is to integrate many weak classifiers to form a strong classifier (e.g., [28,29]). The random forest algorithm is an ensemble learning bagging approach. It combines the ability to solve regression problems and classification problems. A random forest integrated multiple decision trees through the idea of ensemble learning and averaged the output of each decision tree to obtain the final output result in [30].

There are various types of Bitcoin price prediction models, but they do not remove the influence of model differences on prediction. In addition, the differences between the strengths and weaknesses of the prediction models were not considered in the different periods of the datasets, and the generalization abilities of the models were poor. Therefore, in this study, support vector regression (SVR), least-squares support vector regression (LSSVR), and twin support vector regression (TWSVR) were used as the main predictive models. TWSVR has good generalization, high prediction accuracy, relatively simple calculations, and good robustness to outliers (e.g., [31,32,33]). These three models all fit the data by establishing the prediction belt method, which solves the influence of model-type differences in the overall prediction to a certain extent. At the same time, the ARIMA model was set as the benchmark model and was compared with the main prediction model to obtain the best prediction model of Bitcoin price.

In addition, selecting a reasonable hyperparameter for the model can improve the accuracy of the prediction model (e.g., [34,35,36]). The intrinsic relationship between hyperparameters and prediction accuracy is very complex and unknown, which brings great challenges to the selection of hyperparameters. In this study, the whale optimization algorithm (WOA) and particle swarm optimization (PSO) algorithm were selected to optimize the hyperparameters of the models. Both algorithms are fast and effective community search algorithms.

The WOA is a new optimization algorithm inspired by whale hunting behavior (e.g., [37,38]). Whales are social animals and cooperate to drive and round up prey when hunting. The most important feature of the WOA is to simulate the hunting behavior of humpback whales with random individuals or optimal individuals and the bubble-net attack mechanism of a humpback whale with a spiral. The WOA performs a global search in the early stage and a local search in the late stage. The WOA seeks an optimal solution to the problem by mimicking the way whales hunt (e.g., [37,39]). The WOA randomly generates a group of whales, and the position of each whale represents a feasible solution. During a pod hunt, each whale behaves in two ways. During each iteration, each whale randomly chooses one of two behaviors to hunt. One behavior is where all whales move toward each other to surround prey. The other behavior is when whales swim in circles and blow bubbles to repel the prey. Each whale follows two rules of behavior until the best solution is found. Therefore, the WOA has the advantage of fast convergence while avoiding local optimization [40].

PSO is a random search algorithm based on group cooperation developed by simulating the foraging behavior of birds [41]. In PSO, the solution to each optimization problem is a bird in the search space, also known as a “particle”. Each bird (particle) looks for food in the search space, that is, the optimal solution to the optimization problem in [42]. Each particle does not know exactly where the food is but can sense its general direction. Each particle carries on the next search in the direction it decides, records the position of each iteration during the search and shares it with other particles. Finally, in the process of continuous iteration, the specific location of the optimal solution is searched (e.g., [43,44,45]).

The major contributions in this paper are described as follows:

(a)

Expanding the factors that influence Bitcoin price movements. This article considers 27 influencing factors—that is, the impact of other cryptocurrencies on Bitcoin’s price—and uses the Crypto Fear and Greed Index (F&G) as an indicator of the degree of user demand for Bitcoin. In this study, XGBoost and random forest models were used to select the influencing factors of the price of Bitcoin, the variables selected by the two methods were compared with the full variables in the prediction model, and the index system affecting the prediction of the Bitcoin price was obtained.

(b)

Three models, i.e., SVR, LSSVR, and TWSVR, were used as the main prediction models in this study. Since the prediction accuracies of SVR, LSSVR, and TWSVR are easily affected by the hyperparameters of the models, two optimization algorithms, i.e., WOA and PSO, were used in this study to optimize the hyperparameters of the prediction model.

The remainder of this article is organized as follows. The raw data of the Bitcoin price and its influencing factors are explained and interpreted in Section 2. In Section 3, we provide a detailed description of the prediction models and the optimization algorithm for the hyperparameters of the models. In Section 4, feature selection is reported on the original data, and the combined prediction model is constructed and solved. Section 5 provides a discussion on the prediction models and concludes this paper.

2. Data Collection and Preprocessing

Figure 1 plots the main processes of Bitcoin price prediction. It mainly includes four processes: data collection, data processing, feature selection, prediction model, and parameter optimization. We provide a detailed description of these four processes in the following sections.

2.1. Data Collection

This paper mainly studies the prediction of Bitcoin prices. Bitcoin prices have generally tended to rise over time, rising sharply from 2018 and peaking twice in March and November 2021. We selected relevant daily data from 1 January 2018 to 1 April 2022. The variation trend is shown in Figure 2.

The price of Bitcoin is affected by the internal factors of cryptocurrency and external economic factors. The internal factors of cryptocurrency are divided into three secondary indices: Bitcoin-supply factors, Bitcoin-demand factors, and other cryptocurrency’s prices. External economic factors are divided into three secondary indices: financial-indicator factors, macroeconomic factors, and exchange-rate factors. Twenty-seven factors were considered in this study. Figure 3 summarizes the relevant influencing factors of the Bitcoin price.

The Bitcoin and other encryption currency price data were from https://history.btc123.fans/ (accessed on 4 January 2023). Data on Bitcoin-supply factors and Bitcoin-demand factors are available from https://alternative.me/ (accessed on 4 January 2023). The financial-indicator factor data were from https://www.10jqka.com.cn/ (accessed on 4 January 2023). Macroeconomic factors and the exchange rate data were from https://fred.stlouisfed.org/ (accessed on 4 January 2023). The specific data are explained as follows.

• Mining difficulty (MD) means the difficulty of successfully mining Bitcoin data blocks in transaction information, which represents the level of Bitcoin supply.
• The Crypto Fear and Greed Index (F&G) analyzes the current sentiment in the Bitcoin market. Data levels range from 0 to 100. Zero means "extreme fear", while 100 means "extreme greed". It takes into account the volatility of Bitcoin price changes (25%), the current volume of Bitcoin trading (25%), market momentum, and the public’s appreciation of Bitcoin (50%). The Crypto Fear and Greed Index represents public demand for Bitcoin.
• Ethereum Price (ETH), Litecoin Price (LTC), Bitcoin Cash Price (BCH), and the USDT Price Index (USDT) are the prices of other types of cryptocurrencies, which indirectly affect the price fluctuations of Bitcoin.
• The Dow Jones Industrial Average Index (DJIA), National Association of Securities Dealers Automated Quotations Index (NASDAQ), Standard and Poor’s 500 Index (S&P 500), and CBOE Volatility Index (VIXCLS) are important measures of the U.S. stock market volatility.
• The Financial Times Stock Exchange 100 Index (FTSE 100) is an important measure of volatility in the UK stock market.
• The Hang Seng China Enterprises Index (HSCEI), Hang Seng Index (HSI), A/H share premium index (AHP), and Shanghai Securities Composite Index (SSEC) are important measure indices of volatility in China’s stock market.
• Gold price (GOLD) and silver price (SILVER) are important indicators to measure the price fluctuation of rare metals in the United States and are also macroeconomic factors affecting the Bitcoin price change.
• West Texas Intermediate (WTI) is an important index for measuring the price fluctuation of US crude oil and is a macroeconomic factor affecting the Bitcoin price change.
• The effective federal funds rate (EFFR) is an important index to measure the fluctuation of the funds rate in the United States, and it is a macroeconomic factor affecting the change in Bitcoin price.
• The 10-year break-even inflation rate (T10YIE) is an important index for measuring the inflation level in the United States and is a macroeconomic factor affecting the Bitcoin price change.
• The US dollar index &reg (USDX) is an important index that comprehensively reflects the exchange rate of the US dollar in the international foreign exchange market.
• Australian dollar to US dollar exchange rate (AUD∖USD), European dollar to US dollar exchange rate (EUR∖USD), Great Britain pound to US dollar exchange rate (GBP∖USD), 100 Japanese yen to US dollar exchange rate (100 JPY∖USD), Swiss Franc to US dollar exchange rate (CHY∖USD), and Canadian dollar to US dollar exchange rate (CAD∖USD) represent the exchange rates of the US dollar in each foreign exchange market.

2.2. Data Preprocessing

2.2.1. Missing Data Completion through Linear Interpolation

Bitcoin prices and related data are collected through websites and databases, so there are missing data and noise. Missing values are filled in by linear interpolation. The specific formula is expressed as follows:

$$\begin{array}{c}\hfill \frac{Y-Y_0}{Y_1-Y_0}=\frac{X-X_0}{X_1-X_0},\end{array}$$

(1)

where $X_0,X_1$, and X are the values of the previous period, the next period, and the current period of the known sample data, respectively; and $Y_0,Y_1$, and Y are the values of the previous period, the next period, and the current period of the missing sample data, respectively.

2.2.2. Data Normalization and Division

Normalization is used to process the data to eliminate the influence of dimensions of the model [46,47]. The specific formula is expressed as follows:

$$\begin{array}{c}\hfill X_{new}=\frac{X_{old}-X_{min}}{X_{max}-X_{min}},\end{array}$$

(2)

where $X_{min}$, $X_{max}$, $X_{old}$, and $X_{new}$ are the minimum, maximum, original, and normalized values of the sample data, respectively. Figure 4 shows the normalized historical data, including the supply and demand of Bitcoin, other cryptocurrency prices, financial indicators, macroeconomics, and exchange rates.

Through the data of this period, this study aimed to determine the key variables that affect the price change of Bitcoin and more effectively predict the price of Bitcoin on the basis of previous studies. In this study, the data of the first 1315 periods are defined as the training set to fit the model, and the data of the last 200 periods are defined as the test set to evaluate the quality of the model. The test set is approximately 13.2% of the total data.

2.2.3. Feature Selection for Price Prediction of Bitcoin

We drew on the experience of previous studies and adaptively selects more external economic and financial factors of Bitcoin. The total number of influencing variables is as high as 27. The 27 variables were completely included in the prediction model, which may affect the prediction accuracy, so we used the XGBoost algorithm and random forest algorithm to screen the influencing variables. Previous studies have shown that both the XGBoost algorithm and random forest algorithm are effective feature screening models [25,27,48,49]. The comparison between the selected variable group and the original full variable group is shown in Section 4.

3. Price Prediction of Bitcoin

The main prediction models used in this study are support SVR, LSSVR, and TWSVR. Complete features (27), XGBoost features (11), and random forest features (6) were substituted into the prediction model; and the WOA and PSO were used for optimization to form a hybrid model, including data screening, optimization, and prediction. Finally, we used the expected variance score (EVS), coefficient of determination ($R^2$), mean absolute error (MAE), mean square error (MSE), root-mean-square error (RMSE) and mean absolute percentage error (MAPE) to measure the prediction effect of the hybrid model. At the same time, this study uses CPU time to measure the running rate of the hybrid model (e.g., [50,51]).

3.1. Main Forecasting Model

In this study, support vector regression and its variants were selected as the main prediction model of the experiment. SVR, LSSVR, and TWSVR have good generalization, high prediction accuracy, relatively simple calculation, and good robustness to outliers [33]. These models are applicable to the prediction of Bitcoin price at this time.

3.1.1. Support Vector Regression

SVR is a model that introduces the idea of a support vector machine into a regression problem [52]. Differently from the traditional regression model, which directly calculates the loss based on the difference between the model predicted value $f\left(x\right)$ and the real value y, SVR assumes that it can tolerate the deviation of $\epsilon$. It is equivalent to establishing an interval band with width $2\epsilon$. If the predicted value of the model is included in the interval band, then the prediction is considered accurate. The solution procedure of the SVR problem is similar to that of the SVM problem. The interval band must be maximized; that is, ${∥\omega ∥}^{-1}$ is maximized and ${∥\omega ∥}^2$ is minimized. Thus, the SVR problem can be formulated as shown in Figure 5.

$$\begin{array}{cc}\hfill & min\frac{1}{2}{\parallel \omega \parallel }^2,\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\left|y_i-\left({\omega }^T{x}_i+b\right)\right|\le \epsilon ,\forall i.\hfill \end{array}$$

(3)

Introduce the insensitive loss function:

$$\underset{\omega ,b}{min}\frac{1}{2}{\parallel \omega \parallel }^2+C\sum _{i=1}^m{l}_{\epsilon }\left(f\left(x_i-y_i\right)\right.,$$

(4)

where C is the regularization coefficient, which determines the importance of the loss function. $l_{\epsilon }$ is the insensitive loss function, and the specific form is:

$$\begin{array}{c}\hfill l_{\epsilon }\left(Z\right)=\left\{\begin{array}{c}\hfill 0,if\left|z\right|\le \epsilon ;\\ \hfill \left|z\right|-\epsilon ,otherwise.\end{array}\right.\end{array}$$

(5)

Here, the inequality constraints of the original problem are not convex functions, and slack variables $\widehat{\xi }$ and $\check{\xi }$ are introduced

$$\begin{array}{c}\hfill \begin{array}{cc}& y_i-({\omega }^T+b)-\epsilon \le {\xi }_i,\hfill \\ & \underset{lower}{\underbrace{-\epsilon -{\xi }_i\le {\omega }_i+b-y_i}}\le \epsilon +{\xi }_i,\hfill \\ & -\epsilon -{\xi }_i\le \underset{upper}{\underbrace{{\omega }_i+b-y_i\le \epsilon +{\xi }_i}}.\hfill \end{array}\end{array}$$

(6)

$$\begin{array}{c}\hfill \begin{array}{cc}& s.t.f\left(x_i\right)-y_i\le \epsilon +\widehat{\xi },\hfill \\ & y_i-f\left(x_i\right)\le \epsilon +\check{\xi },\hfill \\ & \widehat{\xi }\ge 0,\check{\xi }\ge 0,\forall i.\hfill \end{array}\end{array}$$

(7)

Next, we use the Lagrange multiplier method. The Lagrange multipliers $\widehat{\mu }\ge 0$, $\check{\mu }\ge 0$, $\widehat{\alpha }\ge 0$, and $\check{\alpha }\ge 0$ are introduced to obtain the Lagrange function £º:

$$\begin{array}{cc}\hfill & L\left(\omega ,b,{\widehat{\mu }}_i,{\breve{\mu }}_i,{\widehat{\alpha }}_i,{\check{\alpha }}_i,{\widehat{\xi }}_i,{\breve{\xi }}_i\right)\hfill \\ \hfill & =\frac{1}{2}{\parallel \omega \parallel }^2+C\sum _{i=1}^m\left({\widehat{\xi }}_i+{\breve{\xi }}_i\right)-\sum _{i=1}^m{\breve{\mu }}_i{\breve{\xi }}_i-\sum _{i=1}^m{\widehat{\mu }}_i{\widehat{\xi }}_i\hfill \\ \hfill & +\sum _{i=1}^m{\breve{\alpha }}_i\left(f\left(x_i\right)-y_i-\epsilon -{\breve{\xi }}_i\right)+\sum _{i=1}^m{\widehat{\alpha }}_i\left(f\left(x_i\right)-y_i-\epsilon -{\widehat{\xi }}_i\right).\hfill \end{array}$$

(8)

Take the partial derivatives separately and bring them into the original function to solve the KKT conditions:

$$\left\{\begin{array}{c}{\breve{\alpha }}_i\left(f\left(x_i\right)-y_i-\epsilon -{\breve{\xi }}_i\right)=0,\\ {\widehat{\alpha }}_i\left(f\left(x_i\right)-y_i-\epsilon -{\widehat{\xi }}_i\right)=0,\\ {\widehat{\alpha }}_i{\breve{\alpha }}_i=0,\widehat{{\xi }_i}{\breve{\xi }}_i=0,\\ \left(C-{\breve{\alpha }}_i\right){\breve{\xi }}_i=0,\left(C-{\widehat{\alpha }}_i\right){\widehat{\xi }}_i=0.\end{array}\right.$$

(9)

The final support vector regression model is as follows:

$$f_{SVR}\left(x\right)={\omega }^{T}x+b,$$

(10)

where

$$\begin{array}{cc}\hfill & \omega =\sum _{i=1}^m\left({\widehat{\alpha }}_i-{\breve{\alpha }}_i\right)x_i,\hfill \\ \hfill & b=y_i-\sum _{j=1}^m\left({\widehat{\alpha }}_j-{\breve{\alpha }}_j\right)x_j^T{x}_i.\hfill \end{array}$$

(11)

3.1.2. Least-Squares Support Vector Regression

LSSVR is a fusion algorithm of SVR and ordinary least squares regression [31]. While inheriting the advantages of SVR, LSSVR creatively replaces the insensitive loss function $l_{\epsilon }$ with the error two-norm and replaces the inequality constraint with the equality constraint. The original convex quadratic programming problem is transformed into a linear equation system, which reduces the complexity of the algorithm and improves the running speed of the algorithm. However, while simplifying, LSSVR gives up the robustness of the SVR model for outliers and reduces the generalization ability. The original problem for LSSVR is

$$\begin{array}{cc}\hfill & \underset{\omega ,b}{min}\frac{1}{2}{\parallel \omega \parallel }^2+\frac{1}{2}\gamma \sum _{i=1}^n{\epsilon }_i^2,\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.y_i={\omega }^T\varphi \left(x_i\right)+b+{\epsilon }_i.\hfill \end{array}$$

(12)

The Lagrangian function of least-squares support vector regression is £º

$$\begin{array}{cc}\hfill & L\left(\omega ,b,{\epsilon }_i,{\alpha }_i\right)=\frac{1}{2}{\parallel \omega \parallel }^2+\frac{1}{2}\gamma \sum _{i=1}^n{\epsilon }_i^2-\sum _{i=1}^n{\alpha }_i\left({\omega }^T\varphi \left(x_i\right)+b+{\epsilon }_i-y_i\right),\hfill \\ \hfill & \left[\begin{array}{cccc}I& 0& 0& -\varphi \\ 0& 0& 0& 1^T\\ 0& 0& \gamma I& -I\\ \varphi & 1& I& 0\end{array}\right]·\left[\begin{array}{c}\omega \\ b\\ {\epsilon }_i\\ {\alpha }_i\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ 0\\ y\end{array}\right],\hfill \\ \hfill & \left[\begin{array}{cc}0& 1^T\\ 1& k\left(x,x^T\right)+{\gamma }^{-1}I\end{array}\right]\left[\begin{array}{c}b\\ {\alpha }_i\end{array}\right]=\left[\begin{array}{c}0\hfill \\ y\hfill \end{array}\right],\hfill \end{array}$$

(13)

where $k(x_i,x_j^T)$ is the kernel function, and the specific expression is as follows:

$$k\left(x_i,x_j^T\right)=\phi {\left(x_i\right)}^T\phi \left(x_j\right).$$

(14)

The final least-squares support vector regression model is:

$$f_{LSSVR}\left(x\right)=\sum _{i=1}^n{\alpha }_{i}k\left(x_i,x_j^T\right)+b.$$

(15)

Twin support vector regression is shown in Figure 6.

On the basis of SVR, TWSVR creatively transformed the original hyperplane into two nonparallel hyperplanes and constrained the upper and lower hyperplanes of [32]. The final prediction result is the average of the prediction results of the two hyperplanes. Figure 6 shows that the TWSVR algorithm decomposes an original quadratic programming problem into two smaller quadratic programming problems. The computational speed and generalization ability of TWSVR were effectively improved. Experimental results show that the computational efficiency of TWSVR is four times that of SVR. The original problem for TWSVR is as follows:

$$\begin{array}{cc}\hfill \left(\mathrm{TWSVR}1\right)& \underset{{\omega }_1,b_1,{\xi }_1}{min}{C}_1{e}^T{\xi }_1+\frac{C_3}{2}\left({∥{\omega }_1∥}_2+b_1^2\right)+\frac{1}{2}\parallel \left(Y+e{\epsilon }_2-\left(A{\omega }_1+e{b}_1\right){\parallel }_2\right.\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\left(A{\omega }_1+e{b}_1\right)-\left(Y-e{\epsilon }_1\right)\le {\xi }_1,\hfill \\ \hfill & {\xi }_1\ge 0.\hfill \\ \hfill \left(\mathrm{TWSVR}2\right)& \underset{{\omega }_2,b_2,{\xi }_2}{min}{C}_2{e}^T{\xi }_2+\frac{C_4}{2}\left({∥{\omega }_2∥}_2+b_2^2\right)+\frac{1}{2}\parallel \left(Y+e{\epsilon }_1-\left(A{\omega }_2+e{b}_2\right){\parallel }_2\right.\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\left(Y-e{\epsilon }_2\right)-\left(A{\omega }_2+e{b}_2\right)\le {\xi }_2,\hfill \\ \hfill & {\xi }_2\ge 0.\hfill \end{array}$$

(16)

The Lagrangian function is as follows:

$$\begin{array}{cc}\hfill & L\left({\omega }_1,b_1,{\xi }_1,\alpha ,\beta \right)=C_1{e}^T{\xi }_1+{\alpha }^T\left(\left(A{\omega }_1+e{b}_1\right)-\left(Y-e{\epsilon }_1\right)-{\xi }_1\right)\hfill \\ \hfill & +\frac{1}{2}\left({\left(Y+e{\epsilon }_2-\left(A{\omega }_1+e{b}_1\right)\right)}^T\times \left(Y+e{\epsilon }_2-\left(A{\omega }_1+e{b}_1\right)\right)\right)\hfill \\ \hfill & +\frac{C_3}{2}\left({\omega }_1^T{\omega }_1+b_1^2\right)-{\beta }^T{\xi }_1.\hfill \end{array}$$

(17)

The KKT conditions are as follows:

$$\left\{\begin{array}{cc}\hfill C_3{\omega }_1-A^T\left(Y+e{\epsilon }_2-\left(A{\omega }_1+e{b}_1\right)\right)+A^T\alpha & =0,\hfill \\ \hfill C_3{b}_1-e^T\left(Y+e{\epsilon }_2-\left(A{\omega }_1+e{b}_1\right)\right)+e^T\alpha & =0,\hfill \\ \hfill C_{1}e-\alpha -\beta & =0\hfill \\ \hfill \left(A{\omega }_1+e{b}_1-\left(Y-e{\epsilon }_1\right)\right)\le {\xi }_1,{\xi }_1& \ge 0,\hfill \\ \hfill \left.{\alpha }^T\left(\left(A{\omega }_1+e{b}_1\right)-\left(Y-e{\epsilon }_1\right)-{\xi }_1\right)\right)=0,{\beta }^T{\xi }_1& =0,\hfill \\ \hfill \alpha \ge 0,\beta & \ge 0.\hfill \end{array}\right.$$

(18)

The final twin support vector regression model is as follows:

$$\begin{array}{c}\hfill f_{TWSVR}=\frac{1}{2}({({\omega }_1+{\omega }_2)}^{T}x+(b_1+b_2).\end{array}$$

(19)

3.1.3. Optimization Model

In this study, two optimization algorithms, namely, WOA and PSO, were selected to optimize the prediction. Both algorithms are fast and effective community search algorithms and have solutions for falling into local optima, which are suitable for this study.

(1)

Whale optimization algorithm

Algorithm 1 General framework of the WOA algorithm.

Require: Fitness function, Whale swarm $\Omega$. Ensure: Global optimal solution. 1: Initialize the parameters. 2: Initialize the whale position 3: Calculate the fitness value of each whale. 4: while termination condition does not satisfy do 5:    for $i=1$ to $|\Omega |$ do 6:      The “best and closest” whale Y sought. 7:      if Y exists then 8:         Move according to the formula under the guidance of Y. 9:         Review ${\Omega }_i$. 10:      end if 11:    end for 12: end while 13: Return the global optimal solution.

When the whales are hunting, each whale behaves in two ways. During each iteration, each whale randomly chooses one of two behaviors to hunt. One behavior is where all whales move toward each other to surround prey. Another behavior is when whales swim in circles and blow bubbles to repel prey. Each whale follows two rules of behavior until the best solution is found. The rules for both behaviors are as follows.

①

Surrounding prey behavior

The search scope of the whale algorithm is the global solution space. Since the position of the optimal solution in the search range is not known a priori, the WOA assumes that the current best candidate solution is the target prey or near-optimal solution. After defining the best search agent, other search agents will attempt to update their locations to that of the best search agent. This phase is defined as surrounding prey behavior.

$$\begin{array}{cc}\hfill & A\left(t\right)=2r_{1}a\left(t\right)-a\left(t\right),\hfill \\ \hfill & C\left(t\right)=2r_2,\hfill \\ \hfill & a\left(t\right)=2-2\frac{t}{t_{max}},\hfill \end{array}$$

(20)

where a is the convergence factor. As the number of iterations decreases linearly from 2 to 0, the random number between [0, 1] is taken modulo of $r_1$ and $r_2$. With respect to the hunt, the behavior of whales rounding up prey is defined as follows:

$$\begin{array}{cc}\hfill & \overrightarrow{D}\left(t\right)=\left|C\left(t\right)·\overrightarrow{X_p}\left(t\right)-\overrightarrow{X}\left(t\right)\right|,\hfill \\ \hfill & \overrightarrow{X}(t+1)=\overrightarrow{X_p}\left(t\right)-A·\overrightarrow{D}\left(t\right).\hfill \end{array}$$

(21)

Formula (20) represents the distance between the individual and the prey, and Formula (21) is the position-updating formula of the whale, where t is the current iteration algebra; a and c are the coefficient vectors; and $X_p$ and X are the position vectors of the prey and gray wolf, respectively. The surrounding prey behavior of the schematic diagram of the whale optimization algorithm is shown in Figure 7a.

②

Spiral bubble hunting behavior

Another hunting behavior of whales is the creation of a spiral bubble net. The individual calculates the distance between himself and the best individual in the current population, and then humpback whales pump out bubbles as they spiral to seal off prey. The position update between the humpback whale and prey is expressed by the logarithmic spiral equation.

$$\begin{array}{cc}\hfill & \overrightarrow{D^{\prime }}\left(t\right)=\left|\overrightarrow{X_p}\left(t\right)-\overrightarrow{X}\left(t\right)\right|,\hfill \\ \hfill & \overrightarrow{X}(t+1)={\overrightarrow{D}}^{\prime }\left(t\right)·e^{bl}·cos\left(2\pi l\right)+\overrightarrow{X_p}\left(t\right),\hfill \end{array}$$

(22)

where b is the constant that defines the shape of a logarithmic spiral and l is a random number in [−1, 1]. The spiral-bubble hunting behavior of schematic diagram of the whale optimization algorithm is shown in Figure 7b.

(2)

Particle swarm optimization algorithm

The next direction and position of each particle is indicated by its velocity (v). The velocity of particles is affected in the following three ways: the motion state of the particle itself, the direction and distance between the particle and the optimal solution, and the position information of other particles. The velocity iteration formula is:

$$v_{id}^{k+1}=\omega v_{id}^k+c_1{r}_1\left(p_{id,pbest}^k-x_{id}^k\right)+c_2{r}_2\left(p_{d,gbest}^k-x_{id}^k\right),$$

(23)

where k is the number of iterations, $\omega$ is the inertia weight, $c_1$ is the individual learning factor, and $c_2$ is the group learning factor. $r_1$ and $r_2$ are random numbers for the search between [0, 1]. $v_{id}^k$ is the velocity vector of particle i in dimension d in the KTH iteration, and $x_{id}^k$ is the position vector of particle i in dimension d in the KTH iteration. $p_{id,pbest}^k$ represents the historical optimal position of particle i in dimension d of the KTH iteration, and $p_{d,gbest}^k$ represents the historical optimal position of the population in dimension d of the KTH iteration.

4. Results and Discussion

4.1. Experiment Settings

To evaluate the accuracy and reliability of the prediction of each model in this study, EVS, $R^2$, MAE, MSE, RMSE, and MAPE were used to measure the relationship between the actual Bitcoin price and the predicted Bitcoin price. At the same time, the CPU time was used to measure the running speed of each mixed model. The six measures of accuracy are defined as follows:

$$\begin{array}{cc}\hfill & \mathrm{EVS}=1-\frac{Var(y-\widehat{y})}{Var\left(y\right)},\hfill \\ \hfill & R^2=1-\frac{SSE}{SST},\hfill \\ \hfill & MAE=\frac{1}{n}\sum _{i=1}^n\left|{\widehat{y}}_i-y_i\right|,\hfill \\ \hfill & MSE=\frac{1}{n}\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2,\hfill \\ \hfill & \mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2},\hfill \\ \hfill & MAPE=\frac{100\%}{n}\sum _{i=1}^n\left|\frac{{\widehat{y}}_i-y_i}{y_i}\right|,\hfill \end{array}$$

(24)

where $\widehat{y_i}$ is the predicted value, $y_i$ is the real price of Bitcoin at time ith, and n is the number of predictions. EVS and $R^2$ are the numbers of [0, 1], and the larger the value is, the better the prediction effect of the model. MAE, MSE, RMSE, and MAPE are all methods used to calculate the prediction error, and the smaller the value is, the better the prediction effect of the model. To remove the error of a single experiment, the CPU running time is exactly the 100th time. The computer configuration used in the experiment was as follows. Processor: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80 GHz 2.80 GHz. Belt: RAM 8.00 GB (7.86 GB). Computational speed was limited by Python 3.9.

4.2. Feature Selection

In this study, the XGBoost algorithm and random forest algorithm are used to extract the main factors affecting the price fluctuation of Bitcoin. Both the XGBoost algorithm and the random forest algorithm are effective machine learning methods for variable extraction. The accuracy and scientificity of extracting variables are improved by comparing the two methods. The main factors extracted by the XGBoost algorithm and random forest algorithm replace the original factors to make the prediction of the model more accurate by subsequent modeling and analysis. The screening results of the two models are represented by a bar chart and nightingale rose chart, in which the column length of the bar chart and the size of the rose petal in the rose chart both represent the importance of each variable. The bar chart better shows the difference in the importance of the same variable in different models, as shown in Figure 8. The nightingale rose chart better shows the difference in importance of different variables in the same model in Figure 9.

(a)

XGBoost screened out six important factors in Figure 8. The S&P 500 index is the most important. Moreover, the ETH, LTC, MD, F&G, and GOLD indices are also very important. The six important factors are the supply factor of Bitcoin, the demand factor of Bitcoin, the price factor of other cryptocurrencies, the financial-indicator factor, and the macroeconomic factor, but the exchange rate factor was not included. Eleven important factors were screened by the random forest algorithm, focusing on internal cryptocurrency factors and U.S. stock market factors. The importance of the S&P500 indices screened by the random forest algorithm is significantly higher than that of the other indices. These are the BCH, DJIA, ETH, F&G, GOLD, HSCEI, LTC, MD, NASDAQ, and WTI indicators. Compared with the important indicators extracted by the two methods, the indicators extracted by the XGBoost algorithm are more important and representative. The eleven important factors screened by the random forest algorithm include six important factors screened by the XGBoost algorithm.

(b)

From the perspective of economics, the internal factors of cryptocurrency and the U.S. stock market are the most important factors affecting the price of Bitcoin. The internal factors of cryptocurrency directly affect the price of Bitcoin from the perspective of supply and demand. Bitcoin is also popular in Western countries, so its volatile price is indirectly influenced by the U.S. stock market.

4.3. Combined Model Prediction

First, three models, SVR, LSSVR, and TWSVR, were selected to forecast the Bitcoin price, and the ARIMA model was selected as the benchmark model. At the same time, the prediction model and feature screening model were combined to obtain the best prediction effect. We assume that the regularization factor satisfies the conditions $c_1=c_2=c_3=c_4$, and the RBF is selected as the kernel function in the prediction model.

Second, the parameters of the combined model were debugged by the WOA and PSO, and the prediction index EVS of the combined model was taken as the objective function of the optimization algorithm. Then, six combined prediction models (WOA-SVR, WOA-LSSVR, WOA-TWSVR, PSO-SVR, PSO-LSSVR, and PSO-TWSVR) were obtained.

Finally, we used EVS, $R^2$, MAE, MSE, RMSE, MAPE, and the CPU running time to evaluate and compare the effects of each combination model, as shown in

Table 1. Performance indexes of each combination model.

CATEGORY	MODEL	EVS	${\mathit{R}}^2$	MAE	MSE	RMSE	MAPE	CPU Time (s)
Complete Features	ARIMA	0.723	0.691	0.091	0.008	0.091	0.132	1.173
	WOA-SVR	0.9022	0.879	0.042	0.003	0.055	0.0602	2.5631
	WOA-LSSVR	0.8868	0.869	0.043	0.003	0.057	0.0622	0.5268
	WOA-TWSVR	0.9484	0.923	0.032	0.002	0.044	0.0442	0.6736
	PSO-SVR	0.8964	0.876	0.042	0.003	0.056	0.0614	2.6417
	PSO-LSSVR	0.8835	0.867	0.043	0.003	0.057	0.0625	0.5416
	PSO-TWSVR	0.9484	0.923	0.032	0.002	0.044	0.0442	0.6736
XGBoost Features	WOA-SVR	0.9161	0.894	0.036	0.002	0.049	0.0512	1.9301
	WOA-LSSVR	0.9041	0.887	0.042	0.003	0.054	0.0594	0.3671
	WOA-TWSVR	0.9547	0.929	0.031	0.002	0.042	0.0433	0.4743
	PSO-SVR	0.9074	0.89	0.041	0.003	0.052	0.0571	1.9319
	PSO-LSSVR	0.9041	0.887	0.042	0.003	0.054	0.0594	0.3671
	PSO-TWSVR	0.9239	0.896	0.035	0.002	0.046	0.0475	0.4876
RF Features	WOA-SVR	0.8067	0.78	0.054	0.005	0.072	0.0786	2.3297
	WOA-LSSVR	0.9015	0.881	0.042	0.003	0.055	0.0607	0.3847
	WOA-TWSVR	0.9502	0.919	0.031	0.002	0.042	0.0438	0.5113
	PSO-SVR	0.8041	0.771	0.055	0.005	0.073	0.0792	2.2744
	PSO-LSSVR	0.8994	0.878	0.042	0.003	0.055	0.0611	0.4228
	PSO-TWSVR	0.9491	0.924	0.032	0.002	0.044	0.0441	0.4943

.

It can be seen in Table 1 that the prediction effect of the combined prediction model under the condition of “XGBoost feature” was better than those of the “full feature” and “random forest feature”. In this study, six variables selected by the XGBoost algorithm, S&P 500, ETH, F&G, GOLD, LTC, and MD, were taken as the final variables of SVR and its variant model. In addition, the TWSVR model has a better prediction effect than the other two models according to EVS, $R^2$, MAE, MSE, RMSE, and MAPE. As shown in Figure 10, the column height of the histogram represents the EVS index size of each combination model. The blue columns represent the full features, the red columns represent the random forest features, and the green columns represent the XGBoost features.

Other than PSO-TWSVR, other hybrid models based on the “XGBoost feature” were better than those based on the “full feature” and “random forest feature”. In the SVR, the prediction effect of the model based on “random forest characteristics” was worse than that of the model based on “full characteristics”. With both LSSVR and TWSVR, the predictions of the models based on “random forest characteristics” were better than those of the models based on “full characteristics”. In addition, among all models, PSO-TWSVR and WOA-TWSVR exhibited the best prediction effect. The optimal EVS value of the WOA-SVR model was up to 0.9547, and that of the PSO-SVR model was up to 0.9491.

Among the three prediction models, LSSVR runs the fastest, followed by TWSVR. The slowest is the SVR model. When “full feature” was adopted, in Table 1, we can see that the running time of TWSVR was improved from the 2.5631–2.6417 s of the SVR model to 0.6736 s by parameter tuning of the WOA and PSO, and that the running time was reduced by 1.8895 to 1.9681 s. The running times of the SVR model were 2.5631 and 2.6417 s and the running times of LSSVR were reduced to 0.5268 and 0.5416 s. When the “random forest feature” was adopted, the running time of TWSVR was reduced by 1.4558 or 1.4443 s through parameter tuning of the WOA and PSO. The LSSVR runtime was reduced by 1.563 and 1.5648 s. When using the “XGBoost feature”, the TWSVR running time is reduced by 1.8181 and 1.7801 s through parameter tuning of the WOA and PSO. The LSSVR runtime is reduced by 1.945 and 1.7801 s.

In conclusion, the TWSVR model is superior to the SVR model in both prediction effect and calculation speed, which proves the superiority of this variant model. TWSVR is slightly slower than the LSSVR, but its prediction is much better.

Figure 11 shows the prediction results of three prediction models: SVR, LSSVR, and TWSVR. The red solid line represents the true Bitcoin price, the purple dashed line represents the SVR model’s prediction, the deep pink dotted line represents the LSSVR model’s prediction, and the green dot-dash line represents the TWSVR model’s prediction. The optimal combination model of three prediction curves was selected, that is, the condition of XGBoost feature and parameter optimization with WOA. Through the image in Figure 11, we can clearly see the difference between the predicted results and the real values. The TWSVR model’s predictions were closer to the real prices of Bitcoin. The four curves showed slight differences in the first 70 stages and great differences in the 85–95, 120–125, 140–165, and 180–200 stages. The reason for this is that the SVR and its variant models do not fit the data with large fluctuations well.

Figure 12 shows the fitness curves of the SVR, LSSVR, and TWSVR models using the WOA and PSO for parameter tuning. The objective function of the optimization is EVS. The WOA and PSO are both excellent optimization algorithms, and both optimize the model within 100 iterations. From the image in Figure 12, the WOA is faster than the PSO. Compared with PSO, the WOA more easily jumps out of the local optimum and finds the overall optimum. In this study, the WOA is superior to PSO with respect to the parameter optimization of the prediction model.

Figure 13 shows the EVS of the SVR and its variants, as the explained variance score (EVS) varies with the regularization factor (C) and kernel RBF bandwidth ($\gamma$) based on XGBoost variable 6. Since the variation in the fitting effect of the model is more sensitive when the parameter range is [0, 1], we chose the selected range to be $2^{-9}$ to $2^{10}$. The z-axis coordinate and surface color represent the fitting effect of the model, and the larger the value is, and the brighter the color is, the better the fitting effect of the model. In other words, the TWSVR model has a better fitting effect than the SVR and LSSVR models. In Figure 13a,c, the SVR model and the TWSVR model are more affected by the regularization factor than the kernel RBF bandwidth. The fitting effect of the SVR model is at a higher level when the regularization factor is less than $2^{-1}$ and reaches a peak when the regularization factor is equal to $2^{-5}$. When the regularization factor is greater than $2^{-1}$, the fitting effect of the SVR model is low, even reaching zero. The EVS values of the TWSVR models were all greater than 0.4, and the overall fitting effect was better. The EVS value reached its peak when the regularization factor equals $2^1$ and the EVS value is higher than 0.95. The fitting effect of the LSSVR model is greatly affected by the regularization factor and kernel RBF bandwidth. In Figure 13c, the effect of hills with high middle and low four corners is shown. The EVS value of LSSVR reaches the peak value when the regularization factor is equal to $2^{-1}$ and the regularization factor is equal to $2^{-1}$, and the peak value is 0.90.

5. Conclusions

Bitcoin has been the most successful and popular cryptocurrency in recent years. In this study, XGBoost and random forest algorithms were used to screen the influencing factors. Six important factor groups were obtained via XGBoost, and eleven important factor groups were obtained via random forest. The main prediction models used in this study were SVR, LSSVR, and TWSVR. Complete features (27), XGBoost features (11), and random forest features (6) were substituted into the prediction model; and the WOA and PSO were used to optimize and establish a mixed model including data screening, optimization, and prediction. Finally, we used EVS, $R^2$, MAE, MSE, RMSE, and MAPE to measure the prediction effect of the mixed model. Meanwhile, the CPU time was used to measure the running speed of the hybrid model. The following conclusions were reached:

The combined model, XGBoost-WOA-TWSVR, demonstrated the best prediction effect, and the EVS score reached 0.9547. Our model can effectively predict the price of Bitcoin. In terms of computing speed, TWSVR’s computing speed was not the fastest, but it was not significantly different from the fastest LSSVR model. Through feature screening of the XGBoost and random forest algorithms and parameter tuning of the WOA and PSO, the accuracy of the model was significantly improved.

SVR, LSSVR, and TWSVR were all effective prediction algorithms with significantly improved accuracy and speed compared to the benchmark model. During the Bitcoin price prediction experiment, we found that LSSVR is greatly improved in computing speed compared with the traditional SVR model, but it is somewhat less accurate. Compared with the SVR model, TWSVR improved not only the calculation speed of the model, but also accuracy, and it was approximately four times faster than SVR.