4.1.2. Neural Network Autoregression (NNAR)

Artificial neural network (ANN) methods rely on mathematical models in a similar pattern as 'neurons' in the brain. ANN models help design complex non-linear associations between the dependent variable and its predictors (Adya and Collopy 1998; Hyndman and Athanasopoulos 2018). The simplest ANN models would only have predictors (independent variables or inputs) in the bottom layer and the dependent variable (output) in the top layer, which would be equivalent to a linear regression model. After adding the hidden layer(s) in-between bottom and top layers, the ANN structure becomes non-linear. A sample ANN model is depicted in Figure 2. This type of ANN is called multi-layered feed-forward network, where each layer of neurons (nodes) receive inputs from the previous layer. The inputs to each node are estimated using a weighted linear combination, as in Equation (2):

*n*

*zj* = β*j* + *i*=1 *Wi*,*jXi* (2)

**Figure 2.** An ANN model with four inputs and one hidden layer with three neurons.

Here, *zj* is the value of output node *j*, β*j* is the constant for node *j*, *Wi*,*<sup>j</sup>* is the weight from the input node *i* to output node *j*, *Xi* represents the inputs, and *n* is number of input variables. In the hidden layer, Equation (2) is transformed into non-linear function using sigmoid, as shown in Equation (3).

$$s(z) = \frac{1}{1 + \varepsilon^{-z}} \tag{3}$$

The parameters β1, β2, β3, ... , β*n* and *W*1,1, ... , *W*4,3 are "learned" from the training data. To prevent the weights from becoming too large, usually, the values of the weights are restricted. Decay parameter—the parameter that restricts the weights is typically set to be equal to 0.1 (Hyndman and Athanasopoulos 2018). With time series data such as daily Bitcoin price, lagged values of the time series can be used as inputs in an ANN structure, which is known as neural network autoregression (NNAR). A non-seasonal feed-forward network model with one hidden layer is usually denoted as NNAR (*p,k*), where p represents the number of lags and *k* represents the number of nodes in the hidden layer.
