Narx Neural Networks Models for Prediction of Standardized Precipitation Index in Central Mexico

Abstract

Some of the effects of climate change may be related to a change in patterns of rainfall intensity or scarcity. Therefore, humanity is facing environmental challenges due to an increase in the occurrence and intensity of droughts. The forecast of droughts can be of great help when trying to reduce the adverse effects that the scarcity of water brings, particularly in agriculture. When evaluating the conditions of water scarcity, as well as in the identification and characterization of droughts, the use of predictive models of drought indices could be a very useful tool. In this research, the utility of Artificial Neural Networks with exogenous inputs was tested, with the aim of predicting the monthly Standardized Precipitation Index in 4 regions (Semi-desert, Highlands, Canyons and Mountains) of north-central México using predictor data from 1979 to 2014. The best model was found using the scaled conjugate gradient backpropagation algorithm as the optimization method and was set to the following architecture: 6-25-1 network. The correlation coefficient of predicted and observed Standardized Precipitation Index values for the test dataset was between 0.84 and 0.95. As a result, the Artificial Neural Network models performed successfully in predicting Standardized Precipitation Index at the four analyzed regions. The developed and tested Artificial Neural Network models in this research suggest remarkable prediction abilities of the monthly Standardized Precipitation Index in the study region.

1. Introduction

The change in rainfall patterns is a fundamental research topic because human activities are highly vulnerable to extreme events of either excess or scarcity of rainfall [1,2]. Meteorological droughts occur when the observed precipitation is less than the long-term precipitation average [3]. When droughts are prolonged, they have a direct effect on the reduction of freshwater flows, which brings, consequently, changes in the administration and planning of hydraulic resources, particularly in vulnerable areas to water scarcity such as agriculture [4].

For the assessment of droughts, several tools have been developed. Among those tools, drought indices are one of the most used. McKee et al. [5] developed the Standardized Precipitation Index (SPI) with the purpose of classifying the observed rainfall as a deviation from a rainfall probability distribution function. In recent years, this index has been used as a tool for classifying climatic zones and as drought indicator, allowing comparisons across space and time [6]. One of the benefits of this drought index is its ease of calculation since it only uses rainfall time series records [7]. Giddings et al. [8] used the SPI to develop consistent precipitation zones in whole Mexico. Nonetheless it is possible to define smaller zones or regions with this index [8]. Toward this purpose, the SPI was used by Magallanes-Quintanar et al. [9] to cluster monthly time series belonging to Mexico’s Zacatecas state into groups of similar precipitation regimes, with the aim to compute regional SPI’s, and to estimate trends of the regional SPI’s. In this context, current knowledge on SPI trends suggests that less than the historical average precipitation record occurs in the geographical area of Zacatecas state [9]. However, SPI forecasts toward the near future remain practically unknown. This kind of information could be essential to adjusting the activities of its inhabitants based on planned adaptation strategies designed in a context of water scarcity.

Besides drought assessment, drought forecasting models have been developed, too. Particularly, the use of artificial intelligence (AI) models shows a good performance and accuracy in drought forecasting. When analyzing hydrological data, machine learning methods (an application of AI) have recently proven their effectiveness as they have become more skillful and accurate as well as easier to use [10]. Remarkably, neural networks are an information processing approach that acquires network knowledge through a learning process [4]. When analyzing data on water resources and hydrology (among other fields), neural networks have been used successfully to model and forecast nonlinear time series. As a result, the use of Artificial Neural Network (ANN) models has been a useful data-driven tool for forecasting monthly SPI [3,4,10,11,12,13,14,15]. In some cases, the ANN models were trained using synoptic-scale climate drivers, including Sea Surface Temperatures such as the El Niño Southern Oscillation Index, which are known to influence the variability of both precipitation and temperature predictors [14].

As a summary, the SPI has been used in several regions of the world toward drought assessment and forecasting. However, its application in Mexico by means of neural networks for forecasting purposes remains unstudied. In this study, we applied a nonlinear autoregressive Artificial Neural Network with exogenous inputs as a data-driven model which aimed: (a) to predict regional Standardized Precipitation Index using hydro-meteorological datasets as predictors from 1979 to 2014 and a climate index; (b) to evaluate the models’ performance using key statistical parameters and (c) to analyze the models’ prediction error yield over the validation period.

2. Materials and Methods

2.1. Data

A set of 5 input (or predictor) variables and one covariate were used. Those predictors described the geographic and climatic attributes of weather stations situated in Zacatecas state within the Mexican territory (Figure 1). The site-specific inputs were the station ID, year and month, rainfall, evaporation, maximum temperature and mean temperature. In addition, the Multivariate El Niño Southern Oscillation Index v.2 (MEI.v2) was incorporated as a regression covariate for training the ANN models and the evapotranspiration predictor was assessed using the Thornthwaite [16] method.

Considering that available records of the MEI database begin in the year 1979, the records of predictors or variables of interest should include weather stations with at least 35 years of complete records. For this reason, a total of 25 weather stations were included in this study with records within a time period from 1979 to 2014. The database of input variables was a long–term meteorological dataset (35 years, 1979–2014) supplied by the Mexican ‘Comisión Nacional del Agua’ and includes records of regional weather stations. The database was scrutinized to avoid the presence of outliers or missing data.

2.2. Standardized Precipitation Index

McKee et al. [5] defined the Standardized Precipitation Index (SPI) as the number of standard deviations that cumulative rainfall differs from the historical climatological average and has been used as a tool for monitoring and defining drought. As described by Koudahe et al. [17], the SPI value can be calculated as follows:

A gamma distribution is used to fit the monthly rainfall data, and its probability density function is defined as:

$$g\left(x\right)=\frac{1}{{\beta }^{\alpha }\Gamma \left(\alpha \right)}{x}^{\alpha -1}{e}^{\frac{-x}{\beta }}$$

(1)

for x > 0.

Where the probability density function is given by g(x), α is the shape parameter (α > 0), β is a scale parameter (β > 0), and

$$\Gamma \left(\alpha \right)=\underset{0}{\overset{\alpha }{{\displaystyle \int }}}{y}^{\alpha -1}{e}^{-1}dy.$$

(2)

where Γ(α) is the gamma function.

The parameters α and β are assessed as follows:

$$\alpha =\frac{1}{4A}\left(1+\sqrt{1+\frac{4A}{3}}\right)$$

(3)

$$\beta =\frac{x}{a}$$

(4)

$$A=ln\left(\overline{x}\right)-\frac{{\displaystyle \sum }ln\left(\overline{x}\right)}{n}$$

(5)

where n is the number of precipitation observations and $\overline{x}$ is the arithmetic mean over the analyzed time scale. A cumulative probability G(x) of an observed quantity of rainfall in each month and time scale (if α and β estimators were used to integrate the probability density function with respect to x) is computed as follows:

$$G\left(x\right)={{\displaystyle \int }}_0^{x}g\left(x\right)dx=\frac{1}{{\overline{\beta }}^{\overline{\alpha }}\Gamma \left(\overline{\alpha }\right)}{{\displaystyle \int }}_0^x{x}^{\overline{\alpha }}{e}^{\frac{\overline{x}}{\overline{\beta }}}dx.$$

(6)

The incomplete gamma function is obtained when substituting t for $\frac{\overline{x}}{\overline{\beta }}$ in the previous equation:

$$G\left(x\right)=\frac{1}{\Gamma \left(\overline{\alpha }\right)}{{\displaystyle \int }}_0^x{t}^{\overline{\alpha }-1}{e}^{-1}dt$$

(7)

However, the gamma distribution function is undefined for x = 0 and q = P(x = 0) > 0; where P(x = 0) is the probability of zero precipitation. Therefore, the actual probability of non-exceedance H(x) should be computed as follows [18,19]:

$$H\left(x\right)=q+\left(1-q\right)G\left(x\right).$$

(8)

where q is the probability of x = 0. If m is zero in a sample of size n, then q is calculated as

$$q=\frac{m}{n}$$

(9)

Finally, to compute the SPI, the cumulative probability distribution H(x) is standardized to a normal variable Z ~ N(0,1). Taking into account the SPI yielded results, the classification of wet or drought periods is shown in

Table 1. Classification of standardized precipitation index values.

SPI Value	Class
≥2.0	Extremely wet
1.5 to 1.99	Severely wet
1.0 to 1.49	Moderately wet
−0.99 to 0.99	Near normal
−1.49 to −0.99	Moderately dry
−1.99 to −1.49	Severely dry
≤2.0	Extremely dry

as stated by McKee et al. [5].

Since changes in precipitation have influence in many aspects of the hydrologic cycle, several time scales are used for SPI computation [17]. As pointed out by Caloiero [18], whereas the 3-month SPI value describes short- and medium-term moisture, the 6-month SPI value describes droughts that affect agriculture, and the 12-month SPI describes droughts affecting water supply reservoir levels. In this research, we computed the SPI values at timescales of 12 months.

Due to the arduousness of manually calculating SPI values, various computer programs have been developed for this purpose. In this study we used the ‘SPEI’ [19] package for SPI computation. The ‘SPEI’ package is a library of R system 4.1.1 [20]. Both system and packages are available at the Comprehensive R Archive Network (https://www.cran.r-project.org/ accessed on 30 May 2022).

2.3. Cluster Analysis

Cluster analysis is a multivariate statistical technique that involves group elements (or variables) trying to achieve the maximum homogeneity in each group and the greatest difference between the groups [21]. Cluster analysis has been proven to be a statistical tool capable of defining homogeneous climatic regions using observed values of meteorological variables [21,22]. In this study, the Canberra distance was used as the linkage criteria and the Ward’s method was used as the linkage rule of a tree clustering algorithm with the aim of grouping the whole set of 25 monthly SPI time series corresponding to the 25 weather stations into regions under the basis of similar SPI values.

The result of using the clustering technique was a set of four regions (Semi-desert, Highlands, Mountains and Canyons) based on the similarity of their SPI values.

In this study, we used the ‘hclust’ [20] and ‘ape’ [23] packages or libraries for cluster computation under R system 4.1.1 [20].

2.4. Potential Evapotranspiration Index

The potential evapotranspiration (PET) predictor or variable was estimated using the rainfall and temperature data for each weather station. PET estimates the amount of evaporation and transpiration by a specific crop or ecosystem, it is the theoretical maximum amount of water that can evaporate from a ground completely covered by vegetation and constantly supplied with water [24]. There are several PET estimation techniques depending on data availability. Since monthly rainfall and temperature data were available, the classical Thornthwaite [16] method was used as described by Vicente-Serrano et al. [25]:

$$\mathrm{PET}=16K{\left(\frac{10T}{I}\right)}^m,$$

(10)

where I is a heat index computed as the sum of 12 monthly index values i, the latter resulting from mean monthly temperature T (°C) and m is deduced empirically (m = 6.75 × 10⁻⁵, I³ + 7.75 × 10⁻⁷, I² + 1.79 × 10⁻², I + 0.492),

$$I=16K{\left(\frac{T}{5}\right)}^{1.514}$$

(11)

K is a correction coefficient estimated as a function of the latitude and month

$$K=\left(\frac{N}{12}\right)\left(\frac{NDM}{30}\right).$$

(12)

NDM is the sum of days of the current month and N is the maximum number of sunlight hours computed using

$$N=16K\left(\frac{24}{\pi }\right){\omega }_S$$

(13)

and ${\omega }_S$ = hourly angle of sun rising (${\omega }_S=\arccos \left(-\tan \left(\varphi \right)\tan \left(\delta \right)\right)$), $\varphi$ = latitude in radians, $\delta =0.4093\sin \left(\frac{2\pi J}{365}-1.405\right)$ is the solar declination in radians and J is the average Julian day of the month.

In this study the ‘SPEI’ [19] package or library was used for PET index computation under R system 4.1.1 [20]. Both ‘SPEI’ and R system are available at the Comprehensive R Archive Network (https://www.cran.r-project.org/ accessed on 30 May 2022)

2.5. Multivariate ENSO Index Data

It is well known that El Niño Southern Oscillation (ENSO) is one of the most important coupled ocean-atmosphere phenomena producing global climate variability on interannual time scales [26]. The works of Wolter [27] and Wolter and Timlin [26,28] yield a definition of the Multivariate ENSO Index (MEI) that is computed using the six main observed variables over the tropical Pacific. These variables are sea-level pressure, zonal and meridional components of the surface wind, sea surface temperature, surface air temperature and total cloudiness fraction of the sky. For this study we used a Multivariate ENSO Index Version 2 (MEI.v2) database (1979–2014) as regression covariate for training the ANN models.

MEI.v2 database was computed by the National Oceanic and Atmospheric Administration and is available at http://www.esrl.noaa.gov (accessed on 30 May 2022)

2.6. Neural Network Forecasting

The first approach of Artificial Neural Networks (ANN) was initially proposed by McCulloch and Pitts [29] to analyze problems with variables of a non-linear or stochastic nature. This was the first modern neural model, and it has been taken as the foundation for the development of recent neural models.

To forecast climatological time series including ANN models, several methods have been developed. An ANN is a computation paradigm that attempts to mimic the neural connections in the brain. In particular, the Multilayer Perceptron (MLP) Network has been the ANN approach frequently used when modeling hydrological data [4,30]. When modeling nonlinear time series, besides MLP, a Feed-forward neural network could be a more effective option than conventional methods [31].

When modeling nonlinear time series, the use of a recurrent dynamic network with feedback connections can constitute another ANN. This type of ANN is called a nonlinear autoregressive network with exogenous inputs (NARX). When modeling nonlinear systems (particularly time series), NARX networks have been demonstrated to be a powerful class of models well-suited for this type of problem [32].

Additionally, the use of NARX networks with gradient descent learning algorithms have achieved more effective learning than other types of neural networks (because gradient descent is better in NARX), and they converge faster and generalize better than other type of neural networks [33].

Following Liu et al. [34], the equation defining a NARX model is:

$$\widehat{y}\left(t\right)=f\left(y\left(t-1\right),y\left(t-2\right),\dots ,y\left(t-n_y\right),u\left(t-1\right),u\left(t-2\right),\dots ,u\left(t-n_u\right)\right)+e\left(t\right)$$

(14)

where the target and predicted output variables are, respectively, y(t) and ŷ(t). The next value of the dependent output signal ŷ(t) is regressed on previous values of the output signal and previous values of an independent (exogenous) input signal; u(t) is the input variable of the network; n_u and n_y are the time delays of the input and output variable, and e(t) is the model error between the target and predicted values.

The hidden layer output at time t is obtained as

$$H_i\left(t\right)=H_i\left(t\right)=f_1\left[{\displaystyle \sum }_{r=0}^{n_u}{w}_{ir}u\left(t-r\right)+{\displaystyle \sum }_{l=1}^{n_y}{w}_{il}y\left(t-l\right)+a_i\right]$$

(15)

The weight of the connection between the input neuron u(t − r) and the ith hidden neuron is defined as w_ir. The connection weight between the ith hidden neuron and output feedback neuron y(t − l) is w_il; the bias of the ith hidden neuron is a_i; the hidden layer activation function is f₁ and the hidden transfer and output function is f₂.

The figure below shows a two-layer forward network (Figure 2).

It is well known that all neural networks have at least one input layer and one output layer, although the number of hidden layers is variable [4]. Moreover, the inner structure of an ANN is hard to clarify. Therefore, the architecture of an ANN composed of the number of inputs, the number of hidden units and the structure of these layered units is often established by a trial-and-error approach.

The procedure for implementing an ANN consists of four parts:

(i)

Features or variable selection.

(ii)

Neural network learning by training, test and validation.

(iii)

Varying the structure or architecture.

(iv)

Model confirmation and forecasting.

In this study, we used Matlab R2021b Update 1 [35] and the Deep Learning toolbox 14.3 with the purpose of fitting a NARX Neural Network model to each of the 4 regional time series (Semi-desert, Highlands, Mountains and Canyons) of mean predictors: rainfall (PP), evaporation (EVP), maximum temperature (TMAX), mean temperature (TMED), evapotranspiration (PET) and MEI. v.2 belonging to each region to forecast the SPI index values in each region. Since the model aims to predict the SPI values and SPI is normalized already, no further normalization or standardization was needed.

The final data to train each regional neural network was a matrix of 432 timesteps or months x6 predictors and a vector of 432 timesteps or months x1 response variable. The input predictors to train the models are shown in

Table 2. Summary of input predictors by region used in the ANN models.

Region
Semi-Arid	SPI	PP (mm)	EVP (mm)	TMED (°C)	TMAX (°C)	PET (mm)
Min	−1.9577	0.0000	79.9297	10.1753	21.7807	24.6523
µ	−0.0685	34.2184	163.9863	16.7953	29.3890	65.0969
Max	1.9913	282.3329	304.6525	22.4452	37.0236	116.3594
σ	0.6950	38.2306	47.1019	3.1163	2.6660	23.8464
Highlands	SPI	PP (mm)	EVP (mm)	TMED (°C)	TMAX (°C)	PET (mm)
Min	−1.9803	0.0000	83.3820	10.1441	22.5070	20.2253
µ	−0.0671	37.2929	172.0484	16.7851	29.4178	69.3710
Max	2.0847	263.7820	311.1621	25.2700	36.0990	121.6185
σ	0.7729	42.7967	51.0985	3.2487	2.7512	27.0455
Mountains	SPI	PP (mm)	EVP (mm)	TMED (°C)	TMAX (°C)	PET (mm)
Min	−2.4941	0.0000	66.7167	12.0667	18.6623	24.7254
µ	−0.0903	50.3055	172.1842	19.7793	32.7723	67.3928
Max	2.2986	294.1333	338.0667	26.5333	39.5000	271.1664
σ	0.8401	58.9066	57.1413	3.5075	3.1277	27.3713
Canyons	SPI	PP (mm)	EVP (mm)	TMED (°C)	TMAX (°C)	PET (mm)
Min	−2.7563	0.0000	72.5509	10.1200	24.8750	20.2253
µ	−0.0449	62.5014	153.5088	18.8842	31.6837	69.3710
Max	1.7283	350.5735	654.1389	24.8250	38.3750	121.6185
σ	0.8361	76.3667	57.7539	3.2312	2.8083	27.0455

.

It is worth mentioning that when training multilayer networks, the general practice is to first divide the data into three subsets. The first subset is the training set, which is used for computing the gradient and updating the network weights and biases. The second subset is the validation set. The error on the validation set is monitored during the training process. The validation error normally decreases during the initial phase of training, as does the training set error [35].

The model architecture was established by the trial-and-error method. The training dataset for the model used 70% of the data for each regional SPI time series (1965–2004), 15% of the data was used as the validation dataset and the remaining 15% of the data was used as the testing dataset. The Conjugated Gradient algorithm to train models was selected based on computer memory efficiency over the Levenberg–Marquardt and Bayesian regularization algorithms. The resulting architecture of the NARX model was composed as follows: 6 input neurons, 25 hidden neurons and 1 output neuron with time delay equal to 3 months and training epochs equal to 1000.

The evaluation metric or loss function used to reach the above architecture was Mean Square Error (MSE).

$$\mathrm{MSE}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left({\mathrm{SPI}}_{p_i}-{\mathrm{SPI}}_{o_i}\right)}^2$$

(16)

MSE measures the difference between observed and estimated values. The lower the MSE, the more accurate the prediction is. An additional evaluation metric was the R² representing the goodness-of-fit of the model. A perfect model (unlikely to occur) should have a lower MSE indicating low-accumulated errors and high R² values.

Additionally, we used the difference between the observed and predicted SPI values to estimate the prediction error (PE) of the model [36]:

$$\mathrm{PE}={\mathrm{SPI}}_{o_i}-{\mathrm{SPI}}_{p_i}$$

(17)

3. Results and Discussion

Results of a neural network with an architecture of 6-25-1 for Semi-desert, Highlands, Mountains and Canyons regions can be observed in Figure 3. In general, training and testing the neural networks using a scaled conjugate gradient as a training algorithm estimated low MSE values and high R values (

Table 3. Quantitative measures of the ANN performance in training and test phases for regional SPI time series for Zacatecas State, Mexico. Key measures: Mean Square Error (MSE) and Correlation coefficient (R).

Region	MSE	R
Semi-desert
Training	0.0813	0.9099
Test	0.1197	0.8936
Highlands
Training	0.0826	0.9341
Test	0.0486	0.9631
Mountains
Training	0.0889	0.9365
Test	0.0814	0.9514
Canyons
Training	0.0759	0.9426
Test	0.0991	0.9399

). These results suggest that the 4 regional SPI model’s performance was good.

We identified a good correlation (Figure 4) between the observed and predicted values of SPI within the 64-month test period. This result was confirmed by model parameters of linear regression between the observed and predicted SPI values as can be seen in

Table 4. ANN models’ performance using linear regression models (SPI_p = β₀ + β₁ SPI_o) of observed SPI values (SPI_o) and the forecasted SPI values (SPI_p) within test period for regional SPI time series for Zacatecas State, Mexico.

Region	β0	β1	R2	R
Semi-desert	−0.0099	0.8428	0.9076	0.9526
Highlands	0.0292	0.9417	0.7974	0.8930
Mountains	−0.03158	0.9044	0.7767	0.8813
Canyons	0.0595	0.8738	0.7128	0.8443

.

The prediction ability of the NARX models was assessed by analyzing its performance based on the linear regression statistics for all months within the test datasets by means of the R² and R values (Table 4).

It was evident that the best NARX model predictions correspond to the Semi-desert region. The highest R value (0.9526) belongs to this region. The next best performance was got for the Highlands region next by the Mountains region. Furthermore, the lower R value (0.8443) belongs to the NARX model prediction for the Canyons region. Our results provide convincing evidence of a good SPI prediction ability of the NARX models for all regions in this study.

The probability under normal distribution of the prediction error of the NARX models is summarized in

Table 5. The probability under normal distribution of prediction error (PE) for observed SPI values (SPI_o) and the forecasted SPI values (SPI_p) in test period for regional SPI time series for the state of Zacatecas, Mexico.

Region	PE < 0	PE > 0
Semi-desert	0.4965	0.5035
Highlands	0.5058	0.4942
Mountains	0.5175	0.4825
Canyons	0.5858	0.4142

and shows the under-predictions (PE < 0) and the over-prediction (PE > 0) skills. An error will be zero when the predicted SPI value exactly matches the SPI observed value [37,38]. Our results indicate that PE < 0 occurrences are higher than PE > 0 occurrences for all but Semi-desert regions. Those results suggest that ANN models under-predicted the SPI for all but the Semi-desert region, where the largest difference found was for the Canyons region with a probability of 58.58% and the smallest difference of 49.65% was found for the Semi-desert region. In addition, the minimum probability value of over-prediction with 41.42% was found for the Canyons region and the maximum probability value of over-prediction error was found for the Semi-desert region with 50.35%. Those results provide support to the ones described by summarized statistics of the linear models between the predicted and the observed SPI values (Table 4).

As mentioned earlier, when analyzing hydrological variables, neural networks have shown their great utility for the empirical prediction of these kind of variables [31,39]. Our results compare well with those of Deo and Şahin [14], Ali et al. [4], Soh et al. [15] and Poornima and Pushpalatha [10], that successfully used other approaches of the artificial neural network model to predict the standardized precipitation index per month.

Furthermore, in good agreement with Liu et al. [34], our study corroborates that the NARX network modeling approach may prove to be successful in formulating the non-linear dynamics in complex systems, particularly as a good tool to forecasting time series. Our results broaden those of Evkaya and Furnaz [40] when forecasting drought by means of NARX because they only use SPI time series and SPI wavelet transformed series and focus on the use of 3-month SPI and 6-month SPI series. Moreover, our findings extend those of Giddings et al. [8] and Magallanes et al. [9] because our approach not only allows defining smaller and detailed regional climatic zones in México by means of the SPI but provides a framework to forecast SPI values.

Most notably, this is the first study to our knowledge to apply NARX models for the prediction of monthly standardized precipitation index with exogenous meteorological inputs. Since the developed NARX models combined the effect of rainfall, temperature, evaporation and evapotranspiration, they have potential to be used as an important tool for water resource assessment. Using the combined influence of these parameters, this could be useful for the detection of climatic and agricultural risks.

4. Conclusions

Lately, the prediction of drought is an important issue for meteorology, hydrology, water resource management and sustainable agriculture because of the vulnerability of human activities relating to water use. We develop nonlinear autoregressive network models with exogenous inputs for predicting the values of four monthly Standardized Precipitation Index (SPI) time series belonging to Mexico’s Zacatecas state territory. The models were trained and tested using the regional monthly time series of rainfall, evaporation, maximum temperature, mean temperature, evapotranspiration and multivariate ENSO index as predictors. The output variable was the forecasted SPI time series for each region.

Based on the results of the evaluation metrics, we have demonstrated that the NARX model is a useful AI tool based on climatological data to forecast the monthly SPI values.

In the matter of climate change, drought forecasting by using NARX models as a framework in future studies would be beneficial for generating useful information for making decisions. In addition, future work should evaluate different ANN models, training algorithms and framework approaches to assess different network architecture effects on model performance.