*3.4. Blindfolding Procedure*

In addition to assessing the *R*<sup>2</sup> values (i.e., a measure of predictive accuracy), the Stone–Geisser's *Q*<sup>2</sup> value (i.e., a criterion of predictive relevance) was calculated as well. The blindfolding approach [51] calculates the cv-communality index (*H*2) as well as the cv-redundancy index (*Q*2) for constructs and indicators. An *H*<sup>2</sup> and *Q*<sup>2</sup> values greater than zero confirm that the structural and measurement models are important for forecasting [38]. As can be seen from Table 8, all values of *H*<sup>2</sup> exceed 0 (average values of *H*<sup>2</sup> = 0.492) and all values of *Q*<sup>2</sup> exceed 0 (average value of *Q*<sup>2</sup> = 0.442). Our measurement model shows a higher level of quality as compared with the structural one. Since the value of 0.02 indicates "a small effect size, 0.15 a medium one, and 0.35 a high effect size" [105], both models have a high level of predictive acceptability.


**Table 8.** cv-redundancy (*Q*2) and cv-communality (*H*2) indices.

#### *3.5. Artificial Neural Network Analysis*

The next step is the application of ANN models. They are applied to classify the relative impact of only important predictors acquired from analysis of PLS-SEM. Based on the proposed research model and the results of PLS-SEM, it is possible to create four ANN models: Model 1, where the inputs are OPC, PCIL, and STC and the output is WC; Model 2, where the inputs are STC and WC and the output is PU; Model 3, where the inputs are WC and PU and the output is AT); and Model 4, where the inputs are WC and AT and the output is ExU).

However, before assessing the ANN models, the ANOVA Test of Linearity was applied [15,62]. It tests the presence of non-linear relationships in potential ANN models. The results of the ANOVA test of Linearity are included in Table 9.


**Table 9.** ANOVA Test of Linearity.

The test results show that the connection among construct WC and construct AT has a statistically significant deviation from linearity (with significance *p* < 0.05), while the relationship between construct AT and construct ExU is very close to this conclusion (*p* = 0.055). These results justify the introduction of artificial neural networks, which are capable of considering non-linear effects present and, therefore, of more accurately modeling user behavior.

In this research, ANNs were formed in SPSS 20. All ANN models have one hidden layer [59], and the number of neurons in this hidden layer was regulated by simulation software system [65,106]. There were two hidden layers for all four ANN models. As an activation function in hidden as well as output layers, sigmoid was used [58,101]. An example of an ANN model created in SPSS is shown in Figure 5.

The testing research sample was split into training sub-sample (90% of the data) and testing sub-sample (left over 10% of the data) [19,107]. Among the more common problems in the ANN analysis is the over-fitting—the situation when the ANN model simply remembers all training cases and loses the capability for general analysis, i.e., forecasting the result accurately with a previously invisible set of inputs [64]. To avoid this situation, ten-fold cross-validation was used [52,107].

"The RMSE (Root Mean Square Error) is applied to estimate the predictive accuracy of ANN models" [65,108]. The results that refer to both sets of data (testing and training) for all 10 ANNs were obtained (Table 10).

Low average values for RMSE for all four ANN models, for both training and testing datasets (varying from 0.0913 to 0.1354), reflect the good predictive power of the models [57].

Finally, to determine the relative importance of each predictor (variations of the output for different values of the input), sensitivity analysis of the ANN models was performed. The normalized significance of each predictor was computed by dividing the relative significance values by the largest significance value [16,107]. It is usually expressed in percentages. The results of ANN sensitivity analysis, i.e., the relative and normalized importance of the predictors in each ANN model, were obtained (Table 11).

**Figure 5.** An example of an ANN model—Model 1.

**Table 10.** RMSE values of ANN model.


The most significant predictor of WC is the OPC—(second-order construct), followed by STC (second-order construct) and PCIL (second-order construct), which is in contrast with the PLS-SEM results, in which STC had a stronger effect than OPC. WC was identified as a far more important antecedent of PU than STC, which is the same as predicted by PLS-SEM, with a minor relative difference in influence—the ANN model predicted that this relative difference was slightly lower than that predicted by PLS-SEM. Similarly, WC was identified as a far more significant predictor of AT than PU, which is again the same as predicted by PLS-SEM results. Again, there was a minor relative difference in influence the ANN model predicted that this relative difference was slightly higher than the PLS-SEM predictions. Finally, the ANN model predicted that WC has a stronger influence on ExU compared to AT, which is in line with PLS-SEM findings, but the relative difference was slightly lower. These minor differences between the two techniques and the results obtained reflect the higher prediction accuracy of the ANN models, which consider existing nonlinear effects among variables [62].

**Table 11.** Neural network sensitivity analysis.

