*4.3. Model Implementation*

The parameters of the hybrid model were obtained during the training phase by trial and error including the number of CNN filters, the number of QRNN hidden units, and the dropout rate. As mentioned in different studies [35], kernel size values of 3 and 5 are the most common, so we used kernel size 3 with both datasets in our experiment. A filter can help in extracting more details from a dataset by increasing the number of filters [51]. Thus, for the first CNN layer, we used 64 filters, and for the other CNN, we used 128 filters. Additionally, we set the value of the batch size for the training at 128 and the value of the number of epochs at 10. The details and the selected parameters of the hybrid DL model are presented in Figure 3.

### *4.4. Evaluation Tools and Metrics*

Different evaluation metrics were used in this work to evaluate the performance of the proposed model including accuracy, FPR, TPR, precision, recall, and F-Score. Accuracy represents the ratio of correctly classified threats to the total number of classified threats, so it demonstrates how accurate an model in classifying threats [52]. The FPR represents the ratio of misclassified data as a different type of threat, and the TPR represents a model's ability to correctly classify threats. A low FPR and a high TPR demonstrate the ability of a model to correctly classify cyber threats [53]. Precision, recall, and F-Score were used to evaluate the overall performance of the proposed model; a high value of precision indicates a low FPR, and recall represents a model's ability to correctly classify threats. Equations (1)–(6) represent the evaluation metrics, where *FP* is false positive, *TP* is true positive, *TN* is true negative, and *FN* is false negative.

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN} \tag{1}$$

$$\text{FPR} = \frac{FP}{FP + TN} \tag{2}$$

$$\text{TPR} = \frac{TP}{TP + FN} \tag{3}$$

$$\text{Precision} = \frac{TP}{TP + FP} \tag{4}$$

$$\text{Recall} = \frac{TP}{TP + FN} \tag{5}$$

$$\text{F}-\text{Score} = \frac{2(\text{Precision} \times \text{Recall})}{\text{Precision} + \text{Recall}} \tag{6}$$

### **5. Results and Discussion**

### *5.1. Results and Analysis*

This section presents the results and analysis for model implementation. We used Jupyter Notebook software with the Python programming language. We used the Keras and scikitlearn packages for data pre-processing and implementing the proposed model. We trained the proposed model on a MacBook Air with an Intel Core i5 CPU 1.6 GHz processor and 8 GB RAM. Additionally, we implemented different state-of-the-art ML models on the datasets to compare their performance with that of our proposed model. Figure 4 presents the confusion matrix of our proposed model on the BoT-IoT dataset. The results show that the model correctly classified most of the cyber threat categories. Furthermore, to illustrate the quality of the proposed model, the receiver operating characteristic (ROC) curve is plotted in Figure 5 for the BoT-IoT dataset.

**Figure 4.** Confusion matrix based on the BoT-IoT dataset.

**Figure 5.** ROC curve of using our proposed model on the BoT-IoT dataset.

Figure 6 presents the confusion matrix of our proposed model on the TON\_IoT dataset, and the ROC curve is presented in Figure 7. Both ROC curves show that our proposed model achieved the highest value of 1. Thus, our proposed model performed very well with all the classes.

The results of our proposed model on the testing datasets are presented in Table 4.

**Table 4.** Results of cyber threat classification on both datasets.


**Figure 6.** Confusion matrix based on the TON\_IoT dataset.

**Figure 7.** ROC curve of using our proposed model on the TON\_IoT dataset.

As shown in Table 4, the proposed model achieved high accuracy, with an average of 99.99% on both datasets. The TPR reached averages of 99.92% with the BoT-IoT dataset and 99.99% with the TON\_IoT dataset. The proposed model achieved a low FPR of 0.0003 with the BoT-IoT dataset and 0.001 with the TON\_IoT dataset. Thus, the proposed model showed good performance in classifying the threats with both datasets. Moreover, to demonstrate the effectiveness of the QRNN, we implemented our proposed model with LSTM instead of the QRNN to compare performance. Cybersecurity threats are very critical [54–56], and the results shown in Tables 5 and 6 highlight that our proposed approach could be very effective in dealing with them.

**Table 5.** Comparison of our proposed model while using LSTM and QRNN based on BoT-IoT dataset.



**Table 6.** Comparison of our proposed model while using LSTM and the QRNN based on TON\_IoT dataset.

According to the results in Tables 5 and 6, our proposed model with the QRNN showed the same performance as our proposed model with LSTM in terms of accuracy, precision, recall, and F-Score. In terms of time, the proposed model with the QRNN showed better performance for training the model and testing. The average training time per epoch demonstrated that the QRNN performed faster than LSTM in terms of training the model on both datasets, with a 418.3 s difference on the BoT-IoT dataset and a 19.8 s difference on the TON\_IoT dataset. Additionally, for the classification time on the test dataset, the QRNN model performed faster than LSTM, with a 75 s difference on the BoT-IoT dataset and a 3 s difference on the TON\_IoT dataset. The QRNN showed its effectiveness in increasing the speed of the model while providing a high accuracy and low FPR. Therefore, the model can be used for real-time CTI. We further compared the performance of our proposed model on the BoT-IoT and TON\_IoT datasets against the state-of-the-art models for the multi-class classification of threats. The results of these comparisons are shown in Tables 7 and 8.

**Table 7.** Comparison of our proposed model with state-of-the-art models based on the BoT-IoT dataset.


**Table 8.** Comparison of our proposed model with state-of-the-art models based on the TON\_IoT dataset.


As shown in Tables 7 and 8, though K-NN [19] and RF [28] showed good performance for recall and F-score on the BoT-IoT dataset, our proposed model outperformed the stateof-the-art models on both datasets. Additionally, we implemented different ML models to compare their performance with that of our model. The accuracy, TPR, and FPR values of each model are given are Tables 9 and 10. Our model performed better than the other four models, with accuracy measured as 99.99% on both datasets and low FPR values of 0.0003 on the BoT-IoT dataset and 0.001 on the TON\_IoT dataset. The LSTM model showed good performance in terms of accuracy and FPR, while the GRU showed a high TPR compared to the LSTM on the BoT-IoT dataset. On the TON\_IoT dataset, the GRU performed poorly compared to the other models.


**Table 9.** Comparison of our proposed model with other ML models based on BoT-IoT dataset.

**Table 10.** Comparison of our proposed model with other ML models based on TON\_IoT dataset.


### *5.2. Theoretical and Practical Implications*

This work describes a model that can correctly classify cyber threats with a low FPR while considering time performance. Thus, the proposed model can improve decision making for risk mitigation so that appropriate protection measures against cyber attacks in smart cities can be taken [57,58]. Additionally, this model will benefit organizations and services providers in smart cities because of the high costs of implementing and maintaining cyber security solutions [59]. The organizations and service providers in smart cities can take accurate proactive measures against detected cyber attacks such as data breaches, which will help in saving costs [60]. Furthermore, our proposed model can be implemented in the cloud to monitor cyber security and collect and update cyber threat data from the connected systems in smart cities.
