1. Introduction
By flooding malicious traffic, DoS attacks deplete a targeted system’s network bandwidth and computing resources, preventing the target system from providing legitimate users with regular services. DDoS attacks are becoming increasingly widespread. DDoS attacks seize control of a huge number of infiltrated computers, known as botnets, and conduct synchronized attacks on the victim system, as can be seen in
Figure 1. Together with the emergence and accomplishment of revolutionary Internet technologies, DDoS attacks are escalating in frequency, magnitude and sophistication. Organizations face possible network threats that could have serious consequences for their operations, such as downtime, data breaches, or even extortion demands from attackers [
1].
As indicated in [
2], appropriate DDoS mitigation procedures should be taken upon the occurrence of DDoS attacks. Before any mitigation strategies can be implemented, DDoS attacks must be detected essentially. DDoS attacks were initially detected by traffic engineers using programmed rules. This approach appears to have fallen behind the dynamic and evolving nature of DDoS attacks. As ML (Machine Learning) and DL (Deep Learning) exhibit their enormous potential in a variety of fields, academia and industry are investigating the feasibility of applying ML/DL to detect DDoS. Notable successes have been achieved, as documented in [
3]. Human experts or specific feature selection schemes need to choose features for classification in ML. On the other side, feature engineering is a vital built-in piece of DL. Some successful stories on DL for detecting DDoS will be analyzed in
Section 2.
When it comes to DDoS detection, both ML and DL demand labeled traffic for training. DDoS detection system’s performance is determined by the quality of the training set. A new type of DDoS attack referred to as adversarial DDoS attack could pose new challenges to traditional approaches. Since Szegedy et al. [
4] first proposed the concept of adversarial examples, their work has aroused interest among researchers in the field of adversarial attacks. In the Internet of Things era, the network has become an ideal target for cyber attackers. Hackers often exploit vulnerabilities in artificial intelligence to bring out cyberattacks. Despite the AI communities’ indefatigable endeavors to establish defensive barricades, the number of adversarial attacks is escalating dramatically, and their posing threats proceed to grow. Furthermore, as the economic benefits sprout, the number of adversarial attacks will keep rising. A secure cyber security environment can only be created through the continuous development of deep learning patterns, as suggested by Han et al. [
5].
As stated in [
6], attacking and defending adversaries is an iterative, evolutionary process. Adversary data makers evolve new adversarial attacks by exploring new vulnerabilities and improving algorithms, while defenders analyze the characteristics of alternative threats and employ suitable methods to provide effective defenses against adversarial attacks.
The Generative Adversarial Network (GAN) [
7] is well-known for creating fictitious but realistic-looking data, such as image generation. We believe GAN can also create fraudulent but legitimate-looking traffic, which causes DDoS detection systems to be confused. This study is a corroboration of such a belief. We synthesize adversarial DDoS attacks employing Wasserstein Generative Adversarial Networks [
8] featuring Gradient Penalty (GP-WGAN) [
9]. As we shall see in
Section 4, synthesized traffic can evade DDoS detection systems such as
k-Nearest Neighbour, Multi-Layer and Perceptron Random Forest.
This study has its contribution in the proposal of a novel framework, named Generative Adversarial Networks with Dual Discriminators (GANDD), for the detection of adversarial DDoS attacks. The proposed GANDD has a unique design featuring dual discriminators. One discriminator provides the ability to detect adversarial DDoS attacks, while the other distinguishes legitimate traffic from conventional DDoS traffic. The additional discriminator is designed to target adversary DDoS traffic. To evaluate GANDD’s effectiveness, adversarial DDoS traffic synthesized by GP-WGAN is used to train GANDD. Trained GANDD outperformed DNN (Deep Neural Network), LSTM (Long Short-Term Memory), and GAN, with a TPR (True Positive Rate) of 84.3%. A more sophisticated test was also conducted to examine GANDD’s ability in handling unseen adversarial attacks. GANDD was evaluated with adversarial traffic generated from different training data. GANDD still proved effective with a TPR of around 71.3% compared to 7.4% of LSTM.
The proposed GANDD has its novelty and contribution both in structure and application. To our best knowledge, there are only a small number of studies working on GAN with dual discriminators, such as that in the works of Nguyen et al. [
10] and Zhang et al. [
11]. Nonetheless, these studies developed parallel dual discriminators that worked separately and focused on increasing the quality and diversity of synthesized data generated by the generator, mainly for image processing applications. Meanwhile, our approach has a novel and distinct design with consecutive dual discriminators that complement each other to defend against adversarial DDoS attacks effectively.
The rest of this paper is structured as follows. In
Section 2, previous works and GAN are reviewed briefly.
Section 3 embodies out the framework for the proposed approach. The results of the experiment are then reported in
Section 4. Finally, some conclusions are made in
Section 5.
4. Experiments Results and Discussion
A series of experiments are implemented to verify that the proposed approaches are feasible and effective. The followings statements will be examined in turn in the subsequent experiments:
ML-based DDoS detection is highly effective
Adversarial DDoS attacks can penetrate ML-based DDoS detection
The proposed GANDD is capable of detecting adversarial DDoS attacks
Experiments are conducted on a Windows 10 personal computer with an Intel i7-8700 CPU, 32GB RAM and an NVIDIA RTX 2060 graphics card. The GP-WGAN, for the synthesis of adversarial DDoS attacks, is implemented with PyTorch. The GANDD framework for detecting adversarial DDoS detection is implemented in TensorFlow paired with Keras.
Two well-embraced datasets, NSL-KDD [
27] and CIC-IDS2017 [
28], are adopted in this study. They are datasets presented by the Canadian Institute for Cybersecurity for the study of intrusion detection. The NSL-KDD dataset is an optimized version of the KDD’99 [
29] dataset, which removes redundant and duplicate data from KDD’99. The CIC-IDS2017 dataset contains benign and the most up-to-data common attacks, it also includes the results of the network traffic analysis using CICFlowMeter [
30] with labelled flows based on the time stamp, source, and destination IPs, source and destination ports, protocols and attack (CSV files). We focus on DDoS traffic and use them for training ML-based DDoS detection and the synthesis of adversarial DDoS attacks.
A number of performance indices are employed for comparative study. Referring to
Table 1, the definitions of True Positive Rate (TPR), False Positive Rate (FPR) and F1-score are given in Equations (6)–(8), respectively. The Receiver Operating Characteristic (ROC) curve is used to evaluate the impact of adversarial DDoS attacks and the effectiveness of the proposed GANDD in the detection of DDoS attacks.
4.1. Synthesis of Adversarial DDoS Attacks Using GP-WGAN
Literature reports that ML had been applied to the detection of DDoS with satisfactory achievement. This subsection examines the capability of DDoS detection of RF, KNN, and SVM on NSL-KDD and CIC-IDS2017. Training data are labelled as legitimate or malicious beforehand. 80% of the samples are used for training, and 20% are used for validation. The results of applying RF, KNN and SVM on NSL-KDD and CIC-IDS2017 are reported in
Table 2 and
Table 3, respectively. A higher TPR and F1-score value is an indication of better detection capability. We can see that all three ML-based DDoS detection approaches are considered effective. This finding is consistent with those in previous studies.
In addition to the TPR and the F1-score, the ROC curve is another useful performance metric to assess the detection and classification accuracy. For a ROC curve, a larger Area Under the Curve (AUC) indicates better detection accuracy.
Figure 5 shows that RF has the best detection rate, followed by KNN and then SVM.
4.2. Synthesis of Adversarial DDoS Attacks Using GP-WGAN
GAN is well recognized in the generation of realistic-looking fake images. We expect that GAN can also be used to generate legitimately-like malicious traffic, i.e., adversarial DDoS attacks. We adopt the GP-WGAN architecture discussed in
Section 3 with a network configuration listed in
Table 4 and training parameters given in
Table 5. Gradient Penalty is incorporated to stabilize the training process. The activation function of the neural networks adopts Leaky ReLU instead of sigmoid to improve the robustness of the training process and to prevent the phenomena of vanishing gradient.
After training, the adversarial DDoS attacks synthesized by the GP-WGAN are directed to ML-based DDoS detectors to observe their capability in detecting adversarial DDoS attacks. We experiment with RF, KNN, and SVM on adversarial DDoS attacks trained using NSL-KDD and CIC-IDS2017. As shown in
Table 6, the TPRs drop dramatically to near zero. It implies that adversarial DDoS attacks can easily penetrate ML-based DDoS detectors without being detected. The ROC curves in
Figure 6 also report ML-based DDoS detectors’ inability to deal with adversarial DDoS attacks. We can conclude that adversarial DDoS attacks are a substantial breach of conventional ML-based DDoS detectors. Furthermore, the adopted GP-WGAN is highly effective in synthesizing adversarial DDoS attacks.
Conventional ML/DL-based DDoS detection is trained using standard DDoS datasets. They can successfully deal with conventional DDoS attacks. However, synthesized adversarial attacks are legitimate-like malicious traffic. To figure out the reason behind the severe impact brought about by adversarial DDoS attacks, we virtualize the distribution of training samples and adversarial DDoS attacks for SVM. As revealed in
Figure 7, synthesized adversarial DDoS attacks lay along the discrimination hyperplane of SVM. Such a distribution renders SVM unable to make correct decisions and therefore degrades its detection ability. In other words, conventional ML/DL approaches lack the ability in detecting adversarial DDoS attacks.
4.3. Detection of Adversarial DDoS Attacks with GANDD
In recognizing the potential threats of adversarial DDoS attacks, we develop the GANDD, presented in
Section 3, as a countermeasure. The adversarial DDoS attacks synthesized by the GP-WGAN can be used to train the proposed GANDD to endow it with the ability in detecting adversarial DDoS attacks. As explained in
Section 3, the additional discriminator in GANDD is designed to detect adversarial DDoS attacks. The second discriminator is used to discriminate legitimate traffic and conventional DDoS attacks. The network configurations and the training parameter settings for the proposed GANDD are given in
Table 7,
Table 8 and
Table 9.
A fundamental difficulty in training GANs is the disorientation of training direction caused by the gradients vanishing phenomenon. The GANDD model in this article employs the PReLU activation function to let the model adjust the gradient’s direction on its own, avoiding the issue of information loss. We also add dropout layers to prevent the overfitting problem. The optimization function of the proposed GANDD additionally incorporates Gradient Penalty as a constraint to make the entire training process more stable. Experiment results reveal that the given model configurations effectively reduce the gradient disorientation problem and therefore stabilize the training process.
For the sake of comparative study, we have Deep Neural Network (DNN), LSTM and standard GAN included in the subsequent experiments. We first examine how these DL technologies perform on conventional DDoS attacks. As reported in
Table 10 and
Table 11. All four deep learning technologies under investigation perform well in detecting conventional DDoS attacks on both NSL-KDD and CIC-IDS2017. Among them, the proposed GANDD has the best detection rate.
Figure 8 shows the ROC curves of different DL-based DDoS detection.
We now concentrate our attention on the impact of adversarial DDoS attacks. Adversarial DDoS traffic synthesized by the GP-WGAN is used to train the GANDD and test all four DL technologies. As reported in
Table 12, the TPRs of DNN, LSTM and GAN drop dramatically, showing their inability to deal with adversarial DDoS attacks. This is because the substitute neural network’s capability of generalization is degraded by the adversarial DDoS attack’s atypical probability distribution. On the other hand, the proposed GANDD can still maintain a TPR value of 0.843. This indicates that the proposed GANDD can be an effective solution to challenging adversarial DDoS attacks.
Figure 9 shows the ROC curves of different DL-based DDoS detection on adversarial DDoS attacks. GANDD’s ROC curve has a high AUC value and steep slope, indicating its superiority over DNN, LSTM and GAN.
A more challenging and more realistic scenario is considered in the following experiment. The GANDD is trained with adversarial DDoS traffic synthesized using CIC-IDS2017 but tested with adversarial DDoS traffic synthesized using NSL-KDD. Such an experiment setting is to examine the GANDD’s ability in handling unseen adversarial DDoS attacks. The results are given in
Table 13 and
Figure 10. The performance of DL-based approaches degrades even further. Even the best case, LSTM, has a TPR as low as 0.074. Although the TPR of GANDD decreases, the ROC curve of GANDD also exhibits a drop in both steepness and AUC. GANDD still maintains a TPR value of 0.73. This result reflects the elusiveness of unseen adversarial DDoS attacks. However, GANDD is still considered effective compared to the other approaches.
The menace and evasiveness of adversarial attacks generated by GP-WGAN are evident according to
Table 6. They can breach many commonly used ML-based defense systems. As the best, RF has a TPR as low as 9.7%. Similarly,
Table 12 shows that DL-based defenses are also ineffective with regards to adversarial attacks. Even the best one, LSTM, the TPR is only 11.7%. On the contrary, the proposed GANDD demonstrates its defensive capabilities with a TPR as high as 84.3%. Even for a more challenging and realistic scenario of cordoning unseen adversarial attacks, GANDD still proved effective with a TPR around 71.3% compared to the 7.4% of LSTM. We arrive at the same conclusion in examining
Figure 9 and
Figure 10. With a high AUC of 0.814 and 0.717, respectively, GANDD has steep ROC curves indicating that its sensitivity increases more rapidly than specificity. On the other hand, due to the low TPR rates, the ROC curves of the other DL-based approaches tend to be closer to the diagonal, implying lower sensitivity and higher specificity.
Hu et al. [
22] introduced an exciting GAN-based architecture for DDoS attacks called MalGAN, neglecting the ML black-box detection model. MalGAN’s attack has excellent infiltration power except obstacle from Random Forest, where TPRs remain at 80%. Meanwhile, adversarial attacks from GP-WGAN overcome most ML defending systems including RF, as shown in
Table 6. Furthermore, Hu’s research also does not propose an adequate defence model like GANDD.
Based on the original GAN model, Li et al. [
20] obtained interesting results with the introduction of the “RCC Dual-GAN” featuring two separate GANs. The first model performs the data distribution reference, while the second model responds to overfitting constraints. On the other hand, we had taken a novel and radically different approach by entrenching an additional discriminator in a unified GAN model. Accordingly, the proposed GANDD becomes more comprehensive and coherent, as shown in the analysis above.
5. Conclusions
This study investigates the potential threat of a new type of DDoS attack known as adversarial DDoS attacks. The adopted GP-WGAN architecture could indeed generate legitimate-looking malicious traffic. According to experimental results, the synthesized adversarial DDoS attacks can easily penetrate ML-based detection systems, such as RF, KNN, and SVM. This phenomenon is a sobering reminder that adversarial DDoS attacks require urgent countermeasures. The GANDD architecture proposed in this study is a response to this new threat. The experimental results determine that adversarial DDoS attacks can be efficiently intercepted with an additional discriminator for capturing and tagging adversarial DDoS traffic. The GANDD can achieve TPRs of values 0.985, 0.843 and 0.713 on conventional DDoS attacks, adversarial DDoS attacks, and unseen adversarial DDoS, respectively. The proposed GANDD is considered adequate, although there is still room for improvement for unseen adversarial DDoS. The incorporation of open-set recognition technology could be a promising direction that deserves further investigation.