Leveraging Incremental Learning for Dynamic Modulation Recognition

Abstract

Modulation recognition is an important technology used to correctly identify the modulation modes of wireless signals and is widely used in cooperative and confrontational scenarios. Traditional modulation-recognition algorithms require the assistance of expert experiences, which constrains their applications. With the rapid development of artificial intelligence in recent years, deep learning (DL) is widely advocated for intelligent modulation recognition. Typically, DL-based modulation-recognition algorithms implicitly assume a relatively static scenario in which the signal samples of all the modulation modes can be completely collected in advance. In practical situations, the radio environment is quite dynamic and the signal samples with new modulation modes may appear sequentially, in which the current DL-based modulation-recognition algorithms may require unacceptable time and computing resource consumption to re-train the optimal model from scratch. In this study, we leveraged incremental learning (IL) and designed a novel IL-based modulation-recognition algorithm that consists of an initial stage and multiple incremental stages. The main novelty of the proposed algorithm lies in the new loss function design in each incremental stage, which combines the distillation loss of recognizing old modulation modes and the cross-entropy loss of recognizing new modulation modes. With the proposed algorithm, the knowledge of the signal samples with new modulation modes can be efficiently learned in the current stage without forgetting the knowledge learned in the previous stages. The simulation results demonstrate that the proposed algorithm could achieve a recognition accuracy close to the upper bound with a much lower computing time and it outperformed the existing IL-based benchmarks.

1. Introduction

Modulation recognition is the technology used to correctly identify the modulation mode of a wireless signal without any prior knowledge and is widely used in both cooperative and confrontational scenarios (see [1]). For instance, modulation recognition can be used to select a proper demodulation scheme and enhance the communication efficiency in cooperative scenarios, while in confrontational scenarios, it is extensively used in electronic warfare (see [2,3]). As wireless communication technology has advanced in recent years, the radio environment has become increasingly dynamic, imposing higher demands on the modulation-recognition capability. Generally, modulation-recognition methods can be divided into traditional methods and DL-based methods. Traditional methods recognize the modulation modes by detecting the physical or statistical characteristics of target signals (see [4,5]). These methods are structurally complex, requiring extensive expert knowledge. More importantly, they are sensitive to the radio environment, constraining their wide application, as seen in [6,7,8]. Recently, DL-based methods have attracted extensive attention in the field of modulation recognition and have promoted its development. DL-based methods are typically data driven and optimize the recognition model parameters through statistical theory. DL-based methods were proven to be more robust than traditional methods in previous studies. In particular, O’Shea et al. [9] applied DL to the field of modulation recognition, demonstrating that DL methods outperform traditional methods in recognition accuracy and inference speed. Lin et al. [10] introduced the time and frequency attention mechanism into the convolution neural network (CNN) and enhanced the recognition performance with the assistance of two domain features. Ayman et al. [11] leveraged the fact that wireless signals have temporal correlations and combined a CNN and long short-term memory (LSTM) to further improve the recognition performance.

However, most of the existing DL-based modulation-recognition methods implicitly assume a relatively static scenario in which the signal samples of all the modulation modes can be completely collected in advance. In highly dynamic radio environments, new signals with different modulation modes may appear. It is clear that the emergence of new modulation modes can severely degrade the recognition performance of the well-trained model and new modulation modes cannot be correctly recognized. One straightforward way to solve this issue is to manually label the newly received wireless signal samples, fuse them into the dataset and re-train the model. Nevertheless, re-training a new model from scratch by using all the samples is time inefficient and resource consuming.

To tackle the above issue, researchers started to study the application of incremental learning (IL) to dynamic modulation recognition. In particular, IL is also called continual learning or lifelong learning and is an emerging paradigm in the field of machine learning used to accommodate new data while retaining previously acquired knowledge without re-training a new model from scratch [12]. One typical application of IL is class-increment learning, where the number of tasks cannot be completely predefined since the number of classes to be learned gradually increases over time. The main technical character of IL lies in the fact that by properly incorporating the transfer learning and knowledge distillation [13], the training of the new model that can accommodate new classes does not need to reuse all the old data and can save significant time and computing resources. These lightweight and continual learning capabilities make the IL particularly suitable for modulation recognition in dynamic radio environments in which wireless signals with new modulation modes arrive sequentially. For example, Todd M. et al. [14] applied IL to the field of modulation recognition. Specifically, the initial model therein could only recognize modulated signals with band frequency modulation and four-level pulse amplitude modulation (PAM4), and then the IL was adopted to extend the recognition capability of the model to eight-phase shift keying (8PSK), double side band (DSB) and a total of 11 modulation modes, verifying the feasibility of the IL-based modulation-recognition method. Furthermore, refs. [15,16] also studied the application of IL to the field of modulation recognition. In particular, Charles M. et al. [15] employed the bias correction (BiC) method in IL and discussed the impacts of different learning steps and storage capacities on the performance of the IL algorithm. Qi P. et al. [16] combined federated learning and IL to address the problem of heterogeneous signals that have a non-independent identical distribution across different scenarios.

We noticed that there was a key drawback in the above studies and the performance was unsatisfactory. More specially, the adopted IL algorithms typically calculated the loss of recognizing new modulation modes with the old model in the incremental training. Suppose that the sample feature distribution of the new modulation modes differs largely from that of the old modulation modes. Using the old model to calculate the loss of recognizing the new modulation modes may cause significant errors and inevitably degrades the incremental training efficiency and recognition performance. Alternatively, we designed a novel loss function by combining the distillation loss of recognizing the old modulation modes and the cross-entropy loss of recognizing new modulation modes, and propose a new IL-based modulation-recognition algorithm. The simulation results show the advantages of the proposed new IL-based modulation-recognition algorithm over the existing ones in terms of the recognition accuracy in typical signal-to-noise ratio (SNR) regions.

The structure of this paper is as follows: Section 1 introduces the background of the modulation recognition, together with the motivation of this study; Section 2 provides the problem description; Section 3 elaborates on the design of the proposed IL-based modulation-recognition algorithm; Section 4 analyzes the performance of the proposed algorithm through extensive simulations; and Section 5 concludes the paper.

2. Problem Description

In modulation recognition, the receiver needs to accurately identify the modulation mode of a wireless signal. Suppose that there is an initial wireless signal dataset ${\mathcal{D}}_0$, which is pre-collected and contains $S_0$ modulation modes. Without loss of generality, $(X_i,Y_i)$ is denoted as the i-th signal sample with the corresponding label (modulation mode) in $D_0$. Then, a modulation-recognition model $f(X_i;\Theta )$ can be constructed, where $\Theta$ is the vector of the model parameter variables. By using the dataset to train the model with a certain algorithm, such as stochastic gradient descent, the model can output the accurate modulation mode index of the input signal sample, i.e., $Y_i=f(X_i;{\Theta }_0^{*})$, where ${\Theta }_0^{*}$ is the optimized vector of model parameter.

One main issue with the method above lies in the generalization capability, which means that the well-trained model can work for the modulation modes only in the collected dataset and may fail to recognize a wireless signal with unseen modulation modes. In practical scenarios, the radio environment is highly dynamic and new wireless signals with various modulation modes may appear from time to time, and it is quite challenging to have the access to pre-collect the signal samples with complete modulation modes in advance. Consider signal samples with $T_m(m\ge 1)$ new modulation modes that arrive and are manually labeled at the beginning of the m-th time interval. By denoting $D_{m,old}$ and $D_{m,new}$ as the datasets storing the old and new data in the beginning of the m-th time interval, the overall dataset at the beginning of the m-th time interval can be denoted as ${\mathcal{D}}_m={\mathcal{D}}_{m,old}\cup {\mathcal{D}}_{m,new}$. Then, the target in the m-th time interval is to train a proper model $f(X_i;{\Theta }_m)$ to recognize both old and new modulation modes, i.e., $Y_i=f(X_i;{\Theta }_m^{*})$ for ∃$(X_i,Y_i)\in D_m$, where ${\Theta }_m^{*}$ is the optimized vector of the model parameters.

Using the existing IL-based modulation-recognition methods, it was shown that IL is a promising tool to achieve dynamic modulation recognition, although the current performance is still unsatisfactory. In this study, we designed a new IL-based modulation-recognition algorithm to improve the recognition performance.

3. Proposed IL-Based Modulation-Recognition Algorithm

In this part, we propose a new IL-based modulation-recognition algorithm to achieve dynamic modulation-recognition capability in the presence of continuously arriving new modulation modes. In particular, we first give the main principle of the new algorithm. Then, we elaborate on the details of the design.

3.1. Main Principle of the Proposed Method

The proposed IL-based modulation-recognition method can be divided into an initial stage and multiple incremental stages, as shown in Figure 1. In the initial stage, suppose that there are S modulation modes in the pre-collected dataset of wireless signal samples. The target of the initial stage was to establish an initial deep neural network (DNN) and train the DNN (optimize the DNN parameters) to correctly recognize the corresponding modulation modes of the input wireless signal samples. In this stage, we optimized the DNN structure in our previous work [17] and trained the DNN parameters with supervised learning, forming an initial modulation-recognition model. In each incremental stage, supposing that wireless signal samples subject to new modulation modes arrived sequentially and were manually labeled, we used the IL to update the DNN model to accommodate new modulation modes. This can be justified by the fact that multiple signal samples with different modulation modes may arrive during a certain time interval, though only the modulation modes with enough accumulated signal samples can be used for the model update. In particular, the model update included three modules, i.e., a DNN output port argument to comply with increased modulation modes, an incremental training module for new DNN parameter optimizations and an exemplar update. It should be pointed out that the exemplar in each incremental stage is defined as the stored dataset consisting of both old and new modulation modes while dropping the other samples at the end of the current incremental stage [18], which are used as the old dataset for the training in the next incremental stage. The exemplar mechanism enables the DNN model to revisit the knowledge of partial signal samples with old modulation modes while learning the knowledge from the signal samples with new modulation modes, preventing the catastrophic forgetting issue in IL.

3.2. Initial Stage

We first needed to establish a proper DNN for efficient learning in the initial stage and the following incremental stages. Note that the designed DNN in our previous work [17] showed feature extraction capabilities superior to other existing DNN designs. In particular, as shown in Figure 2, the time-domain signal sequence in [17] is first transformed into a frequency-domain signal sequence and then both the time-domain and frequency-domain signal sequences are concurrently processed by real-value convolution and complex-value convolution, forming four data flows, which are then fused with a proper attention mechanism for adaptive multi-domain feature extractions and modulation recognition. The time-frequency attention mechanism assigns adaptive weights to different features in the time and frequency domains, allowing the algorithm to focus more on important features, thus further enhancing the recognition performance. In the proposed algorithm, the attention mechanism is influenced by the convolution and channel attention mechanisms. It first performs convolution operations on both the time-domain and frequency-domain features, then computes a global average for each channel, calculates the weights of the channel features through the Sigmoid function and finally multiplies the features by the corresponding weights to achieve efficient feature fusion.

Nevertheless, the designed DNN in [17] cannot be directly used in IL due to the imbalance between the old data and new data. Specifically, to accelerate learning the knowledge from new data in incremental learning, the new data size is typically larger than the old data size, and the total training loss of the incremental learning is the summation of the training losses of the old data and new data. Thus, there exists an inherent amount imbalance between the old and new data in the training process of incremental learning. In other words, the total training loss may be dominated by the training loss of the new data. Then, the model parameters are mainly updated in the direction of minimizing the training loss of the new data. This causes a bias in the recognition accuracy between the old and new modulated signals [19].

To mitigate the impact of data imbalance on the performance, we added an output-correction layer after the output layer in [17] to correct the bias. To achieve this, we first removed the Softmax activation function of the neurons in the output layer of the DNN model, after which we added a new layer named the output-correction layer, as shown in Figure 2. According to [15,19], the bias can be effectively corrected by adding an output-correction layer. Without the output-correction layer, the output values subjected to the new modulation modes are much larger than those of the old modulation modes, i.e., the model tends to classify the input as new modulation modes. To solve this issue, the output-correction layer first scales the values from the output layer adaptively, and then maps the scaled values onto (0, 1). Mathematically, the processing of the output-correction layer can be described as

$$y_i=Sigmoid(w_i\ast x_i+b_i),$$

(1)

where $x_i$ denotes the value from the i-th port in the output layer, and $w_i$ and $b_i$ denote the parameters of the i-th port in the output-correction layer, which are both learnable. To this end, supervised learning can be used to train the DNN model until convergence. More details can be referenced in [17].

3.3. Incremental Stage

As mentioned above, each incremental stage includes three modules, i.e., a DNN output port argument for complying with increased modulation modes, incremental training for the parameter optimizations of the new DNN model and an exemplar update. The signal-processing procedure in each incremental stage is given in Algorithm 1. In the m-th incremental stage, the algorithm first augments the output and output correction ports based on the number of new emerging modulation modes, where the parameters are randomly initialized. Subsequently, the loss is calculated based on the input data and proposed loss function, and the parameters of the two parts of the model are alternately updated according to the corresponding gradients. Finally, the stored signals are updated based on the signal storage capacity, i.e., the exemplar is updated. In the following, we elaborate on the design of each module.

Algorithm 1: IL-based modulation-recognition algorithm in the m-th incremental stage.

3.3.1. Dnn Output Port Argument

It is clear that if we wanted to use the well-trained DNN in the previous stage to recognize the wireless signal samples with new modulation modes, we needed to revise the old DNN structure. To achieve this, we denoted the number of modulation modes in the beginning of the old data (exemplar) and new data in the m-th stage as $S_m$ and $T_m$, respectively, and we added $T_m$ neurons without adopting any activation function in the output layer, as shown in Figure 2. Accordingly, we also added $T_m$ neurons with Sigmoid activation functions in the output-correction layer. Then, there are $S_m$ + $T_m$ neurons in both the output layer and the output-correction layer, which corresponds to $S_m$ + $T_m$ modulation modes (including $S_m$ old modulation modes and $T_m$ new modulation modes). In particular, each added neuron in the output layer is fully connected to the neurons in the previous layer and is connected uniquely to an added neuron in the output-correction layer.

3.3.2. Incremental Training

The incremental training module aims to optimize the parameters of the revised DNN model (new DNN model) after the output port arguments with both old data (exemplar) and new data. The main idea of the incremental training is twofold. First, since a part of the revised DNN model in the current stage is the same as the old DNN model in the previous stage, the knowledge represented by the old DNN model in the previous stage can be transferred to the new DNN in the current stage, i.e., the well-trained parameters in the old DNN model can be reused in the new DNN model with minor updates. This operation helps to accelerate the training of the new DNN model with partial old data. Second, both the exemplar and new data are jointly utilized to optimize the revised DNN model so that the knowledge in new modulation modes can be learned without forgetting the knowledge in old modulation modes. This can prevent the catastrophic forgetting issue.

Here, we denote ${\Theta }_m=[{\Theta }_{m,1},{\Theta }_{m,2}]$, in which ${\Theta }_{m,1}$ represents the vector of the parameter variables before the DNN output port arguments and ${\Theta }_{m,2}$ represents the introduced vector of the parameter variables due to the DNN output port argument in the m-th stage. Since the vector of the parameter variables before the DNN output port argument in the m-th stage is identical to the vector of the parameter variables after the DNN output port argument in the (m − 1)-th stage, we have ${\Theta }_{m,1}={\Theta }_{m-1}$. To train ${\Theta }_m$, the two sub-vectors in ${\Theta }_m$, i.e., ${\Theta }_{m,1}$ and ${\Theta }_{m,2}$, are alternatively trained until both of them converge in the m-th stage.

To begin with, an epoch is defined as the duration of training ${\Theta }_{m,1}$ or ${\Theta }_{m,2}$ once in an incremental training stage, the epoch index is defined as k, and the initial values of ${\Theta }_{m,1}$ and ${\Theta }_{m,2}$ in the k-th epoch of the m-th stage are defined as ${\Theta }_{m,1}^{\left(k\right)}$ and ${\Theta }_{m,2}^{\left(k\right)}$. It is clear that multiple epochs are needed for both ${\Theta }_{m,1}$ and ${\Theta }_{m,2}$ to converge in the m-th stage and the converged values of ${\Theta }_{m,1}$ and ${\Theta }_{m,2}$ in the m-th stage are denoted as ${\Theta }_{m,1}^{*}$ and ${\Theta }_{m,2}^{*}$.

In the first epoch (i.e., $k=1$) of the m-th stage, the knowledge from the (m − 1)-th stage is transferred to the m-th stage by initializing ${\Theta }_{m,1}$ as ${\Theta }_{m,1}^{\left(1\right)}={\Theta }_{m-1}^{*}$. Meanwhile, ${\Theta }_{m,2}^{\left(1\right)}$ is randomly initialized. Then, a mini-batch $D_m^{\left(k\right)}=D_{m,old}^{\left(k\right)}\cup D_{m,new}^{\left(k\right)}$ with the size $N_m^{\left(k\right)}$ is randomly sampled from the datasets $D_m$, where $D_{m,old}^{\left(k\right)}$ and $D_{m,new}^{\left(k\right)}$ denote the samples from the dataset $D_{m,old}$ and $D_{m,new}$ in the k-th epoch of the m-th stage, respectively. After this, we designed a novel loss function as

$$L({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})=L_D^{old}({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})+L_C^{new}({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}),$$

(2)

where the first term on the right is the distillation loss of recognizing old modulation modes and is defined by

$$\begin{array}{cc}\hfill & L_D^{old}({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})\hfill \\ & =-\sum _{\left(X_i,Y_i\right)\in {\mathcal{D}}_{m,old}^{\left(k\right)}}[\sum _{y=S_m+1}^{y=S_m+T_m}log\left(1-f_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\right)+\hfill \\ \hfill & \sum _{y=1}^{S_m}(f_{m-1,y}^{*}\left(X_i;{\Theta }_{m-1,1}^{*},{\Theta }_{m-1,2}^{*}\right)log{f}_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)+\hfill \\ \hfill & \left(1-f_{m-1,y}^{*}\left(X_i;{\Theta }_{m-1,1}^{*},{\Theta }_{m-1,2}^{*}\right)\right)log\left(1-f_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\right)\left)\right],\hfill \end{array}$$

(3)

and the second term of Equation (2) on the right denotes the cross-entropy loss of recognizing the new modulation modes and is defined by

$$\begin{array}{cc}\hfill & L_C^{new}({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})\hfill \\ & =-\sum _{\left(X_i,Y_i\right)\in {\mathcal{D}}_{m,new}^{\left(k\right)}}[\sum _{y=1}^{y=S_m+T_m}({\delta }_{y,Y_i}log{f}_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)+\hfill \\ \hfill & \left(1-{\delta }_{y,Y_i}\right)log{f}_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\left)\right],\hfill \end{array}$$

(4)

where $X_i$ denotes the i-th modulated signal in the dataset, while $Y_i$ is the corresponding modulation mode index. ${\delta }_{y,Y_i}$ is one for $Y_i=y$ and is zero otherwise. $f_{m,y}^{\left(k\right)}(·)$ denotes the value at the y-th port of the output-correction layer in the k-th epoch of the m-th stage, while $f_{m-1,y}^{*}(·)$ denotes the value at the y-th port of the output-correction layer in the well-trained model in the (m − 1)-th stage.

Then, ${\Theta }_{m,1}^{\left(k\right)}$ is frozen and ${\Theta }_{m,2}^{\left(k\right)}$ is updated in the direction of minimizing the loss function, i.e.,

$${\Theta }_{m,2}^{\left(k\right)}\leftarrow {\Theta }_{m,2}^{\left(k\right)}-\eta \ast {\displaystyle \frac{1}{N_m^{\left(k\right)}}}\sum _{\left(X_i,Y_i\right)\in {\mathcal{D}}^{\left(k\right)}}{\nabla }_{\left({\Theta }_{m,2}^{\left(k\right)}\right)}L({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}),$$

(5)

where $\eta$ denotes the learning rate and ${\nabla }_{\left({\Theta }_{m,2}^{\left(k\right)}\right)}L({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})$ denotes the partial derivative of the loss function with respect to ${\Theta }_{m,2}^{\left(k\right)}$.

In the second epoch (k = 2), ${\Theta }_{m,1}^{\left(2\right)}={\Theta }_{m,1}^{\left(1\right)}$ and ${\Theta }_{m,1}^{\left(2\right)}={\Theta }_{m,1}^{\left(1\right)}$ are initialized, and a mini-batch $D_m^{\left(k\right)}=D_{m,old}^{\left(k\right)}\cup D_{m,new}^{\left(k\right)}$ with size $N_m^{\left(k\right)}$ is randomly sampled and the loss is calculated with Equations (2)–(4). To this end, ${\Theta }_{m,1}^{\left(k\right)}$ is updated in the direction of minimizing the loss, i.e.,

$${\Theta }_{m,1}^{\left(k\right)}\leftarrow {\Theta }_{m,1}^{\left(k\right)}-\eta \ast {\displaystyle \frac{1}{N_m^{\left(k\right)}}}\sum _{\left(X_i,Y_i\right)\in {\mathcal{D}}^{\left(k\right)}}{\nabla }_{\left({\Theta }_{m,1}^{\left(k\right)}\right)}L({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}).$$

(6)

For k > 2, Equations (5) and (6) are used to update ${\Theta }_{m,2}^{\left(k\right)}$ and ${\Theta }_{m,1}^{\left(k\right)}$ when k is odd and even, respectively. This incremental training process continues until both ${\Theta }_{m,1}$ and ${\Theta }_{m,2}$ converge to ${\Theta }_{m,1}^{\ast }$ and ${\Theta }_{m,2}^{\ast }$, respectively, which leads to ${\Theta }_m^{\ast }=[{\Theta }_{m,1}^{\ast },{\Theta }_{m,2}^{\ast }]$.

It should be noted that the proposed IL-based modulation-recognition algorithm with the newly designed loss function from Equations (2)–(4) is different from the existing IL-based modulation-recognition algorithm with the following distillation loss, i.e.,

$$\begin{array}{cc}\hfill & L({\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)})\hfill \\ & =-\sum _{\left(X_i,Y_i\right)\in {\mathcal{D}}_m^{\left(k\right)}}[\sum _{y=S+1}^{S+T}({\delta }_{y,Y_i}log{f}_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\hfill \\ & +(1-{\delta }_{y,Y_i})log\left(1-f_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\right))\hfill \\ \hfill & +\sum _{y=1}^S(f_{m-1,y}^{\ast }\left(X_i;{\Theta }_{m-1,1}^{\ast },{\Theta }_{m-1,2}^{\ast }\right)log{f}_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\hfill \\ & +\left(1-f_{m-1,y}^{\ast }\left(X_i;{\Theta }_{m-1,1}^{\ast },{\Theta }_{m-1,2}^{\ast }\right)\right)log\left(1-f_{m,y}^{\left(k\right)}\left(X_i;{\Theta }_{m,1}^{\left(k\right)},{\Theta }_{m,2}^{\left(k\right)}\right)\right)\left)\right].\hfill \end{array}$$

(7)

By comparing Equation (7) to Equations (2)–(4), it is clear that Equation (7) ignores the fact that the well-trained DNN model in the previous stage (old DNN model) contains little knowledge of the new arriving modulation modes. Using the old DNN model to calculate the loss of recognizing the new modulation may cause significant errors and degrade the training efficiency. In contrast, our designed loss function evaluates the loss components of recognizing the old and new modulation modes separately and is able to reveal a more correct loss, which enhances the incremental training efficiency.

3.3.3. Exemplar Update

To prevent the catastrophic forgetting issue [20] of IL, i.e., the DNN model may loss the learned knowledge when learning new knowledge, the exemplar mechanism is widely used in the classical IL algorithm. In other words, after each incremental training stage, only partial sample data are stored in the memory as old data for the training of the next incremental stage. The selection criteria of the exemplars is twofold. First, both the amount and quality of the wireless signal samples subject to different modulation modes should be balanced to guarantee the performance of each incremental learning stage. Second, the memory is typically limited and the total amount of stored data should not exceed a specific threshold.

Although the number of stored signal samples can be easily guaranteed, the quality of wireless signal samples for IL is hard to correctly define. There exist two widely used exemplar-selection methods based on different understandings of the quality of wireless signal samples for IL, i.e., the random-selection algorithm and herding algorithm [18]. In particular, the random-selection algorithm randomly picks a fixed number of sample data of each modulation mode to store and drops the other data. This method implicitly assumes that the data diversity has a larger impact on the performance. The herding algorithm [18] picks the samples of a specific modulation mode according to its feature similarity to the center of all the sample features with the same modulation modes. This method emphasizes that the data feature is more important to the IL performance.

In this study, we considered three candidates for the exemplar selection, i.e., the random-selection algorithm, the herding algorithm and the SNR-based algorithm. In particular, the SNR-based exemplar-selection algorithm stores the samples with the highest SNR. Then, we identified the random exemplar-selection algorithm as the optimal one for modulation recognition through simulations in the next section, i.e., the sample dataset ${\mathcal{D}}_{m+1,old}$ in the beginning of the (m + 1)-th stage was constructed by randomly storing a fixed number of each modulation mode in ${\mathcal{D}}_m$ at the end of the m-th stage.

4. Results Discussion

In this part, we provide simulation results to verify the advantages of the proposed algorithm. In particular, we first introduce the benchmarks and then give the simulation setup, followed by the performance comparison and analysis with benchmarks.

4.1. Benchmarks

In the field of modulation recognition, the term “automatic” is used to describe a modulation-recognition model that has appropriately learned the features of different modulated signals in advance and can recognize different modulated signals without the help of a human. The term “dynamic” is typically used to describe the radio environment, in which unseen modulated signals may appear. Although dynamic modulation recognition generally belongs to the automatic modulation-recognition category while considering a special scenario in which modulated signals arrive sequentially, conventional automatic modulation-recognition algorithms cannot be directly used for fair comparisons. Instead, we adopted two widely used upper-bound and lower-bound (i.e., all-data learning and no-data learning) IL-based algorithms to verify the advantages of the proposed algorithm. Meanwhile, two existing IL-based modulation-recognition algorithms (i.e., iCaRL and BiC) were also used as benchmarks to demonstrate the superiority of the proposed algorithm.

• All-data learning: The all-data learning method, which is also known as the joint-learning method, assumes infinite memory capacity and stores all historical data for IL, allowing for using all old and new data during training, where new data are considered the old data in the next incremental stage. It is clear that this method assumes an ideal condition and the corresponding performance is typically used as the upper bound of other methods (see [12,15]).
• No-data learning: In contrast to all-data learning, no-data learning considers the worst-case scenario in which there is no memory to store any old data and only new data can be used for the incremental training. It should be noted that the no-data learning shares the same idea as sequential learning, which learns different modulation modes in different stages and only new data are utilized in each stage. It is clear that this method suffers from a severe catastrophic forgetting issue since each incremental training stage ignores the knowledge of the old data. Then, the performance of this method can be used as the lower bound of other methods.
• iCaRL: The iCaRL algorithm proposes to prevent the catastrophic forgetting issue by using the herding algorithm. In addition, the algorithm also adopts knowledge distillation technology to enable the new DNN model to learn from both old and new modulation modes. More details can be referenced in [18].
• BiC: The BiC algorithm adopts random exemplar selection to prevent the catastrophic forgetting issue. In addition, the algorithm proposes to add an output-correction layer to deal with the data imbalance between the old and new data. More details about this algorithm can be found in [19].

4.2. Simulation Setup

It should be pointed out that we considered both typical communication modulation modes and radar modulation modes in the experiments, meaning that our proposed algorithm can be used in both communication and radar scenarios, even in integrated communication and radar scenarios. Therefore, this study considered 13 modulation modes, which were binary phase shift key (BPSK), quadrature PSK (QPSK), 8PSK, continuous phase frequency shift key (CPFSK), Gaussian FSK (GFSK), DSB, single side band (SSB), linear frequency modulation (LFM), 16 quadrature amplitude modulation (16QAM), 64QAM, pulse compression modulation (PCM), wideband frequency modulation (WBFM) and PAM4 in the order of learning. The simulated channel environment was a Rice channel with a K-factor of four, and eight sinusoids were used to simulate the scattering path. The SNR ranged from −5 dB to 15 dB, with 2000 frames per mode at each SNR and 128 sampling points per frame. In addition, the bandwidth of the signal was 100 MHz, the carrier frequency was 2.4 GHz and the sampling frequency was 400 MHz. Considering the relative movement between the signal emitter and the signal receiver, the maximum Doppler shift was set to 20 KHz.

Here, the ratio of the dataset for training and validation was 7:3, and the simulation was implemented by using an Nvidia RTX 3080Ti GPU manufactured by Micro-Star in Chengdu, China. based on Pytorch 2.0.1. During the training, the batch size was set to 256 and the adopted optimizer was adaptive moment estimation (Adam) with a learning rate of 0.001. The exponential decay rates for the first and second moment estimates, i.e., $\beta$1 and $\beta$2, were 0.9 and 0.999, respectively. In the initial stage and each incremental stage, the parameters ${\Theta }_1$ and ${\Theta }_2$ were trained for five epochs each. The number of modulation modes in the initial stage was three, and two new modulation modes appeared at each incremental stage. The modulation modes in the initial stage and new modulation modes in each incremental stage are listed in

Table 1. New modulation modes in different stages.

Stage	New Modulation Modes
Initial stage	BPSK, QPSK, 8PSK
Incremental stage 1	CPFSK, GFSK
Incremental stage 2	SSB, DSB
Incremental stage 3	LFM, 16QAM
Incremental stage 4	64QAM, PCM
Incremental stage 5	WBFM, PAM4

. Without special instructions, the random exemplar-selection method was adopted and the total memory capacity was 29,400 samples. The main parameters in the simulation are summarized in

Table 2. Parameter settings.

Parameter Name	Value
Bandwidth	100 MHz
Carry frequency	2.4 GHz
Sample frequency	400 MHz
Doppler shift	20 KHz
K-factor	4
SNRS	−5:1:15 dB
Samples of each modulation mode	2000 × 21
Training and validation ratio	7:3
Epochs	10
Exemplar capacity	29,400
Batch size	256
Optimizer	Adam
Learning rate	0.001
$\beta$1	0.9
$\beta$2	0.999

.

In real-world applications, datasets may be much larger and contain more modulation modes. It is worth noting that our proposed algorithm can be applied to a dataset with any size and modulation mode. In other words, our proposed algorithm can be directly used in real-world applications. In fact, a larger dataset may enhance the training quality in the field of DL.

4.3. Performance Comparison and Analysis

Figure 3 shows the recognition performance curves of the different algorithms, where the all-data and no-data learning methods were used as the upper and lower bounds, respectively. It can be seen that the recognition performance curves of the proposed algorithm and the all-data method (the upper bound) were close, had the same trend and were higher than the other benchmarks. In particular, as the number of incremental modulation modes grew, the average recognition accuracy first increased and then decreased. This was because if the new modulation modes were easier to recognize than the old modulation modes, the recognition accuracy of the new modulation modes was higher than that of the old modulation modes, leading to a higher average recognition accuracy. Otherwise, the recognition accuracy of the new modulation modes was lower than that of the old modulation modes, leading to a lower average recognition accuracy. To be specific, the recognition accuracy gap between the proposed algorithm and the all-data (upper-bound) algorithm was no higher than 5% in the considered setup. Meanwhile, the proposed algorithm outperformed the iCaRL by up to about 9% and outperformed the BiC algorithm by up to about 7%. For fair comparisons, the performances of the different algorithms were evaluated with the same method in the same hardware and software environment and the results also coincide with our analysis.

The all-data method stores all the historical data, which naturally results in a good performance. However, this method comes with the issues of increased memory requirements and a longer training time. The average training time of the all-data method and the proposed algorithm are shown in

Table 3. Average training time of each epoch (s/epoch), where “No. of mod.” denotes number of modulation modes.

	No. of mod.	3	5	7	9	11	13
Algorithm
Proposed		4.09	6.58	6.80	6.81	6.69	6.77
All data		4.10	11.1	15.96	19.89	25.18	29.09

. It can be observed that the average training time of the proposed algorithm was much shorter than that of the all-data method as the number of new modulation modes grew. Furthermore, the average training time in each incremental stage almost remained constant and was higher than that in the initial stage. This was because the knowledge distillation in each incremental stage caused additional computing complexity compared with the training in the initial stage.

In addition, compared with the conventional methods, such as the all-data algorithm, the proposed algorithm could relieve the burden on the computational resource, which was typically reflected in the training time, as shown in Table 3. Indeed, the computational resource constraints could be further tackled by properly reducing the model complexity with ignorable recognition accuracy loss. For real-time application scenarios, the inference time of the model is the main metric and can be largely reduced with parallel computing, such as in a field-programmable gate array or an application-specific integrated circuit [21].

4.4. Recognition Performances with Different Loss Functions

As mentioned above, the proposed IL-based modulation-recognition algorithm adopted a newly designed loss function. Then, we compared the performance of the IL-based modulation-recognition algorithm with different loss functions in Figure 4. It can be observed that the performance of the IL-based modulation-recognition algorithm with the newly designed loss function was generally better than the algorithms with the no knowledge distillation and conventional knowledge distillation loss functions. This result demonstrates the advantages of the proposed loss function. In particular, the performance gap became large as the number of new modulation modes grew. This was because as the number of modulation modes grew, the recognition task became more complex and it was more challenging for the existing algorithms to distinguish different modulation modes correctly.

4.5. Recognition Performances with Different Exemplar-Selection Schemes

To determine a proper exemplar-selection scheme for the proposed IL-based modulation-recognition algorithm, we provide the performance of the proposed algorithm with different exemplar-selection schemes, including the random-selection scheme, herding algorithm and the SNR-based scheme, in Figure 5. In the figure, the performance of the random-selection strategy is higher than the other two schemes. In particular, the herding algorithm emphasizes the central features instead of diversities. In practical situations of modulation recognition, the highly dynamic radio environment leads to non-central features in signal samples and the diversity is more important than the central features for modulation-recognition tasks. Similar statements apply for the phenomenon where the random-selection scheme was better than the SNR-based scheme, which can be verified by Figure 6, in which the SNR-based scheme worked well only in the high-SNR region.

4.6. Recognition Performances with Different Memory Capacities

Using memory to store exemplars can effectively prevent the catastrophic forgetting issue, and the trade-off between the memory size and accuracy is also worth studying. Figure 7 shows the impacts of different memory sizes on the performance, where the memory size to store exemplars was set to 2, 1, 0.5, 0.3 and 0.1 times the 29,400 samples. Additionally, the all-data and no-data methods were still used as the upper and lower bounds. It can be seen that as the size of the exemplar increased, the performance of the IL-based modulation-recognition algorithm improved, demonstrating that storing more historical samples could effectively deal with the catastrophic forgetting issue. It is worth noting that even with only 2940 (0.1 times) samples in the exemplar, the accuracy still exceeded 70% when recognizing 13 modulation modes, which was significantly higher than the accuracy of the no-data method.

5. Conclusions

This study investigated the dynamic modulation recognition in a practical situation in which the signal samples with new modulation modes could appear sequentially and it was demanding to completely collect the signal sample of all the modulation modes in advance. To tackle this issue, we proposed a novel IL-based modulation-recognition method, which consists of an initial stage and multiple incremental stages. The main novelty of the proposed method lies in the new loss function design in each incremental stage, which combines the distillation loss of recognizing old modulation modes and the entropy loss of recognizing new modulation modes. With the proposed algorithm, the knowledge of the signal samples with new modulation modes could be efficiently learned while retaining the knowledge learned in the previous stage. The simulation results demonstrated that the proposed algorithm could achieve a recognition accuracy close to the upper bound with a much lower computing time and it outperformed the existing IL-based benchmarks.

It is worth pointing out that thanks to the recent advances in graph processing, the graph-based modulation-recognition algorithm that utilizes the spatial–spectral features of modulated signals is a promising research direction. Nevertheless, this kind of algorithm typically suffers from longer training and inference times. In our future work, we will leverage the advances in the graph-processing field and develop a lightweight graph-based modulation-recognition algorithm.