Deep Learning-Based Multi-Feature Fusion for Communication and Radar Signal Sensing

Abstract

Recent years witness the rapid development of communication and radar technologies, and many transmitters are equipped with both communication and radar functionalities. To keep track of the working state of a target dual-functional transmitter, it is crucial to sense the modulation mode of the emitted signals. In this paper, we propose a deep learning-based intelligent modulation sensing technique for dual-functional transmitters. Different from existing modulation sensing methods, which usually focus on communication signals, we take both communication and radar signals into consideration. Typically, the dominant features of communication signals lie in the time domain, while those of radar signals lie in both time and frequency domains. To enhance the sensing accuracy, we first exploit real and complex value convolution operations to extract both time-domain and frequency-domain features of emitted signals from the target transmitter. Then, we fuse the extracted features by assigning proper weights with the attention mechanism. Simulation results reveal that the proposed technique can improve the sensing accuracy by up to 4% on average compared with benchmarks.

1. Introduction

In recent years, with the rapid development of communication and radar technologies, many transmitters are equipped with both communication and radar functionalities [1,2]. To keep track of the working state of a target dual-functional transmitter, it is crucial to sense the modulation mode of the emitted signal. Concretely, for a high-rate data transmission purpose, a higher order modulation mode is preferred [3], such as quadrature amplitude modulation 16 (QAM16). For the target detection with a longer distance, the pulse compression modulation (PCM) is a better choice than linear frequency modulation (LFM), due to PCM signals having a higher range resolution and a higher signal-to-noise ratio (SNR) than LFM signals [4]. Therefore, it is possible to identify the working state of a target dual-functional transmitter by sensing the modulation mode of emitted signals.

Modulation sensing is adopted to enhance the communication efficiency in a wide range of cooperative and non-cooperative scenarios. To be specific, modulation sensing can be used to enhance the communication efficiency in multi-point cooperative transmission scenarios and can also be used to understand the signals of non-cooperative transceivers for competition purpose, e.g., cognitive radio and electronic warfare [5,6,7]. With the radio environment becoming increasingly complicated, how to effectively and accurately sense the modulation modes of the emitted signals from target transceivers has become a hot research direction. In general, modulation sensing techniques mainly include traditional and deep learning (DL) methods. Traditional methods could be roughly divided into two groups: likelihood-based (LB) and feature-based (FB) [8]. In particular, LB methods transform a modulation sensing problem into a multi-hypothesis testing problem and correctly identify corresponding modulation modes by setting appropriate discriminant thresholds. The disadvantages of these methods lie in that it is difficult to determine proper discriminant thresholds and the performance is typically sensitive to the radio environment [9]; meanwhile, the computational complexity is high. FB methods extract some physical or statistical characteristics, such as cyclic moments or higher order cumulants [10], and adopt proper classifiers, such as support vector machines or K-nearest neighbor, to sense the modulation modes of emitted signals. In practical applications, these methods heavily depend on manual analysis [11], and the performance is unacceptable when modulated signals are complicated [12].

Recently, data-driven DL methods have attracted a lot of attention of the researchers in academia [13,14,15,16,17] and strongly promoted the development of modulation sensing technology. In particular, O’Shea et al. propose a convolution neural network (CNN) architecture and demonstrate that DL methods outperform several traditional methods in terms of sensing accuracy and inference speed [18]. Jakob et al. propose a complex value convolution (CVC) method to extract complex feature of signals, which is proved to improve the sensing accuracy [19]. Lin et al. introduce the frequency sequence and the time-frequency attention (TFA) mechanism into CNN to further boost the sensing accuracy [20]. Godwin et al. adopt multiple skip connections in a modified CNN architecture, which is proved to improve the stability of the network [21]. Ayman et al. propose a modulation recognition method leveraging temporal correlation of features, which integrates CNN and LSTM models [22]. Zhang et al. suppose that different features are complementary, and propose a CNN based multi-delay feature fusion scheme which performes well at low SNRs [23]. In general, DL methods typically consist of two successive stages. In the first stage, amounts of representations of received signals are fed into DL methods to train the neural networks, which is time-consuming but conducted off-line. In the second stage, well-trained neural networks are utilized to sense the modulation modes of received signals, which is conducted online and time-saving. During training, massive simulation data required in DL methods are readily available. Compared with traditional methods, DL methods typically have a lower computational complexity but can achieve better performance. Meanwhile, DL methods avoid manual feature selections and can be applied in more complicated scenarios.

We notice that most existing works focus on communication signals while ignoring radar signals and cannot be applied in sensing the working state of dual-functional transmitters. Meanwhile, the main features of communication signals typically lie in the time domain, while those of radar signals lie in both time and frequency domains. Thus, the modulation sensing algorithm for dual-functional transmitters needs to be carefully designed to accurately identify the working state of a dual-functional target transmitter. In this paper, we propose a DL-based intelligent modulation sensing technique. In particular, to fully and succinctly represent the emitted signals from the target transmitter, we first exploit real value convolution (RVC) and CVC nerual networks to extract multiple features from both time and frequency domains. Further, we fuse the extracted features by assigning proper weights with the attention mechanism. Simulation results reveal that the proposed technique can improve the sensing accuracy by up to 4% on average compared with benchmarks.

The structure of this paper is as follows: Section 1 introduces the background and applications of modulation sensing technology and compares DL algorithms with traditional methods; Section 2 describes two readily available signal representations in both time and frequency domains for the proposed technique; Section 3 introduces the structure and theory of the proposed algorithm in detail and analyzes the time and space complexity; Section 4 analyzes the performance of the proposed algorithm by conducting simulation experiments; and the conclusion is given in Section 5.

2. Signal Representation

The radio signal from a dual-functional target transmitter can be typically represented in both time and frequency domains. In particular, the time-domain signal conveys the variation pattern of the signal amplitude in terms of the time, while the frequency-domain signal can directly reflect the energy distribution in terms of frequency. There exists complementary information between different representations of signals [7], meaning that inputting both time and frequency domain sequences could extract the complementary features between both domains, thus enhancing the sensing performance. In this section, we introduce two readily available representations of a received signal in both time and frequency domains.

2.1. Time-Domain Representation

A received complex signal in the time domain can be represented as

$$r\left(t\right)=x\left(t\right)\ast h\left(t\right)+n\left(t\right),$$

(1)

where $x\left(t\right)$ represents the modulated complex signal emitted from the target dual-functional transmitter, $h\left(t\right)$ represents the channel impulse response, $\ast$ represents the convolution operation, and $n\left(t\right)$ represents the additive complex noise at the receiver. Upon receiving the signal, the receiver converts it into a baseband discrete form with an analog-to-digital converter. Without abusing the notations, we denote the discrete form of the received complex signal as $r\left[n\right]$, which is usually decomposed into two orthogonal components:

$$r\left[n\right]=r_I\left[n\right]+j{r}_Q\left[n\right],$$

(2)

where n is the sampling index, j is the imaginary unit, and $j^2=-1$, $r_I\left[n\right]$ and $r_Q\left[n\right]$ represent the in-phase (I) and quadrature (Q) components of the receiving complex signal, respectively. Since the IQ sequence is the raw data obtained at the receiver, it is most readily available as the input for DL methods.

2.2. Frequency-Domain Representation

A typical method to obtain the frequency-domain sequence of a complex signal is to conduct the discrete fourier transform (DFT) operation to the corresponding time-domain sequence, i.e.,

$$\begin{array}{cc}\hfill R\left[k\right]& =\mathrm{DFT}\left(r\left[n\right]\right)=\sum _{n=0}^{N-1}r\left[n\right]W_N^{kn},k=0,1,\cdots ,N-1,\hfill \end{array}$$

(3)

where N is the DFT length, $W_N^{kn}=e^{-j\frac{2\pi kn}{N}}$, and $R\left[k\right]$ is the complex sequence in frequency domain.

It is noted that DFT has a variety of fast algorithms, collectively known as fast fourier transform (FFT), which can effectively reduce the signal processing complexity. The FFT algorithm is widely used in both industry and academia, and there are many mature and simple implementation schemes. For the convenience of subsequent processing, we adopt the FFT the same length as the time-domain sequence.

3. Proposed Modulation Sensing Algorithm

As mentioned above, time-domain and frequency-domain representations of signals convey different features; meanwhile, RVC and CVC can extract different features from time-domain and frequency-domain signals. In particular, RVC can extract independent features of the real and imaginary parts of signals, while CVC can extract interaction features between both parts [19,24]. In this section, we propose an intelligent modulation sensing algorithm by first extracting multiple features of time-domain and frequency-domain signals and then fusing the extracted features with the attention mechanism. Finally, a classifier is used to map the fused features to the sensing result. In the following section, we will first provide the algorithm structure and then elaborate on the algorithm designs.

3.1. Algorithm Structure

As shown in Figure 1, the proposed algorithm has three main modules including feature extraction, feature fusion, and classifier. In particular, the feature extraction module transforms the input sequences into multiple feature flows by conducting both CVC and RVC operations, the feature fusion module deeply fuses multiple feature flows into one fused feature flow with the attention mechanism, and the classifier module maps the fused features to the sensing results.

3.2. Feature Extraction

The time-domain sequence, denoted as T in Figure 1, could be expressed as Equation (2), while the frequency-domain sequence, denoted as F, could be obtained by conducting an FFT operation. Since the methods of processing T and F are identical, we only elaborate on the procedure of processing T for simplicity.

As T is a complex vector, the CVC operation [25] can be expressed as

$$\begin{array}{c}\hfill T\ast h=(T_r+j{T}_i)\ast (h_r+j{h}_i)=(T_r\ast h_r-T_i\ast h_i)+j(T_r\ast h_i+T_i\ast h_r),\end{array}$$

(4)

where h is the complex convolution filter, $h_r$ and $h_i$ represent real and imaginary parts of h, and $T_r$ and $T_i$ represent real and imaginary parts of T, respectively.

As neural networks only support the calculation of real values [26], a complex value sequence can be represented as real and imaginary parts. Then, the CVC operation can be rewritten as

$$\begin{array}{c}\hfill T\ast h=\left[\begin{array}{c}Re(T\ast h)\\ Im(T\ast h)\end{array}\right]=\left[\begin{array}{cc}\hfill h_r& \hfill -h_i\\ \hfill h_i& \hfill h_r\end{array}\right]\ast \left[\begin{array}{c}{T}_r\hfill \\ T_i\hfill \end{array}\right].\end{array}$$

(5)

The CVC operation needs two real value convolution filters $h_r$ and $h_i$, meaning that a CVC layer in the neural network could be implemented by two RVC layers. Without special instructions, 1 × 3-sized filters are used in all convolution layers in the proposed algorithm.

By performing RVC and CVC in terms of T and F, four feature flows can be extracted, which are denoted as $T_R$, $T_C$, $F_R$, and $F_C$, respectively, in Figure 1. The dimension of each feature flow is C × 2 × N, where C denotes the number of channels.

3.3. Feature Fusion

It should be noted that there exits some complementary information among different feature flows, which may have different contributions to the sensing performance. In other words, different feature flows need to be carefully fused according to their contributions to the sensing result. Four feature flows in Figure 1 have distinct effects on the sensing accuracy, but it is challenging to quantify these effects and obtain the optimal weights with closed forms. So, motivated by the successful application in multi-modal feature fusion [24], we also adopt the attention mechanism to automatically learn the optimal weights to enhance the performance.

The overall structure of the feature fusion module is shown in Figure 2. In particular, four feature flows are first concatenated in the dimension of channel and then input into the feature fusion module, i.e.,

$$CF=\left[\begin{array}{c}{T}_R,T_C,F_R,F_C\end{array}\right],$$

(6)

where $CF$ denotes the concatenated features, and the dimension is 4C × 2 × N.

Upon receiving $CF$, the feature fusion module processes it with a depthwise separable convolution (DSC) layer [27], which aims to reduce parameters in the neural network. Then, the output of DSC layer is fed into an attention block, which distributes proper weights to different features. In particular, a DSC layer consists of a channel-wise convolution layer and a 1 × 1 convolution layer, which is used to change the dimension of the feature map. In the paper, we reduce the channel dimension of features to C. Here, a skip connection is introduced to prevent gradient vanishing.

The structure of the attention block is shown in Figure 3. In particular, the input features are first processed by a squeeze layer [28] and an activation (Sigmoid) layer to calculate proper weights of different feature channels. The meaning of proper weights is that the weights could help the neural network to focus on significant features and ignore interference, which are obtained automatically rather than artificially assigned. In this paper, the channel attention mechanism is adopted to assign weights according to the importance of different features, thus improving sensing accuracy. In particular, the squeeze layer is to calculate the global average value of the features in the dimension of the channel, whose dimension is equal to the channel dimension of the input feature map, i.e., C. It can be expressed as

$$Y_i=\frac{1}{W\times H}\sum _{j=1}^W\sum _{k=1}^H{X}_{i,j,k},i=1,\cdots ,C,$$

(7)

where X denotes the input feature map, and the subscripts represent indexes of the channel width and height, respectively. The Sigmoid function could be expressed as

$$y=\frac{1}{1+e^{-x}}$$

(8)

Then, the attention block outputs the fused features after multiplying pixels in each channel by the corresponding weight.

Figure 3. Attention block structure. Dark blue squares represent the input features, other colors represent corresponding weights of different channels.

Figure 3. Attention block structure. Dark blue squares represent the input features, other colors represent corresponding weights of different channels.

3.4. Classifier

Upon obtaining the fused feature flow, the residual blocks and fully connected (FC) layers are designed to further extract the dominant features contained in the fused feature flow and map the dominant feature to the sensing result.

3.4.1. Residual Block

The structure of the residual block is shown in Figure 4a, in which the input is processed by two residual units [29] and a maxpool layer in a sequential order. In particular, the kernel size of the maxpool layer is set to 2 × 2, and the stride is set to $(2,2)$ to prevent overfitting, which can be expressed as

$$y_h=Max(x_{0,2h},x_{0,2h+1},x_{1,2h},x_{1,2h+1}),$$

(9)

where the first dimension of the subscript represents the index of the width of the feature map, and the second dimension represents the index of the height. Since the width of input feature map is 2, the first dimension of the output feature map is ignored.

The structure of residual unit is shown in Figure 4b, in which $\mathit{FF}$ denotes the input features and $\mathit{f}\left(\mathit{FF}\right)$ denotes the result by processing $\mathit{FF}$ with two cascaded RVC layers.

Here, a skip connection is also adopted to prevent a vanishing gradient in each residual unit. Then, the output $F{F}^{\prime }$ of each residual unit can be expressed as

$$F{F}^{\prime }=FF+f\left(FF\right)$$

(10)

3.4.2. FC Layer

The feature map ultimately needs to be mapped to the correct modulation mode, so a flatten layer is needed after the residual block. In particular, the feature map tends to be multi-dimensional, while the input of the FC layer is a vector. The function of the flatten layer is to arrange pixels of the multi-dimensional feature map into a vector in a certain order, which may be different for different DL frameworks. Then, several FC layers are adopted to change the number of neurons to map to the probability that the signal belongs to each modulation mode. The equation of FC layer could be expressed as

$$y=f_{act}(wx+b),$$

(11)

where x is the input feature vector, w and b denote weights and bias, respectively, and $f_{act}$ denotes the activation function. Except for the output layer which uses Softmax, other FC layers in the proposed algorithm adopt Selu as the activation function, which is expressed as

$$f\left(x\right)=\lambda \left\{\begin{array}{cc}x\hfill & \mathrm{if}x>0\hfill \\ \alpha \left(e^x-1\right)\hfill & \mathrm{if}x\le 0\hfill \end{array}\right.$$

(12)

where $\lambda =$ 1.0507, $\alpha =$ 1.6733.

We optimize the number of residual blocks [29], the number of FC layers, and the number of neurons in each layer [24] through extensive cross-validations. In particular, the number of residual blocks is one, while the numbers of neurons in the FC layers are 128 and 64, respectively.

3.5. Complexity Analysis

The time and space complexity of the proposed algorithm is dominated by the calculation and parameters in RVC, CVC, DSC, and FC layers.

3.5.1. Time Complexity

Time complexity is typically measured by multiply–accumulate operations [30]. The time complexity of each convolution layer depends on the size and number of convolution kernels. In particular, to obtain a pixel in the output feature map, the number of multiply–accumulate operations in each channel is equal to the size of the convolution kernel. Therefore, the time complexity of RVC operations could be expressed as

$$\mathcal{O}\left(\sum _l{m}_l^h{m}_l^w{k}_l^h{k}_l^w{c}_{l-1}{c}_l\right),$$

(13)

where l denotes the index of the RVC layer, $m_l^h$ and $m_l^w$ denote the height and width of the output feature map, $k_l^h$ and $k_l^w$ denote the height and width of the convolution kernel, and $c_l$ denotes the channel number of the l-th convolution layer, respectively.

As a CVC layer is implemented by two RVC layers, the time complexity of CVC is twice of that in Equation (13). Meanwhile, the time complexity of the DSC layer consists of two parts: channel-wise convolution and 1 × 1 convolution, i.e.,

$$\begin{array}{c}\hfill \mathcal{O}\left(m_l^h{m}_l^w{k}_l^h{k}_l^w{c}_l+m_l^h{m}_l^w{c}_l{c}_{l+1}\right)\end{array}$$

(14)

The calculation of the FC layer, i.e., Equation (11), is equivalent to multiplying weights matrix w with input vector x and then adding bias vector b. Therefore, the time complexity of FC layers could be obtained by summing up the calculations in each layer, i.e.,

$$\mathcal{O}\left(\sum _l{n}_{l-1}{n}_l\right),$$

(15)

where $n_l$ denotes the number of neurons in the l-th FC layer.

Therefore, the overall time complexity of the proposed algorithm can be obtained by summing up the time complexity of each module, i.e.,

$$\mathcal{O}\left(3\sum _l{m}_l^h{m}_l^w{k}_l^h{k}_l^w{c}_{l-1}{c}_l+\sum _l{n}_{l-1}{n}_l+(m_l^h{m}_l^w{k}_l^h{k}_l^w{c}_l+m_l^h{m}_l^w{c}_l{c}_{l+1})\right)$$

(16)

3.5.2. Space Complexity

Space complexity is typically measured by the number of trainable parameters, which are mainly distributed in RCV, CVC, DSC, and FC layers. In particular, the trainable parameters of RVC consist of weights and bias, and therefore, the space complexity of RVC can be expressed as

$$\mathcal{O}\left(\sum _l(k_l^h{k}_l^w{c}_{l-1}{c}_l+c_l)\right)$$

(17)

As a CVC layer is implemented by two RVC layers, the space complexity of CVC is twice of that in Equation (17); meanwhile, the space complexity of DSC is

$$\mathcal{O}\left(k_l^h{k}_l^w{c}_l+c_l{c}_{l+1}+c_l+c_{l+1}\right)$$

(18)

In the FC layer, the weight matrix and bias vector are both trainable variables, so the space complexity of FC layers could be expressed as

$$\mathcal{O}\left(\sum _l(n_{l-1}{n}_l+n_l)\right)$$

(19)

Therefore, the overall space complexity of the proposed algorithm is

$$\mathcal{O}\left(3\sum _l(k_l^h{k}_l^w{c}_{l-1}{c}_l+c_l)+\sum _l(n_{l-1}{n}_l+n_l)+k_l^h{k}_l^w{c}_l+c_l{c}_{l+1}+c_l+c_{l+1}\right)$$

(20)

4. Simulation Results and Analysis

4.1. Dataset and Algorithm Architecture

Although there exist some public datasets on modulation sensing, they only include communication signals and ignore radar signals. To build the dataset for fair comparisons, we refer to communication signals of the public dataset RML2016.10a in [18] and radar signals in [4]. Meanwhile, we also refer to the corresponding parameter settings and signal generation processes. In particular, we consider eight typical modulation modes of communication signals including binary phase shift keying (BPSK), quadrature PSK (QPSK), 8PSK, continuous phase frequency shift keying (CPFSK), Gaussian FSK (GFSK), QAM16, QAM64, and analog modulation (AM), as well as two representative modulation modes of radar signals including LFM and PCM. The simulated channel environment considers two cases, Gaussian channel and Rice channel, with SNRs ranging from −5 dB to 15 dB. Thereinto, K-factor of the Rice channel is 4, and 8 sinusoids are used to model the fading process. By considering the relative motion between the target dual-functional transmitter and the receiver, we set the maximum value of Doppler shift to be 20 kHz. The bandwidth of the signal is 100 MHz, the carrier frequency is 2.4 GHz, and the up-sample ratio is 8. Generated signals have 1024 sampling points per frame, and 128 continuous points are randomly selected in the dataset.

The dataset includes 2000 frames with 128 samples of each modulation type per SNR in each channel, and the ratio of the dataset for training, validation, and testing is 3:3:4. In addition, our simulation experiments are implemented by using Nvidia RTX 3080Ti GPU based on Pytorch 2.0.1. Through extensive cross-validations and end-to-end supervised training, we optimize the hyperparameters as follows: the optimizer adopted is Adam, whose initial learning rate is 0.001, and the loss function is categorical crossentropy; epochs are set to 30, while the training process would be terminated if the validation loss fails to decrease within 5 epochs. Meanwhile, the model with minimum validation loss is chosen for sensing the modulation modes of testing signals, because it exhibits the best generalization performance. The architecture of the proposed algorithm in Figure 1 is summarized in

Table 1. Model layout.

Layer	Output Dimensions
Input	2 × 128
CVC/RVC	16 × 2 × 128
Concatenate	64 × 2 × 128
DSC	16 × 2 × 128
Attention Block	16 × 2 × 128
Residual Block	16 × 1 × 64
FC/Selu	128
FC/Selu	64
Output/Softmax	10

. It is known that a larger scale neural network has a stronger learning and representation ability. In real-world scenarios with large-scale datasets, we can still use the framework in Figure 1 and balance the scalability and efficiency of the proposed algorithm by adapting the number of layers or the number of neurons in each layer.

4.2. Ablation Experiment

As mentioned previously, our hypothesis is that complementary features can be extracted from time-domain sequence T and frequency-domain sequence F; meanwhile, CVC and RVC could extract different features from the same sequence. To demonstrate this idea, we design an ablation experiment [31]. In the experiment, an input sequence (T or F), feature extraction operation (CVC or RVC), or attention block is removed from the proposed algorithm. The number of filters in the feature extraction module of each ablated algorithm is doubled to ensure that both numbers of trainable parameters and neurons are close to the proposed algorithm. The results in the Gaussian channel are shown in Figure 5.

From Figure 5, the sensing accuracy of the proposed algorithm increases rapidly from 71.2% to 99.1% as SNR grows from $-5$ dB to around 8 dB and then gradually converges to 1 as the SNR further grows. The sensing accuracy of the ablated algorithm using only T (proposed without F) as the input converges to around 94.2%, while the sensing accuracy of the ablated algorithm using only F merely converges to 85.6%. It can be concluded that the time-domain features are more important to improve the sensing accuracy compared with frequency-domain features. This is because we adopt eight up-samples to process each symbol in the time domain, and there exist time correlations among different sample points, making it easier for the neural network to extract features and enhance the sensing accuracy, while there is little correlation in the frequency-domain signals. Meanwhile, the sensing accuracy can be further improved by properly fusing multiple features in both time and frequency domains. Similarly, we can also conclude that the CVC operation is able to extract more information than the RVC operation, and the sensing accuracy can be further enhanced by properly combining CVC and RVC operations. In addition, the sensing accuracy without the attention block decreases around 2% compared with the proposed algorithm, indicating that the attention mechanism can help the neural network pay more attention to important features, thus enhancing the performance.

4.3. Algorithms Comparison

To demonstrate the advantages of the proposed algorithm, we compare the sensing accuracy of the proposed algorithm with benchmarks, which are widely adopted in modulation sensing [7,24], including dual-stream CNN-LSTM [9], TFA CNN [20], CNN-LSTM [32], and improved CNN [33]. In particular, dual-stream CNN-LSTM takes IQ and amplitude-phase sequences as inputs and utilizes CNN and LSTM layers to sense the modulation modes. TFA CNN adopts multiple attention modules to explore features in spectrogram images, while CNN-LSTM sensing corresponding modulation modes by using CNN and LSTM layers to extract features in IQ sequences and fourth order cumulants. In addition, improved CNN processes the IQ sequence by adopting convolution and dense layers. The numbers of trainable parameters of algorithms in this experiment are shown in

Table 2. Algorithm parameters.

Algorithm	Parameters
Dual-stream CNN-LSTM	613 K
TFA CNN	104 K
CNN-LSTM	467 K
improved CNN	159 K
Proposed	145 K

, and the convergence curves in the Gaussian channel are shown in Figure 6.

From Figure 6, it can be observed that the sensing accuracy of the improved CNN increases gradually from around 65% to 86% after 30 epochs. The sensing accuracy of dual-stream CNN-LSTM, TFA, and CNN-LSTM is comparable, increasing from around 67% to around 90% after 30 epochs; although, there exist some fluctuations during the training phases, and it does not increase any more. Meanwhile, we observe that the sensing accuracy of the proposed algorithm can converge to around 94% after only five epochs. Therefore, the proposed algorithm is superior to the benchmarks in terms of both convergence rate and converged sensing accuracy. Moreover, the inference time of each algorithm is shown in

Table 3. Inference time of different algorithms.

Algorithm	Inference Time (µs/Frame)
Dual-stream CNN-LSTM	78.4
TFA CNN	21.8
CNN-LSTM	47.4
improved CNN	19.8
Proposed	14.7

. The inference speed of the proposed algorithm can reach 14.7 µs per frame, which is much faster than benchmarks. It is reasonable since this algorithm has fewer parameters and can be computed in parallel in the feature extraction module, while the algorithms with LSTM layers have slower inference speed due to time dependence.

Figure 7 and Figure 8 illustrate the sensing accuracy of different algorithms with different SNRs in the Gaussian and Rice channels, respectively. In general, the proposed algorithm performs better than benchmarks in any SNR condition, which is between −5 dB and 15 dB. From Figure 7, in the low and medium SNR regions, i.e., SNR is between −5 dB and 5 dB, the sensing accuracy of the proposed algorithm is around 5% higher than those of CNN-LSTM and improved CNN, and is slightly higher than those of dual-stream CNN-LSTM and TFA. In the high SNR region, i.e., the SNR is between 5 dB and 15 dB, the sensing accuracy of the proposed algorithm approaches 1 quickly and is around 8% higher than that of the improved CNN and is around 4% higher than those of the dual-stream CNN-LSTM, CNN-LSTM, and TFA algorithms. In fact, this is reasonable since the proposed algorithm can extract more features and fuse them in a better way to enhance the sensing accuracy compared with benchmarks. In the low SNR region, the signal is seriously impaired by the noise, resulting in ambiguous features for the neural network to learn, so the advantages of the proposed algorithm are not as obvious as those in the high SNR region.

From Figure 8, it can be observed that in the Rice channel, the sensing performance curves of all algorithms have a similar trend as those in the Gaussian channel. The sensing accuracy of the proposed algorithm is still about 4% higher than the benchmarks. Nevertheless, due to the impairment of the Rice channel on the signal, the sensing performance of all algorithms in the low and median SNR regions (lower than 10 dB) slightly degrades, e.g., about 1.5% lower on average.

4.4. Length Comparison

Figure 9 shows the performance with different sample points of the proposed algorithm. It can be observed from the figure that a larger number of samples generally leads to a higher sensing accuracy. This is because as the sequence length increases, the information of the signals becomes more abundant and features become more obvious, which are easier for the neural network to extract.

4.5. Confusion Matrices

To study the sensing accuracy of different modulation modes, we provide confusion matrices of different algorithms when the SNR is 5 dB in Figure 10. Generally, the proposed algorithm can accurately distinguish communication and radar signals and has almost perfect sensing capability in terms of BPSK, CPFSK, GFSK, AM, PCM, and LFP when the SNR is 5 dB. Compared with benchmarks, the proposed algorithm has a higher sensing accuracy for QPSK, 8PSK, QAM16, QAM64, PCM, and LFM signals. Especially for the QAM signals, it is challenging for benchmarks to distinguish QAM16 and QAM64 signals. The proposed algorithm can improve the sensing accuracy of QAM16 by 5%, 9%, 8%, and 32%, respectively, compared with dual-stream CNN-LSTM, TFA CNN, CNN-LSTM, and improved CNN. In addition, both QAM16 and QAM64 are high order modulation modes, and the SNR required to distinguish them is higher than the SNRs of other modulation modes. Therefore, the major challenge lies in the correct sensing of high-order modulation modes in the low SNR region. Generally, the proposed algorithm outperforms the benchmarks for the sensing of both communication and radar signals.

5. Conclusions

In this paper, we first introduce the background and applications of modulation sensing technology and analyze and compare DL methods with traditional methods. Therewith, two readily available representations of signals in both time and frequency domains are introduced. Then, a DL-based multi-feature fusion technique is proposed to sense the modulation modes to acquire the working state of dual-functional transmitters with both communication and radar capabilities. In particular, the proposed technique first conducts both RVC and CVC operations to time-domain and frequency-domain sequences of received signals to explore multiple features. Then, the attention mechanism is leveraged to adaptively fuse the features by assigning proper weights. Finally, a classifier is adopted to sense different modulation modes based on fused features. Simulation results reveal that the proposed technique can generally improve the sensing accuracy by up to 4% on average compared with benchmarks.