A Method for Detecting Anomalies in an Electromagnetic Environment Situation Using a Dual-Branch Prediction Network

Abstract

Electromagnetic environment situation anomaly detection is a prerequisite for electromagnetic threat level assessment, and its research is of great practical value. However, because of the complexity of the electromagnetic environment, electromagnetic environment situation anomaly detection is not efficient. Therefore, we propose a dual-branch prediction network-based electromagnetic environment situation anomaly detection method to predict the future and achieve anomaly detection by fusing different development characteristics of electromagnetic environment situations learned by other branches. We extract the electromagnetic environment situation state and trend features using the manual feature extraction module and mine the electromagnetic environment situation in-depth data distribution features using ConvLSTM, improve the dynamic time regularization model according to the physical characteristics of electromagnetic space, and then provide the anomaly detection method. We experimentally demonstrate the effectiveness of the proposed method in electromagnetic environment situation prediction and anomaly detection accuracy.

1. Introduction

With the progress of science and technology, radio communication equipment has been more and more widely used, and the electromagnetic environment has become increasingly complex. The electromagnetic environment situation (EMES) is the current state and development trend of the electromagnetic environment in a specific area, a specific period, and a specific frequency band [1]. Electromagnetic environment situation anomaly detection (EMESAD) refers to detecting problems in electromagnetic environment data that do not conform to the expected behavior pattern [2,3]. If there is a significant change in electromagnetic activity in a specific area, time, or frequency band, and the magnitude of the change exceeds a given threshold, the electromagnetic environment can be considered abnormal. The detection of anomalies provides technical support for radio spectrum monitoring and maintenance of radio communication order services on the civil side. It guides our reconnaissance and interference on the military side, providing an essential basis for battlefield posture judgment and threat level assessment. Therefore, the study of abnormal electromagnetic environment situations and their detection method has important theoretical significance and practical military value.

Military concepts have been constantly updated in recent years, military requirements for electromagnetic environment applications have increased, and scholars have studied electromagnetic environments more deeply. However, the current research on the complexity of the electromagnetic environment is more prominent, the research on electromagnetic environment situation is less prominent, and the research on the anomaly detection of electromagnetic environment situation is even less prominent. The existing related studies focus on electromagnetic environment situation generation [4,5,6], while there are only a few reports on electromagnetic environment situation anomaly detection. The literature [7] implemented anomaly detection of electromagnetic environment situations based on fuzzy neural networks, but the ANFIS algorithm relies on rules and has insufficient self-learning capability. The literature [8] studied the analysis and judgment of electromagnetic situation based on deep learning and simulated red-blue confrontation. However, the article did not have enough depth for mining the electromagnetic environment, and the algorithm was not efficient. Based on the above status quo, we apply the deep learning method to the anomaly detection of electromagnetic environment situations and study the anomaly detection of electromagnetic environment situations based on prediction.

Prediction-based anomaly detection for electromagnetic environment situations consists of two main components: a deep-learning-based prediction technique and an anomaly detection technique. Deep-learning-based prediction techniques are currently a popular approach in anomaly detection. The literature [9,10] used prediction; the former applied RNN to anomaly detection of dam cracks, and the latter used LSTM to predict the changes in meteorological data and indoor temperature and achieved better prediction and anomaly detection results. The literature [11] used a framework combining CNN and LSTM to detect spectrum usage anomalies in a data-driven manner and proved the technique’s feasibility in the spectrum domain. The literature [12] improved the basic framework of CNN-LSTM by replacing CNN with GAN and LSTM with bi-directional LSTM, which improved the accuracy and overcame the drawbacks of weak modeling and the long dependency time of LSTM. However, the above deep learning methods extract data features of samples directly without considering the possible physical significance and physical change patterns of training samples, and the extracted features are incomplete. The literature [13,14] improved the above techniques by fully considering the possible physical characteristics of the model and the impact of physical features on the target. The literature [13] implements the prediction function through a feature fusion strategy that actively gives the network module-specific functions to extract the motion features of the temporal images, but only explores the surface information of the images without profoundly exploring the intrinsic physical meaning of the images. The literature [14] directly divides the deep learning network into two independent branches to extract physical features and data features and then fuses the two for prediction and achieves better results. However, these two studies’ physical feature extraction methods are targeted for general video prediction tasks. At the same time, the electromagnetic environment situation has its unique physical characteristics, so we also need to study the deep feature extraction methods suitable for electromagnetic environment situation prediction.

Anomaly detection technology is mainly realized by anomaly assessment indicators and setting appropriate thresholds. The accuracy of anomaly detection depends primarily on the accuracy of the prediction and the accuracy of the assessment indicators. Anomalies in the electromagnetic environment situation are detected by comparing the predicted and actual values, and the evaluation indicators should consider the state and trend of the electromagnetic environment. The complexity and volatility of the electromagnetic field can lead to a generated electromagnetic environment situation with fluctuations in the location of the radiation source and power fluctuations. The most important thing for the evaluation metrics is to allow for the existence of such fluctuations. Traditional image comparison methods such as MSE and MAE are hard metrics based on comparisons between each pixel point and do not consider the similarity of local regions. The SSIM algorithm is based on local statistics that consider the spatial structure, but again lacks localization comparisons of location points within the region [15]. Therefore, we need a more flexible method for anomaly assessment, and dynamic time warping (DTW) is a loose measure of similarity between two time series of different lengths. The literature [16] used dynamic time warping (DTW) to align the predicted and actual series temporally, and the fluctuation of the two sequences within a particular position does not affect the judgment of the similarity of the series. The literature [17] combined DTW with time as a loss function to ensure efficient prediction. However, not many studies have applied DTW to image comparison, mainly by transforming two-dimensional data into one-dimensional data for comparison using dimensionality reduction. The literature [18] uses a reduced-dimensional DTW to match images and stitch them. The images are downscaled using wavelet transform to compare the similarity between sequences based on the DTW in terms of eigenvalues. For this reason, we can understand that dynamic time warping (DTW) does not affect the similarity between two sequences with inevitable fluctuations and is suitable for comparing the similarity between two electromagnetic environment situations. However, the electromagnetic environment situation based on the electromagnetic spectrum map is an image sequence, so the DTW-based image similarity comparison method remains to be studied.

In summary, electromagnetic environment situation anomaly detection must first master its activity pattern, i.e., obtain its state and trend characteristics through training, obtain predicted values, and then judge whether it is anomalous based on the difference between the test and predicted values. Considering the current research status, we have conducted an in-depth study on the prediction and anomaly detection techniques of electromagnetic environment situations.

The main contributions of this paper are as follows:

• We propose a deep learning prediction model for electromagnetic environment situations based on a dual-branch prediction network (EMESNet). We construct an artificial feature extraction module (EMESCell) that conforms to the electromagnetic spatial distribution based on the characteristics of electromagnetic environment situations, use it to extract the state and trend features of electromagnetic environment situation, and use ConvLSTM to mine the deep data distribution features of electromagnetic environment situations. Then, the characteristic information of the two branches is combined to convert the prediction of the electromagnetic environment situation into a prediction of the change law of the electromagnetic environment situation to realize the accurate forecast of the future electromagnetic environment situation, strengthen the weight of the short-term historical data in the prediction, and ensure the long-term memory always exists.
• The DTW-based image similarity comparison method (Im-DTW) to find anomalies. We consider that the traditional hard metric is unsuitable for comparison among electromagnetic environment situations, so we extend the resilient one-dimensional DTW to two dimensions, adopt the block comparison method, gradually compare each region of the electromagnetic environment situation, and weigh the similarity of each part. Thus, this method ensures that the electromagnetic environment situation anomaly detection is more in line with the physical characteristics of electromagnetic space.
• We tested EMESNet using the dataset of electromagnetic environmental situations, and EMESNet had the best prediction under the same experimental conditions. Moreover, we set several evaluation metrics, including Im-DTW, to analyze the feasibility of Im-DTW in the comparison of electromagnetic environment situations.

In the subsequent sections, our work is discussed in detail. Section 2 focuses on analyzing the theoretical model of electromagnetic environment situation anomaly detection. Section 3 introduces the electromagnetic environment situation prediction model (EMESNet) and the anomaly detection model (Im-DTW). Section 4 carries out experimental validation to analyze the anomaly assessment index, the electromagnetic environment situation prediction capability, and anomaly detection capability. Finally, we summarize the experimental results and analyze the future direction of electromagnetic environment situations and their anomaly detection.

2. Problem Statement

As mentioned earlier, the electromagnetic environment situation is a kind of time-series image data, containing both the status and trend characteristics of electromagnetic field distribution on the general elements of the time-series image, so the electromagnetic environment situation anomaly detection network should apply to both the available video prediction task and its changes in physical characteristics. Electromagnetic wave propagation in space follows the geopotential distribution and waveguide equation, so we can consider that the electric field strength at one area is related to the electric field strength at adjacent locations, i.e., to the geopotential distribution. Based on this consideration, we believe that predicting an electromagnetic environment situation can be transformed into predicting the law of change in the electromagnetic environment situation. The normal electromagnetic environment situation has its intrinsic regularity, and the prediction can be regarded as the superposition of the previous moment’s electromagnetic environment situation and the inherent law of change. The last moment’s electromagnetic environment situation is a known value, so we need to explore the change law of the electromagnetic environment situation and superimpose it into the electromagnetic environment situation to predict the future moment’s electromagnetic environment situation and then carry out anomaly detection.

We assume that the electromagnetic environment situation in the specified period originates from the potential space $\mathcal{H}$. Then, the electromagnetic environment situation $X_{t+1}$ in the future moment also originates from the potential space $\mathcal{H}$. We denote its in-depth features as $H_{t+1}$ and $H_{t+1}\in \mathcal{H}$. Here, $H_{t+1}$ consists of a linear superposition of the deep data distribution features $D_t\in \mathcal{D}$ of the electromagnetic environment situation and the status and trend features $K_t\in \mathcal{K}$ before the moment $t$, where $\mathcal{D},\mathcal{K}\subseteq \mathcal{H},\hspace{0.33em}\hspace{0.33em}\mathcal{D}\cap \mathcal{K}=\varnothing$. $\mathcal{D},\mathcal{K}$ is the set of deep data distribution features and deep status and trend features of the electromagnetic environment situation at past moments, respectively [14], i.e., $H_{t+1}=D_t+K_t$. Therefore, the electromagnetic environment situation at moment $t+1$ can be expressed by Equation (1).

$$X_{t+1}=X_{t+1}^d+X_{t+1}^k\hspace{0.33em}=\mathcal{M}\left(D_t+K_t\right)={\mathcal{M}}_d\left(D_t\right)+{\mathcal{M}}_k\left(K_t\right)$$

(1)

${\mathcal{M}}_d\left(D_t\right),{\mathcal{M}}_k\left(K_t\right)$ represents the electromagnetic environment situation generated based on status and trend features and the electromagnetic environment situation generated based on data distribution features, respectively.

Therefore, we assume that the input data of the electromagnetic environment situation is $X_{1:t}=\left\{X_1,X_2,\cdots ,X_t\right\}\in {ℝ}^{t\times c\times m\times n}$; $t$ is the number of time series of the electromagnetic environment situation; $c,m,n$ are the number of channels, length, and width of the electromagnetic environment situation at a single moment, respectively; and all of the electromagnetic environment situation data are derived from the potential space $\mathcal{H}$. $X_{1:t}$ is input into the electromagnetic environment situation prediction model to obtain the predicted value ${\widehat{X}}_{t+1}$ of the electromagnetic environment situation at the moment of $t+1$. Then, the expected value at the moment of $t+1$ is compared with the actual value to perform anomaly detection. This is shown in Figure 1.

We assume that the electromagnetic environment situation anomaly detection function is $A\left(·\right)$. $A\left(·\right)$ can be a metric evaluation function such as MAE. Our aim is that the anomaly detection function $A\left(·\right)$ can judge all of the electromagnetic spectrum maps of this dataset as similar or even consistent, thus allowing for inevitable volatility of the electromagnetic environment situation. Therefore, for this problem, we make one assumption and two corollaries.

Assumption 1.

A suitable assessment metric can classify the standard electromagnetic environmental situation and the fluctuating electromagnetic environmental situation within the normal range into the same category under the normal electromagnetic environment situation. The values obtained by the metric are all standard values.

Corollary 1.

A suitable assessment metric can determine all the data in the normal electromagnetic environmental situation as the same assessment value or as having more minor fluctuations in that assessed value.

Corollary 2.

From Assumption 1 and Corollary 1, it is clear that the assessment metric should be insensitive to small fluctuations in the electromagnetic environmental situation.

Based on determining the assessment metrics, we compare the predicted value ${\widehat{X}}_{t+1}$ and the actual value $X_{t+1}$ at the moment $t+1$ to obtain the assessment value $A\left({\widehat{X}}_{t+1},X_{t+1}\right)$. Assuming the threshold of abnormality judgment is $\alpha$, the abnormality judgment is as in Equation (2).

$$X_{k+1}=\left\{\begin{array}{l}normal,\hspace{1em}A\left({\widehat{X}}_{t+1},X_{t+1}\right)\ge \alpha \\ anomaly,\hspace{1em}A\left({\widehat{X}}_{t+1},X_{t+1}\right)<\alpha \end{array}\right.$$

(2)

3. Electromagnetic Environment Situation Anomaly Detection Model

The electromagnetic environment situation anomaly detection model mainly consists of two parts: the situation prediction module and the anomaly detection module. The prediction module predicts the future data by establishing the sample space of a normal situation and combining the data of past moments. The anomaly detection module compares the actual data with the predicted data to achieve the purpose of anomaly detection.

To this end, this section introduces a new electromagnetic environment situation prediction model, which extracts state and trend features and in-depth data features in two branches, and then fuses them and performs prediction. We design an artificial feature extraction network based on the distribution pattern of electromagnetic space to extract the state and trend features, and use ConvLSTM to extract the deep data distribution. The training process of deep learning allows the model to obtain the changing pattern of normal electromagnetic environment situation and then predict the future electromagnetic environment situation. In addition, we extend the one-dimensional DTW to two dimensions, derive a mathematical model based on the DTW for anomaly detection, and analyze its complexity.

3.1. EMESNet: Electromagnetic Environment Situation Prediction Module

3.1.1. Overall Architecture

The main objective of EMESNet is to learn the mapping of input data to potential space $\mathcal{H}$ from historical EMES data and to predict EMES data at future moments by deep distribution data features $\mathcal{D}$ and status and trend features $\mathcal{K}$. To achieve this goal, we are inspired by LSTM and the literature [14] to pass deep-level features in time series by status and trend feature-based and data distribution feature-based recurrent neural networks, and output the two in parallel to obtain the EMES at future moments. The overall framework of EMESNet is shown in Figure 2.

Figure 2a shows the basic framework of a conventional recurrent neural network, which extracts temporal features and achieves prediction by bypassing state information between time series. Assume that the input data of the electromagnetic environmental situation are $X_{1:t}=\left\{X_1,X_2,\cdots ,X_t\right\}\in {ℝ}^{t\times c\times m\times n}$; $t$ is the number of temporal sequences of the electromagnetic environment situation and $c,m,n$ are the number, length, and width of the channels of the electromagnetic environment situation at a single moment, respectively. The network uses the input data to predict the latter data ${\widehat{X}}_{t+1}$, where the state transfer information of the two parallel modules is passed separately between each moment to ensure that the status and trend and data distribution characteristics of the electromagnetic environment situation can be learned separately and independently.

Figure 2b shows a simplified framework of EMESNet, a parallel network, where the EMES data at each moment are input into the network, and the features are extracted and implemented in the left and right parallel modules for different functions, respectively. Finally, both outputs are fused to obtain the EMES at the predicted moment.

The left branch of Figure 2b represents the status and trend feature-based electromagnetic environment situation prediction module, $X_{t+1}^k={\mathcal{M}}_k\left(K_t\right)$. This module is mainly used to extract the state characteristics of a single moment using the known laws of electromagnetic environment situations and electromagnetic field distribution to transfer the state characteristics between each moment by the recurrent neural network, and to explore the trend features in its continuous-time to predict the future electromagnetic environment situation $X_{t+1}^k$. This module takes pure status and trend information as input from the physical interpretation of deep learning. After passing through the deep learning network, what is inevitably obtained is the deep state and trend features under the physical information, thus realizing the physical constraints and guided training of the neural network.

The right branch of Figure 2b represents the data distribution feature-based electromagnetic environment situation prediction module, i.e., $X_{t+1}^d={\mathcal{M}}_d\left(D_t\right)$. As there are few extractable artificial features of the electromagnetic environment situation, this module is a data-driven approach mainly used to mine the data distribution information of the electromagnetic environment situation, extract deep features, and predict the future electromagnetic environment situation $X_{t+1}^d$. This module differs from the left branch in that it entirely relies on the raw data of the electromagnetic environment situation to obtain features and usually uses a recurrent neural network as the basic architecture because it must contain time-series features.

3.1.2. EMESCell: A Prediction Module Based on the State and Trend Characteristics of the EMES

EMESCell is the left branch in Figure 2b, and its primary function is to predict the electromagnetic environment situation in the future moment using the changing pattern of state and trend information of the electromagnetic environment situation. The main point of the prediction is to extract the deep features of the historical data of the electromagnetic environment situation to grasp its change pattern. We believe that the law of electromagnetic environment situation development can be reflected in the regulation of change in electromagnetic environment situation at each moment. The two moments before and after the electromagnetic environment situation are most closely related, and the electromagnetic environment situation in the last moment must be adjusted to change based on the last moment. Therefore, we assume that the electromagnetic environment situation of the next moment can be expressed as a linear superposition of the electromagnetic environment situation of the previous moment and its state and trend information change law, as in Equation (3).

$$X_{t+1}^k={\mathcal{M}}_k\left(K_t\right)=X_t+\mathsf{\Phi }\left(K_t\right)$$

(3)

where $X_t$ is the electromagnetic environment situation at moment $t$, and $\mathsf{\Phi }\left(K_t\right)$ is the amount of change in the electromagnetic environment situation at the moment $t+1$. $\mathsf{\Phi }\left(K_t\right)$ is essentially a law predictor, derived from the historical state and trend information change law, implemented in the network by the state transfer equation of the recurrent neural network.

$\mathsf{\Phi }\left(K_t\right)$ has two core functions: state and trend feature extraction and change pattern extraction, as shown in Figure 3.

State and trend feature extraction: When we use a data-driven model to explore the intrinsic variation pattern of an electromagnetic environmental situation, we first preprocess the electromagnetic environmental situation and construct a data format that can be used to learn this inherent pattern. We consider that the electric field strength at each point in a region and the electric field strength within its neighborhood have some intrinsic connection, which may be determined by factors such as topography, and thus is fixed, such as in the field of electric wave propagation, where the fluctuation equation can represent this relationship. In such a premise, the part that changes, excluding the fixed intrinsic law, is the law of development of the electromagnetic environment situation over time.

We discretize this law to each location in the region and calculate the gradient $\frac{\partial X_{t+1}^k}{\partial x},\frac{\partial X_{t+1}^k}{\partial \left(-x\right)},\frac{\partial X_{t+1}^k}{\partial y},\frac{\partial X_{t+1}^k}{\partial \left(-y\right)}$ of the electric field intensity in the four directions at the location points in the area to obtain the amount of variation in each location point in the four directions. The farthest data gradient along the respective direction is 0, which can be considered the boundary that will be the law of variation of the electromagnetic environment situation in that direction. We use the convolutional network to merge the various quantities in the four directions to extract the prediction law of the electromagnetic environment situation at the future moment ${\tilde{s}}_{t+1}$, i.e.,

$${\tilde{s}}_{t+1}=f_{CNN}\left(\left.\frac{\partial X_{t+1}^k}{\partial x}\right|\left.\frac{\partial X_{t+1}^k}{\partial \left(-x\right)}\right|\left.\frac{\partial X_{t+1}^k}{\partial y}\right|\frac{\partial X_{t+1}^k}{\partial \left(-y\right)}\right)=f_{CNN}\left[g\left(X_{t+1}^k\right)\right]$$

(4)

$g\left(·\right)$ denotes the electromagnetic environment situation gradient information extraction function and $f_{CNN}\left(·\right)$ represents the physical feature extraction network based on a convolutional neural network.

Pattern extraction: We predict the next moment of the electromagnetic environment situation $X_{t+1}^k$ based on the state and trend information, and then extract the state features of the expected situation ${\tilde{s}}_{t+1}$. As shown in Figure 3, in the input data of the electromagnetic environment situation at this moment, the same state and trend features are obtained using the state feature extraction network $f_{CNN}\left(·\right)$; the difference between the two is found and given a weight value to obtain the amount of change in the electromagnetic environment situation $\Delta s_{t+1}$ at the last moment, as shown in Equation (5).

$$\Delta s_{t+1}=\left({\tilde{s}}_{t+1}-s_t\right)\cdot w_s$$

(5)

$\Delta s_{t+1}$ represents the amount of electromagnetic environment situation change in the two moments before and after the situation. This variable contains the predicted electromagnetic environment situation data $X_{t+1}^k$, while $X_{t+1}^k$ also includes the magnitude of electromagnetic environment situation change in the previous moment. Therefore, $\Delta s_{t+1}$ can express the temporal characteristics as the state transfer quantity between the temporal networks. $w_s$ is the weight of the situation change value, which is continuously trained by deep learning to better transmit timing information.

In summary, EMESCell implements the electromagnetic environment situation prediction by extracting the state and trend feature extraction module and the law extraction module, and Equation (3) can be expressed as

$$X_{t+1}^k={\mathcal{M}}_k\left(K_t\right)=X_t+\mathsf{\Phi }\left(K_t\right)\hspace{1em}=X_t+w_{t+1}^k\cdot \Delta s_{t+1}=X_t+w_{t+1}^k\cdot \phi \left(\Delta s_t\right)$$

(6)

$$\begin{array}{l}\phi \left(\Delta s_t\right)=w_s\cdot \left\{f_{CNN}\left[g\left(X_{t+1}^k\right)\right]-f_{CNN}\left[g\left(X_t\right)\right]\right\}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.17em}\hspace{1em}\hspace{1em}=w_s\cdot \left\{f_{CNN}\left[g\left(X_t+w_t^k\Delta s_t\right)\right]-f_{CNN}\left[g\left(X_t\right)\right]\right\}\end{array}$$

(7)

where $w_t^k$ is a memory factor that adjusts $\Delta s_{t+1}$ to the next moment of the electromagnetic environment situation by deep learning.

Therefore, the EMESCell has only two input data $X_t,\Delta s_t$ at the exact moment, and we try to achieve the electromagnetic environment situation prediction at the $k+1$ moment by $k$ prediction steps. So, we need to stack $k$ EMESCells to achieve the electromagnetic environment situation prediction, and $\Delta s_t$ is the state transfer variable in each sequential EMESCell. We use the memory factor $w_k$ to transfer the electromagnetic environment situation data in the past $k$ moments for long-term memory.

3.2. Im-DTW-Based Anomaly Detection Module

We next establish a mathematical model for anomaly detection to realize the anomaly detection of electromagnetic environment situations mainly by evaluating the function $A\left(·\right)$ to compare whether or not the predicted data and the actual data are similar. Based on the shortcomings of existing image similarity comparison algorithms, we propose an image similarity comparison algorithm (Im-DTW) based on dynamic time regularization (DTW), which uses the elasticity metric of DTW in sequence comparison to achieve similarity comparison of local regions, allows appropriate fluctuations of points within areas, and is suitable for similarity comparison between electromagnetic spectrum maps.

Given two one-dimensional univariate sequences $x\in {ℝ}^{1\times m}$ and $y\in {ℝ}^{1\times n}$, whose sequence lengths are $m$ and $n$, the distance matrix $D\in {ℝ}^{m\times n}$ between the sequences $x$ and $y$ is obtained by the distance calculation function $f\left(x,y\right)$. Each path formed in the distance matrix $D$ from $\left(1,1\right)$ to $\left(m,n\right)$ by moving is denoted as $\mathrm{A}\in \mathcal{A}{}_{m,n}$, where $\mathcal{A}{}_{m,n}$ means the set of all directions, then the sum of the distances on each path can be represented as $〈A,f\left(x,y\right)〉$. Therefore, the DTW objective is to find an optimal path ${\mathrm{A}}^{\ast }$ such that the sum of lengths on that path is minimized, i.e.,

$$\begin{array}{c}\hfill DTW\left(\mathbf{x},\mathbf{y}\right)=\underset{{\mathrm{A}}_i\in \mathcal{A}{}_{m,n}}{min}〈{\mathrm{A}}_i,f\left(\mathbf{x},\mathbf{y}\right)〉\end{array}$$

(8)

Therefore, the similarity between sequences $x$ and $y$ is also shown in Equation (8), and the smaller the $\mathrm{DTW}\left(x,y\right)$, the higher the similarity between the two.

Based on the derivation idea of one-dimensional DTW, we use DTW to compare two-dimensional images and propose a similarity comparison algorithm for two-dimensional images with rows and columns as local regions. We adopt rows and columns as local areas for two reasons; first, DTW is only suitable for dealing with one-dimensional data, and DTW operations on two-dimensional data will map the paths to a four-dimensional space, which will weaken the interpretability of the optimal pathways and dramatically increase the computational complexity; second, the continuity of DTW makes it possible to match them in a range of time steps, with one-to-many and many-to-one properties, which can calculate similarity with a specific range of fluctuations, in line with the physical laws of the changing electromagnetic environment situation.

We establish a DTW-based image similarity comparison model based on the above considerations. Given two two-dimensional image data $X\in {ℝ}^{m\times n}$ and $Y\in {ℝ}^{m\times n}$, with $X$ and $Y$ with equal length and width, $m$ and $n$, respectively, the images $X$ and $Y$ are divided into $m$ rows ($n$ columns) by row, denoted as $X=\left\{x_1,x_2,\cdots ,x_m\right\}=\left\{x^1,x^2,\cdots ,x^n\right\}$, where $x_1$ represents row and $x^1$ represents column, and then the row (column) similarity r-DTW (c-DTW) is calculated. To calculate the row similarity r-DTW, for example, we define the row similarity as the average of the similarity of the corresponding row sequences in the two images, so the row similarity of the images and based on DTW is expressed as

$$\begin{array}{c}\hfill r-DTW\left(X,Y\right)=\frac{1}{m}\sum _{i=1}^{m}DTW\left({\mathbf{x}}_i,{\mathbf{y}}_i\right)=\frac{1}{m}\sum _{i=1}^m\underset{{\mathrm{A}}_i\in \mathcal{A}{}_{m,n}}{min}〈{\mathrm{A}}_i,f\left({\mathbf{x}}_i,{\mathbf{y}}_i\right)〉\end{array}$$

(9)

Similarly, the column similarity c-DTW can be expressed as

$$c-DTW\left(X,Y\right)=\frac{1}{n}\sum _{j=1}^{n}DTW\left({\mathbf{x}}^j,{\mathbf{y}}^j\right)=\frac{1}{n}\sum _{j=1}^n\underset{{\mathrm{A}}_j\in \mathcal{A}{}_{m,n}}{min}〈{\mathrm{A}}_j,f\left({\mathbf{x}}^j,{\mathbf{y}}^j\right)〉$$

(10)

Usually, the image change can be decomposed into two changes in the vertical direction, and it is incomplete to compare the similarity of images based on row similarity or column similarity only. Therefore, we combine the two linearly, then the Im-DTW of images $X$ and $Y$ is expressed as

$$\mathrm{Im}-\mathrm{DTW}\left(X,Y\right)=\alpha \cdot r-\mathrm{DTW}\left(X,Y\right)+\left(1-\alpha \right)\cdot c-\mathrm{DTW}\left(X,Y\right)$$

(11)

$\alpha$ is a coefficient indicating the degree of involvement of row similarity and column similarity in the image comparison. The smaller the $DTW\left(X,Y\right)$, the greater the similarity between the two images. The complexity of calculating the DTW and the best path ${\mathrm{A}}^{\ast }$ of two one-dimensional sequences in the dynamic programming algorithm is only $\mathcal{O}\left(mn\right)$. In contrast, the complexity of computing the similarity of two-dimensional images proposed in this paper is only $\mathcal{O}\left(\left(m+n\right)\cdot mn\right)$, which improves the computational efficiency and takes into account the local space variation compared with the complexity $\mathcal{O}\left(m^2{n}^2\right)$ when converting all two-dimensional data into one-dimensional data to calculate the DTW.

In Section 3.2, we proposed a way to judge anomalies outside the traditional assessment metrics such as MAE. Im-DTW is proposed explicitly for the normal fluctuation of the electromagnetic environment situation, which has both the precise judgmental nature of MSE and the statistical characteristics of SSIM. We can choose the appropriate evaluation model according to different test environments and application scenarios as a way to better detect anomalies.

4. Experiment

4.1. Experimental Settings

Dataset: We use the electromagnetic environment situation dataset for EMESNet detection. The EMES dataset is the electromagnetic spectrum map of the Taklamakan region over a period with a center frequency of 15 MHz, generated by three simulation softwares, MATLAB 2018b, Tuxin Earth 4, and Wireless Insite. The size of all images in the dataset is 80 × 80. The training data simulate the normal electromagnetic environment situation under the specified definition. The six test sets simulate the changes in the electromagnetic spectrum map caused by the behavior of electromagnetic radiation source moving, switching on and off, and power adjustment over a period. This dataset can be used for electromagnetic environment situation prediction and anomaly detection. The specific description and documentation of the dataset are available at this URL: https://github.com/huyilin0621/Electromagnetic-Environment-Situation (accessed on 1 July 2022).

Network architectures: EMESNet is consistent in all training sets, in which the electromagnetic environment situation prediction module based on data features adopts the ConvLSTM model. With three layers of the network, the number of hidden neurons in each layer is 64, 64, and 64, and the convolution kernel size in each layer is 3 × 3. In addition, the state and trend feature-based EMES prediction module contains a total of 64 EMESCells with a convolution kernel of 7 × 7. The training process of all algorithms uses the Adam optimizer, the loss function uses the mean square error (MSE) loss function, the initial learning rate is 0.001, the prediction step size is set to 4, the batch size is set to 16, and the epoch is taken as 100.

Evaluation metrics: We use the evaluation metrics MSE, MAE, and SSIM, which are commonly used in time-series image prediction, to test the experimental prediction effect. We also use Im-DTW as the evaluation metrics of this experiment simultaneously to verify the applicability of Im-DTW in this specialized field of electromagnetic environment and compare it with other algorithms, in which the Im-DTW algorithm takes 0.5. The lower MSE, MAE, Im-DTW, and higher SSIM in the evaluation index all indicate better prediction performance of the algorithm.

4.2. Suitability Analysis of Im-DTW

We analyzed theoretically that hard metrics such as MSE do not consider the spatial characteristics and fluctuations in the electromagnetic environment situation. At the same time, SSIM performs image comparison using local statistical information, and the method is too general and lacks precision. Therefore, we propose the Im-DTW algorithm as an evaluation metric to compare electromagnetic environment situations.

We use training data 1 from the dataset to test the effectiveness of the above four evaluation metrics in comparing electromagnetic environment situation. Training data 1 is a standard electromagnetic environment situation dataset with the same radiation sources for all electromagnetic spectrum map stations, the noise of −15 dB, unknown radiation sources within 200 m, and fluctuations in radio transmit power within 10%.

To this end, we assume that the value of the assessment index is $A=\left\{a^i,i=1,\cdots ,N\right\}$, and considering the inconsistency of the basic measure of each assessment metric, the results of each assessment metric are first normalized to obtain $A_{norm}=\left\{a_{norm}^i,i=1,\cdots ,N\right\}$, and then the standard deviation of $A_{norm}$ is used as a measure of the ability of the assessment metric to compare the similarity of electromagnetic environmental situation $Q$, as shown in Equation (12).

$$Q=\frac{1}{N}{\displaystyle \sum _{i=1}^N\sqrt{{\left(a_{norm}^i-\frac{1}{N}{\displaystyle \sum _{i=1}^N{a}_{norm}^i}\right)}^2}}$$

(12)

We first fix the noise intensity and calculate it to compare the standard electromagnetic environment situation with the normal electromagnetic environment situation using different evaluation metrics. As shown in Figure 4, we add Gaussian white noise with a noise intensity of 0.01, input 1000 random normal electromagnetic environment situation plots in training data 1, and compare each input data with the standard electromagnetic environment situation under different evaluation metrics. Each subgraph in Figure 4 represents a different assessment metric; the vertical coordinate is the value after normalizing the obtained 1000 assessment values, and the horizontal coordinate represents the 1000-input normal electromagnetic environment situation. From the analysis of this figure, MSE and MAE are calculated in the same way, and thus remain consistent. Although the overall distribution of Im-DTW is uniform with MAE, we can find that the dense area of the distribution of Im-DTW is more compressed than that of MSE. The compression region of Im-DTW is concentrated between 0.0 and 0.6, while the compression region of MSE is concentrated between 0.0 and 0.7, with a tendency of polarization. According to Corollary 1 and Corollary 2, the fluctuation range of Im-DTW is smaller, has a higher degree of volatility filtering for the electromagnetic environment, and performs better in the analysis of the electromagnetic environment situation. Meanwhile, the distribution of SSIM covers between 0.2 and 0.8 and, intuitively, it seems that SSIM performs more consistently among the four evaluation metrics.

We experimented by adding noise and defined that any fluctuation in noise intensity in the range of 0.02 was considered a normal electromagnetic environment situation. We then add Gaussian white noise of different powers to test the tolerance of each assessment index to fluctuations in the electromagnetic environment. As shown in Figure 5, the horizontal coordinate indicates the noise of different intensities within the normal fluctuation range, and the vertical coordinate indicates the fluctuation degree Q. In Figure 5, the Q values of the four evaluation indicators decrease with the slight increase in the noise, which proves that all four evaluation indicators can filter the normal fluctuation in the electromagnetic environment within a specific range. However, in comparison, the Q value of SSIM is smaller than others, indicating that, the smaller the fluctuation degree of SSIM, the stronger the filtering ability of noise. We analyze this because SSIM itself is an evaluation metric based on area statistics, and adding white noise does not change much for the statistical properties of the sliding window. For example, if SSIM is used to compare Gaussian white noise, the evaluation value obtained will also be stable and unchanged owing to the stable statistical properties of Gaussian white noise, thus the evaluation value of SSIM will fluctuate slightly. From this point of view, the Q value of Im-DTW is more stable than MSE and more suitable for the analysis of the electromagnetic environment situation, although it covers the hard metric method.

4.3. Comparison of the Predicted Results of the Electromagnetic Environment Situation

We use the PhyDNet model and ConvLSTM model from the literature [14] as the comparison algorithm in this paper. The PhyDNet model keeps the same network structure as the original paper, and the ConvLSTM is set with the same parameters as the ConvLSTM branch in EMESNet, as a way to verify whether or not the artificial feature extraction module contributes to the data feature extraction network. We use the results of these three networks for comparison with two primary purposes: first, to compare ConvLSTM with the other two networks as a way to reach the prediction accuracy of the network based on the fusion of state and trends and data distribution features and the network based on data features; and second, to compare PhyDNet and EMESNet as a way to verify that the state feature extraction approach of EMESNet can be applied to the electromagnetic environmental situation domain. We tested the three test data of dataset 2 with the above three networks and calculated the values of the four evaluation metrics, as shown in

Table 1. Comparison of the prediction effects of different models on different data sets.

	Test 1			Test 2			Test 3
	MSE	SSIM	Im-DTW	MSE	SSIM	Im-DTW	MSE	SSIM	Im-DTW
PhyDNet	0.000766	0.9671	0.02087	0.000840	0.9670	0.02326	0.001055	0.9629	0.03357
ConvLSTM	0.000842	0.9659	0.02034	0.000914	0.9654	0.02230	0.001105	0.9652	0.03258
EMESNet	0.000706	0.9727	0.02012	0.000789	0.9729	0.02297	0.001135	0.9659	0.03722

.

In Table 1, the same prediction results are compared with different assessment metrics. In the three test sets, it is evident that Im-DTW has a high agreement with MSE, and when the value of Im-DTW is significant, the value of MSE is also large, which indicates that Im-DTW takes into account the characteristics of both traditional evaluation metrics and EMES, and further proves that this evaluation metric can be used for EMES. We compare the prediction accuracy of each model on the electromagnetic environmental situation dataset using MSE as the evaluation metric, and the EMESNet proposed in this paper achieves the best results on both test set 1 and test set 2, with at least 8.49% and 6.46% improvement in prediction accuracy, respectively, compared with the other two models. SSIM and Im-DTW are used as evaluation metrics and are consistent with MSE, and compared with PhyDNet, SSIM improves by 0.57% and 0.61%, respectively, and Im-DTW improves by 3.72% and 1.26%, respectively, on the first two datasets. The models corresponding to three evaluation metrics in test set 3 are inconsistent, and PhyDNet works best if we consider the prediction accuracy. However, if the volatility of the electromagnetic environment is taken into account and a certain amount of error is allowed to exist, ConvLSTM is the best. Taking test set 1 as an example, we show the prediction results of the three models, as shown in Figure 6.

Figure 6a–d show the input data of electromagnetic environment situation with a step size of 4 and $t=46\sim 49$, and the electromagnetic environment situation of $t=50$ is predicted by the model (Figure 6e). Figure 6f–h show the prediction results of ConvLSTM, PhyDNet, and EMESNet, respectively. Compared with Figure 6e, the EMES maps generated by the three models and the EMES maps at the time are consistent overall, among which the low-field strengths of ConvLSTM and PhyDNet are in high agreement with the target data, but the prediction results of the high-field strength regions, especially the field strengths of individual radiation sources, are more granular and less accurate. Compared with Figure 6e, EMESNet has a lower prediction value in the low-field intensity region. However, the overall smoothness and lack of granularity of the first two figures and the size difference of the four radiation sources are also consistent with the target data, which have a higher prediction accuracy. The results also show that the deep learning model that fuses data distribution features and state and trend features provides a better prediction and can better extract data features. This also proves that EMESNet successfully applies the basic idea of PhyDNet to the electromagnetic environment situation prediction. The state feature extraction method of EMES is more suitable for the feature extraction of the electromagnetic environment situation, so it shows higher accuracy than others.

4.4. Electromagnetic Environment Situation Anomaly Detection

According to the predicted results of the electromagnetic environment situation, the anomaly of our electromagnetic environment situation is detected. Test set 1 simulates the region suffering from three different intensities of noise interference, and test set 2 and test set 3 emulate the power adjustment of radiation sources on a fixed time interval, where the power of test set 2 is smaller than that of test set 3. We believe that the above-described interference and power adjustment are anomalous radiation source behaviors; the anomaly of test set 1 is continuous over a period, and there is the possibility of being predicted by the model, while the anomalies of test sets 2 and 3 are intermittent, the occurrence of anomalies by chance is significant, and the model is not easy to predict. Therefore, we perform anomaly detection on the three test sets, use Im-DTW as the evaluation index, and introduce the ROC curve and AUC area to comprehensively judge the performance of the neural network model. The ROC curve is the receiver operating characteristic curve for measuring the dichotomous performance with false alarm probability as the horizontal coordinate and recall as the vertical coordinate. The AUC is the area under the ROC curve, which is a quantitative method to compare the ROC curves. The ROC curves of the three test sets are shown in Figure 6, and the corresponding AUCs of different models are shown in Figure 7 and

Table 2. AUC of different evaluation indicators under different models.

	Test 1		Test 2		Test 3
	MAE	Im-DTW	MAE	Im-DTW	MAE	Im-DTW
ConvLSTM	0.5748	0.5467	0.5815	0.5887	0.6458	0.6711
PhyDNet	0.6004	0.5525	0.6164	0.6102	0.7153	0.7411
EMESNet	0.5583	0.5741	0.5863	0.6052	0.7595	0.8545

.

On the three test sets, the performance of each model performs differently. Comparing test set 1 and test set 3, using Im-DTW as the evaluation metric, EMESNet achieved the optimal level of AUC on both test set 1 and test set 3, which were 3.91% and 15.3% higher, respectively, than the other models, and the models performed comparably on test set 2, with EMESNet only 0.78% lower than the other models. From the analysis of the anomaly type of the data, the anomaly type on test set 1 is the addition of continuous moment noise disturbance. Because of the model’s prediction performance, this continuously occurring noise has the same characteristics, other than intensity, and is easily predicted by the model. Thus, the anomaly detection rate of the anomaly occurrence period is reduced, showing the side effect of EMESNet’s good prediction, as performed on test set 1. In addition, there are minor power adjustments in test set 2 and significant power adjustments in test set 3 due to the radiation source timing adjustment of the transmit power adjustment. Therefore, on test set 2, the electromagnetic environment situation does not change much, and the detection capability of all models is comparable, so we cannot fully judge which model has better anomaly detection. On test set 3, the power adjustment is significant, and the electromagnetic environment situation varies greatly. However, EMESNet has the highest detection probability under the current anomaly detection mechanism because it better captures the features of the normal electromagnetic environment situation, and thus is more biased in predicting the normal electromagnetic environment situation. Among these three datasets, the anomalies in test set 3 are more contingent, more sudden, and less predictable, making it more suitable for anomaly detection. In contrast, test set 1 has more continuous moment-to-moment variability and is ideal for electromagnetic environment situation prediction.

In addition, the experiments also conducted a cross-sectional comparative analysis between the traditional evaluation algorithm MAE and the evaluation index proposed in this paper, Im-DTW. Nine sets of AUC data were formed by the three models in the three test sets, and the detection rate was higher than that of MAE when Im-DTW was used as the evaluation index in six of the results. In particular, in test set 3, all of the anomaly detection rates obtained with Im-DTW as the evaluation metric were higher than those with MAE, with a minimum improvement of 3.6%. The experiment proves that Im-DTW is more suitable for electromagnetic environment situation anomaly detection and can improve the anomaly detection rate.

5. Conclusions

Currently, the effective detection of anomalies in the electromagnetic environment situation faces a severe challenge in the increasingly complex electromagnetic environment. To address the current state of research, we propose a feature extraction model (EMESCell) to fuse state and trend features of the electromagnetic environment situation and then design a dual-branch electromagnetic environment situation prediction model (EMESNet), which connects state and trend features and data distribution features. We examined the prediction effect of the model in three test sets, and EMESNet showed good prediction capability. At the same time, we considered the volatility of the electromagnetic environment and based on the Im-DTW image similarity comparison method to discover anomalies, and the experiments proved that the detection rate of the Im-DTW-based anomaly detection method is generally improved compared with that of other algorithms.

In future work, there are directions that still need to be expanded for the anomaly detection of the electromagnetic environment situation. The first is to realize the anomaly cognition of the electromagnetic environment situation, combine semantic information and physical features to locate and analyze the anomaly, and realize the intelligent decision of radio management [14,19]; the second is to adopt multi-task learning and multi-dimensional data processing, fuse multiple data features, improve the complexity of the electromagnetic environment situation prediction model, and tap the deep features of the electromagnetic environment situation to improve the accuracy and efficiency of anomaly detection [17,20].