GAN-rcLSTM: A Deep Learning Model for Radar Echo Extrapolation

Abstract

The target of radar echo extrapolation is to predict the motion and development of radar echo in the future based on historical radar observation data. For such spatiotemporal prediction problems, a deep learning method based on Long Short-Term Memory (LSTM) networks has been widely used in recent years, although such models generally suffer from weak and blurry prediction. This paper proposes two models called Residual Convolution LSTM (rcLSTM) and Generative Adversarial Networks-rcLSTM (GAN-rcLSTM): The former introduces the residual module, and the latter introduces the discriminator. We use the historical data of 2017 and 2018 in the Jiangsu region as training and test sets. Experiments show that in long sequence forecasts, our model can provide more stable and clear images, while achieving higher CSI scores.

1. Introduction

An accurate weather forecast system can greatly reduce the economic losses caused by natural disasters. According to the period validity, as shown in the

Table 1. Types of weather forecasts.

Forecast Types	Period Validity
nowcasting	0–2 h
short-term forecasting	0–12 h
short-range forecasting	1–3 days
medium-range forecasting	4–10 days
long-range forecasting	>30 days

, forecast types can be divided into nowcasting, short-term forecasting, short-range forecasting, medium-range forecasting, and long-range forecasting. At present, the forecasting methods can be mainly divided into two categories: One is the Numerical Weather Prediction (NWP) method, and the other is the extrapolation method based on the radar echo. The NWP methods take the spatial distribution of meteorological elements at a certain time as the initial value, establish a complex physical state equation under given boundary conditions, and simulate the atmospheric movement in the future through a large computer. The NWP methods are very mature in short-range and medium-range forecasting operations. However, due to the complexity of the solution process of the differential equation system, NWP methods have the problems of prediction delay and low resolution. Therefore, extrapolation methods based on radar echoes are more widely used in short-term and nowcasting operations (0–12 h). A radar echo sequence contains a lot of information, from which we can analyze the intensity, location, and movement of typhoons, thunderstorms, rainstorms, squall lines, hail, tornadoes, and other disastrous weather. The target of radar extrapolation is to predict the future fixed length radar echo map of a region based on the previously observed radar echo map.

At present, the traditional extrapolation methods are mainly centroid tracking [1,2], cross-correlation [3,4], and optical flow [5,6,7,8,9,10]. The cross correlation method divides the whole data area into several small areas, and then calculates the correlation coefficient between the small areas of the radar echo image at adjacent times. The average motion of the echo region is determined by the correlation between regions. However, in severe convective weather, tracking failure will increase significantly. The optical flow method calculates the instantaneous velocity field of the pixel movement on the target surface by making two basic assumptions (constant brightness and small moving distance of the target), and then predicts the next frame image. The advantage of the optical flow method is that it can detect and identify the position of the moving target without knowing the information of the scene. The optical flow method makes two basic assumptions: The image brightness is constant, and the moving distance of the target is small. The next frame image is predicted by calculating the instantaneous velocity field of pixel motion on the target surface. The advantage of optical flow method is that it can detect and identify the position of moving targets without understanding the scene information, but the computational cost is too high to meet the needs of high-resolution real-time prediction.

In recent years, deep-learning-based models have made great progress in the field of spatiotemporal prediction [11,12,13,14,15,16,17,18,19,20,21]. Because spatiotemporal prediction is a sequence to sequence problem, Recurrent Neural Network (RNN) [22] and its variants, Long short-term memory (LSTM) [23] and Gate Recurrent Unit (GRU) [24], have been widely used. The LSTM model uses a ‘gate’ structure to control the flow of information and avoid the disappearance of gradient in RNN. LSTM has three gates in total, including the input gate, forget gate, and output gate. The forget gate controls how many cell states are discarded at the last moment. The input gate controls how much new information will enter the current cell state. Finally, the output gate determines which parts of the cell state need to be output. Each gate consists of a full connection layer and a sigmoid function. The current time input signal $x_t$ and the previous time hidden layer output $h_{t-1}$ determine the size of the gate. Zero stands for “filter all information”, and one stands for “pass all information”. RNN, LSTM, and GRU all use full connection operations in input-to-state and state-to-state. Srivastava [25] applied LSTM and a self-coding mechanism to video prediction. One LSTM is used as an encoder to extract the feature of the input sequence, and the other LSTM copies the state of the encoder and circularly predicts the subsequent time as a decoder. Shi [26] proposed ConvLSTM and used it to predict precipitation in Guangdong, Hong Kong, and the Macao Bay area. ConvLSTM uses convolution operation instead of full connection operation, which gives LSTM the ability to extract spatial features and greatly reduce the number of network parameters. Xu [27] proposed a dynamic convolution network, which generates a convolution kernel for each input through a generator. The adaptive mechanism of DFN makes the model more robust. Shi [28] proposed TrajGRU using the idea of dynamic convolution network. This model is more flexible than convGRU with fixed connection structure. In addition, aiming at the imbalance problem of low proportion of high precipitation samples in real life, the measurement methods of balanced mean square error (B-MSE) and balanced absolute error (B-MAE) are also proposed. TrajGRU improves the accuracy of the model for predicting problems with imbalanced samples, such as lightning and heavy precipitation. Wang [29,30] extended ConvLSTM and proposed PredRNN and PredRNN++. In addition to the vertically transmitted states C and H, these models also have the zigzag propagation state M, so that the cell can see the information of all layers at the previous time. The basic unit of PredRNN++ is a cascade structure, and the results of State C are used for the calculation of state M. At the same time, inspired by highway [31], a gradient high unit (GHU) similar to the GRU structure is inserted between the first and second layers to alleviate the feature loss of state M in the long-term prediction process. PredRNN and PredRNN++ make the connection between different layers tighter through additional propagation paths but require more gates to control the flow of information, making the model more complex. Wang [32] proposed the Memory In Memory (MIM) networks.Experiments show that the forget gate in the long and short-term memory plays a limited role in the long-term prediction process, so it is replaced by two cascaded mim-n and mim-s. The difference between the two time hidden layers is used as an input to increase the oblique propagation path of the feature. Wang proposed E3DLSTM [33], which replaces the forget gate with the recall gate structure. The recall gate is calculated by multiplying the candidate state R and the State C of multiple times in the past. E3D-LSTM alleviates the problem of forgetting the gate gradient saturation, but the recall gate structure in the model uses matrix multiplication, which greatly increases the computational cost required by the model. Chen [20] proposed a coding prediction model (3DCNN-bcLSTM) combining 3DCNN and bi-directional convolution LSTM. He [19] developed an improved model called M-ConvGRU, which performs convolution operations on the input data and previous output of a GRU network. Geng [21] defined a Multi-Scale Criss-Cross Attention Context Sensing Long Short-Term Memory (MCCS-LSTM). MCCS-LSTM integrates the attention mechanism into ConvLSTM, which improves the model context utilization. Ravuri [34] proposed a probabilistic forecasting model called DGMR to improve the forecasting ability of heavy precipitation weather. DGMR uses convolution to constrain the spatial consistency and temporal consistency of the forecast sequence to produce clear forecasts. In general, ConvLSTM first reduces the number of parameters of LSTM-based deep models to a level that can be operationalized. After that, some research improved ConvLSTM to improve the feature extraction ability, including increasing the gradient propagation route, incorporating an attention mechanism, dynamic convolution, etc. After several years of development, state-of-the-art deep learning models have achieved higher accuracy than traditional methods in the short-term radar extrapolation problem in regular weather.

However, almost all models based on LSTM have two obvious problems. The first is echo decay. Recent evidence shows that stacking layers in an end-to-end fashion enables the network to extract more complex feature patterns, so network depth is critical [35,36,37,38]. However, some studies also reveal that [39,40,41] has a degradation problem in deep networks. As the network depth increases, the model’s score becomes saturated and then drops rapidly. In deep LSTM models, the gradients are difficult to propagate to the bottom layer, and the extrapolation results are always gradually lower than the true value.The second is blur. Usually after an hour, the predicted image will become blurred. Because the traditional LSTM network takes the prediction of the previous time as the input of the next time. This training method does not take the prediction sequence as a whole, but calculates the mean square error hourly and then calculates the average value. During training, the neural network will give a more conservative prediction in order to avoid gradient punishment. In order to solve these two problems, we propose two models: residual convolution LSTM (rcLSTM) and Generative Adversarial Networks-residual convolution LSTM (GAN-rcLSTM).

2. Materials and Methods

The radar echo image transformed by Cartesian coordinate system can be regarded as a gray image, and each pixel value is distributed between 0 and 70. Suppose we express the meteorological data as a dynamical system over time. The observation at any time can be represented by a tensor $\mathcal{X}\in R^{P\times M\times N}$, where $M\times N$ represents the spatial region, and P is the number of meteorological factors. Then, the prediction of sea surface temperature anomalies can be expressed as an unsupervised spatiotemporal prediction problem, that is, according to a $\mathit{\text{length-J}}$ sequence, we can predict the most likely distribution of the sequence after K times in the future:

$${\widehat{\mathcal{X}}}_{t+1},\dots ,{\widehat{\mathcal{X}}}_{t+K}=\underset{{\mathcal{X}}_{t+1},\dots ,{\mathcal{X}}_{t+K}}{argmax}p\left({\mathcal{X}}_{t+1},\dots ,{\mathcal{X}}_{t+K}\mid {\mathcal{X}}_{t-J+1},\dots ,{\mathcal{X}}_t\right)$$

(1)

2.1. rcLSTM

Inspired by Resnet [39], which has made great breakthroughs in the field of pattern recognition, we add a gradient propagation route from the bottom layer to the top layer of LSTM. The input of each layer is the sum of the outputs of the previous two layers, so there will be a short-circuit mechanism to ensure that the gradient is almost never equal to zero. The structure of the whole network is shown in the Figure 1.

ST-LSTM [29] is a spatiotemporal memory mechanism based on LSTM. Each ST-LSTM cell contains two memories: temporal memory C, which is updated along the horizontal direction, and spatial memory M, which is updated along the zigzag direction. Like ConvLSTM, ST-LSTM also uses a convolution operator in the state-to-state and input-to-state transitions to replace the full connections operation. Input gate $i_t$ and forget gate $f_t$ control the flow of information on memory C and M. Finally, they will be concatenated and sent to the output gate $o_t$ to calculate the final hidden state $h_t$. The equations of ST-LSTM are shown as follows:   

$$\begin{array}{cc}\hfill & \left(\begin{array}{c}{g}_t\\ i_t\\ f_t\end{array}\right)=\left(\begin{array}{c}\hfill tanh\\ \hfill \sigma \\ \hfill \sigma \end{array}\right)W_1\ast \left[{\mathcal{X}}_t+{\mathcal{H}}_t^{k-2},{\mathcal{H}}_{t-1}^k,{\mathcal{C}}_{t-1}^k\right]\hfill \\ \hfill & {\mathcal{C}}_t^k=f_t\odot {\mathcal{C}}_{t-1}^k+i_t\odot g_t\hfill \\ \hfill & \left(\begin{array}{c}{g}_t^{\prime }\\ i_t^{\prime }\\ f_t^{\prime }\end{array}\right)=\left(\begin{array}{c}\hfill tanh\\ \hfill \sigma \\ \hfill \sigma \end{array}\right)W_2\ast \left[{\mathcal{X}}_t+{\mathcal{H}}_t^{k-2},{\mathcal{M}}_t^{k-1}\right]\hfill \\ \hfill & {\mathcal{M}}_t^k=f_t^{\prime }\odot tanh\left(W_3\ast {\mathcal{M}}_t^{k-1}\right)+i_t^{\prime }\odot g_t^{\prime }\hfill \\ \hfill & o_t=tanh\left(W_4\ast \left[{\mathcal{X}}_t,{\mathcal{C}}_t^k,{\mathcal{M}}_t^k\right]\right)\hfill \\ \hfill & {\mathcal{H}}_t^k=o_t\odot tanh\left(W_5\ast \left[{\mathcal{C}}_t^k,{\mathcal{M}}_t^k\right]\right),\hfill \end{array}$$

(2)

2.2. GAN-rcLSTM

We introduced the generative adversarial networks [42] in the field of unsupervised learning in recent years to solve the fuzzy problem. We replaced the original generator with rcLSTM and replaced the noise with historical radar observations as input. In order to speed up the convergence of the model, we used Layer Normalization between the convolutional layers. The generator is used to generate the echo sequence of the next t times and then concatenated with the real value of the previous t times as a whole. The discriminator is used to distinguish whether it is a complete real value or a generated value. The loss function is Binary Cross Entropy, and the optimization process can be expressed as follows:

$$\begin{array}{cc}\hfill & \underset{D}{max}V(D,G)=E_{x\sim p_{\mathrm{data}}\left(x\right)}[log\left(D\left(x_{1:T+n}\right)\right)]+E_{x\sim p_{\mathrm{data}}\left(x\right)}[log(1-D\left([x_{1:T},G\left(x_{1:T}\right))\right])]\hfill \\ \hfill & \underset{G}{min}V(D,G)=E_{x\sim p_{\mathrm{data}}\left(x\right)}[log(1-D\left([x_{1:T},G\left(x_{1:T}\right))\right])]\hfill \end{array}$$

(3)

The extrapolated echo sequence became clearer by iteratively alternating training generator and discriminator. During the test phase, only the generator needs to be kept. The structure of the whole model is shown in the Figure 2. The optimization process is shown in Algorithm 1. In the early stage of training, the generator was weak, and the generated samples were significantly different from the real data. Since the discriminator can easily identify false samples, $log\left(1-D\left(\left[x_{1:T}^{\left(i\right)}G\left(x_{1:T}^{\left(i\right)}\right)\right]\right)\right)$ saturates quickly and cannot provide enough gradients. So, in practice, the generator can be trained to maximize $log\left(D\left(\left[x_{1:T}^{\left(i\right)}G\left(x_{1:T}^{\left(i\right)}\right)\right]\right)\right)$.

Algorithm 1: Minibatch stochastic gradient descent training of GAN-rcLSTM. The hyperparameter k for the number of steps applied to the discriminator is set to 4.

for number of training iterations do    for k steps do      • Sample minibatch of m samples with length $T+n$ {$x_{1:T+\mathrm{n}}^{\left(1\right)},\dots ,x_{1:T+\mathrm{n}}^{\left(m\right)}$} from ground truth generating distribution $p_{\mathrm{data}}\left(x\right)$.      • Update the discriminator by ascending its stochastic gradient:            ${\nabla }_{{\theta }_d}\frac{1}{m}{\sum }_{i=1}^{m}logD\left(x_{1:T+\mathrm{n}}^{\left(i\right)}\right)+log\left(1-D\left(\left[x_{1:T}^{\left(i\right)}G\left(x_{1:T}^{\left(i\right)}\right)\right]\right)\right)$    end for    • Sample minibatch of m samples with length $T+n$ {$x_{1:T+\mathrm{n}}^{\left(1\right)},\dots ,x_{1:T+\mathrm{n}}^{\left(m\right)}$} from ground truth generating distribution $p_{\mathrm{data}}\left(x\right)$.    • Update the generator by descending its stochastic gradient:          ${\nabla }_{{\theta }_g}\frac{1}{m}{\sum }_{i=1}^{m}log\left(1-D\left(\left[x_{1:T}^{\left(i\right)}G\left(x_{1:T}^{\left(i\right)}\right)\right]\right)\right)$ end for Any standard gradient-based learning rule can be used for gradient-based updates.

2.3. Data

We used RadarJS as the data set, which is provided by Jiangsu Meteorological Observatory of China and covers the years 2017 to 2018. The detailed features of the radar are shown in

Table 2. Radar parameter details.

Radar Parameters	Value
Band	S
Wavelength	10 cm
Frequency	2.86 GHZ
Pulse recurrence frequency	322–1014 Hz
Pulse width	1.57 $\mathsf{\mu }$s
Peak power	650 KW
Antenna gain	44 dB
Antenna aperture	8.5 m
Beam width	0.95°
Operation mode	STSR
Volume scan mode	PPI
Sagittal resolution	250 m

. The average elevation in Jiangsu is less than 50 m, and the plain area accounts for 86.89%, which greatly reduces the impact of ground clutter. The radar echo map is made up of 7 Doppler climate radar stations, which are scanned every 6 min, covering an area of about 311 km × 352 km (117.2° E–121.0° E, 31.9° N–34.7° N). After the conversion of the Cartesian coordinate system, the composite reflectivity was obtained, that is, the maximum value of the vertical projection of different elevation angles on the ground. The original radar resolution was $523\times 523$. Considering the limitations of existing computing power, we used average pooling to downsample all data. After processing, the resolution of the radar echo map was $100\times 100$. In order to further improve the data quality, we only kept the summer data (May–August) and filtered out some anomalous fragments such as hardware failure, biological interference, and electromagnetic interference. Finally, the 8 month radar data were divided into 71 consecutive, unequal length data files. Moreover, 65 files were randomly selected as the training set and the remaining 6 as the test set. We extracted 10,080 sequences from the training set and 1580 sequences from the test set. Considering the current computing power and the needs of meteorological operations application, the predicted length of our use case is 10 frames (1 h) and 20 frames (3 h). All use cases in the test set were only used for evaluation and display on the screen, and they were not fed into the model.

3. Results

3.1. Implementation

All experiments are implemented on Pytorch, and we use Adam [43] as the optimizer with L2 loss. In order to make the results comparable, all models have comparable roughly equal parameters numbers. The model consists of 4 layers of LSTM, and each layer of hidden states is 64. The batch size is set to 8, and the starting learning rate is 10${}^{-3}$. During the whole training process, the random seed is fixed to 0. All models will be trained for 20 epochs on a computer with a single NVIDIA GTX 1080ti GPU. We use Critical success index (CSI), false alarm rate (FAR), and probability of detection (POD) as metrics to evaluate model prediction accuracy. The three skill scores are defined as:

$$\begin{array}{cc}\hfill \mathrm{CSI}& =\frac{\mathrm{hits}}{\mathrm{hits}+\mathrm{misses}+\mathrm{falsealarms}}\hfill \\ \hfill \mathrm{FAR}& =\frac{\mathrm{falsealarms}}{\mathrm{hits}+\mathrm{falsealarms}}\hfill \\ \hfill \mathrm{POD}& =\frac{\mathrm{hits}}{\mathrm{hits}+\mathrm{misses}}\hfill \end{array}$$

(4)

where hits (prediction > threshold, truth > threshold), misses (prediction < threshold, truth > threshold), and false alarms (prediction > threshold, truth < threshold) will be calculated pixel by pixel. In our experiment, the thresholds are set to 10, 20, and 30 dBZ. Higher CSI and POD denote a better prediction result.

To show the predictive ambiguity problem, we use three different methods as quantitative metrics. The first is the pixel-based method called the Sum of Modulus of gray Difference (SMD) [44], the second is the Tenengrad [45] method based on the gradient magnitude, and the third is the Laplacian operator method [46,47] based on the second derivative. $I(m,n)$ represents the radar echo intensity function of size $N\times M$.

SMD calculates the sharpness of the image by differentiating adjacent pixels in the horizontal and vertical directions and then using the accumulated value of their modulus values. The calculation process is as follows:

$$SMD\left(I\right)=\sum _m^M\sum _n^N\left\{\right|f(m+1,n)-f(m,n)|+|f(m,n+1)-f(m,n)\left|\right\}$$

(5)

The Tenengrad method uses the Sobel operator to extract the edge. The calculation process of the Sobel operator and the gradient magnitude is shown as follows:

$$S_x=\left(\begin{array}{ccc}\hfill -1& \hfill 0& \hfill 1\\ \hfill -2& \hfill 0& \hfill 2\\ \hfill -1& \hfill 0& \hfill 1\end{array}\right)S_y=\left(\begin{array}{ccc}\hfill 1& \hfill 2& \hfill 1\\ \hfill 0& \hfill 0& \hfill 0\\ \hfill -1& \hfill -2& \hfill -1\end{array}\right)$$

(6)

$$S(m,n)=\sqrt{{\left[G_x(m,n)\right]}^2+{\left[G_y(m,n)\right]}^2}$$

(7)

where the $G_x$ and $G_y$ are, respectively, the convolution of the input image $I(m,n)$ with $S_x$ and $S_y$. The Tenengrad ambiguity of the entire image is the sum of the squares of the gradient magnitudes:

$$TEN\left(I\right)=\sum _m^M\sum _n^N{\left[S(m,n)\right]}^2$$

(8)

The Laplacian method extracts the high-frequency components of the image through the Laplacian operator, and then uses the sum or variance as the sharpness of the image:

$$L=\frac{1}{6}\left(\begin{array}{ccc}\hfill 0& \hfill -1& \hfill 0\\ \hfill -1& \hfill 4& \hfill -1\\ \hfill 0& \hfill -1& \hfill 0\end{array}\right)$$

(9)

$$\begin{array}{cc}\hfill LAP\left(I\right)& =\sum _m^M\sum _n^N\left|L(m,n)\right|\hfill \\ \hfill LA{P}_{-}VAR\left(I\right)& =\sum _m^M\sum _n^N\left[\right|L(m,n)|-\overline{L}{]}^2\hfill \\ \hfill \overline{L}& =\frac{1}{NM}\sum _m^M\sum _n^N\left|L(m,n)\right|\hfill \end{array}$$

(10)

3.2. Discussion

For LSTM-like networks, depth is the hyperparameter that affects the results most significantly, and

Table 3. MSE scores for the 10-frames-lead forecast for the models with different depths.

Depth	ConvLSTM	PredRNN	MIM	rcLSTM
1	21.32	-	-	-
2	20.58	18.18	18.61	18.16
3	18.29	17.66	16.98	17.32
4	17.04	16.33	15.95	15.76
5	17.12	16.51	15.49	15.35
6	17.03	17.28	16.73	15.28
7	19.28	17.13	16.98	15.16

shows the MSE scores of models with different depths. The results in the table show that when the number of layers is lower than five, with the increase in depth, the fitting ability of all models is gradually enhanced, and the prediction error is gradually reduced. However, when the number of layers exceeds five, the errors of other models no longer decrease and even increase, while the error of rcLSTM can still continue to decrease. Although the seven-layer rcLSTM can achieve better prediction results, in order to make the number of parameters of all models comparable and the training cost of the models acceptable, we fix the depth of all models to four.

Figure 3 shows two prediction examples of PredRNN and its improved model rcLSTM. In the prediction sequence of PredRNN, the echo with yellow color code (intensity greater than 30 dbz) disappears after five times, while rclstm can stably maintain the echo intensity. Figure 4 shows the proportion of the high-intensity echo. We can find that the echo intensity distribution of PredRNN shows a rapid downward trend, while the distribution of rcLSTM is closer to the real value, which is more obvious at a high threshold. The experimental results show that rcLSTM has stronger retention ability for high-intensity echo.

The sequence length was extended to 30 in order to compare the performance of each model in longer forecast period. Figure 5 shows the process of GAN-rcLSTM making the extrapolation results gradually clear. The first row is the sequence of real observations. Lines 2 to 7 are the extrapolation results. In the initial stage, the prediction effect of GAN-rcLSTM is very blurred and distorted. In the 10th epoch, high-intensity echoes greater than 30 dBZ in the southeast direction could already be predicted. In the 15th to the 20th epochs, the edge of the echo is sharper, and the details are closer to the real value, but there is still an error in the generated echo motion trajectory, and the overall orientation is biased to the west. The sequences generated in epoch 25 to 30 are sharper, while the low-intensity noise echoes with an intensity less than 5 dbz at the edges are significantly reduced.

Figure 6 shows the frame-wise accuracy scores of all models on the test set. We can find that the score drop of the GAN-rcLSTM is more gradual compared to other models. After 10 steps, our model outperformed the other models in both CSI and POD scores. We visualize two examples in Figure 7. GAN-rcLSTM maintains steady sharpness for three hours of forecast, while all other models start to blur after half an hour.

Table 4. Quantitative accuracy metrics of prediction results for different methods at a lead of 30 frames.

Model	MSE	CSI	POD	FAR
ConvLSTM	34.629	0.323	0.360	0.268
PredRNN	33.508	0.373	0.424	0.298
MIM	31.366	0.356	0.395	0.235
GAN-rcLSTM	32.662	0.402	0.472	0.259

shows the average accuracy scores of all models. We can find that GAN-rcLSTM does not achieve the best MSE score because the loss function during training is Binary Cross Entropy (BCE). However, it is more meaningful that it achieves the highest CSI and POD metrics within the acceptable FAR range.

Figure 8 and

Table 5. Quantitative sharpness metrics of prediction results for different methods at a lead of 30 frames.

Model	SMD	Tenengrad	Laplacian
ConvLSTM	7582	1,370,620	3.582
PredRNN	7534	1,394,062	4.441
MIM	7300	1,408,276	3.824
GAN-rcLSTM	8356	1,719,049	8.406

show the change in sharpness of the prediction process for different models. The scores of LSTM-like models drop rapidly after 5 frames (half an hour), while GAN-rcLSTM can maintain a relatively stable sharpness. Because LSTM calculates the mean square error time by time independently and then calculates the average value when calculating the loss, the later frames will therefore be more and more conservative in order to avoid gradient punishment, while GAN-rcLSTM takes the prediction sequence as a whole and simulates the real radar echo indexing through the constraints of the discriminator. We think that GAN-rcLSTM can provide better forecasts in short-term forecasting tasks.

After training, GAN-rcLSTM only keeps the generator and no longer needs the discriminator. So, forecast speed is perfectly acceptable in an operational setting.

Table 6. Time consumed for different models to make predictions. Each pair of numbers represents the time spent using GPU acceleration and using only the CPU. The statistical results are from the average time-consuming of 20 runs, in seconds.

Model	Time Required to Predict 10 Frames (GPU/CPU)	Time Required to Predict 20 Frames (GPU/CPU)	Time Required to Predict 30 Frames (GPU/CPU)
ConvLSTM	0.7 s/48.5 s	1.0 s/75.4 s	2.3 s/126.8 s
PredRNN	0.9 s/51.2 s	1.5 s/78.3 s	2.2 s/131.9 s
MIM	1.8 s/57.8 s	2.9 s/94.8 s	3.9 s/143.0 s
GAN-rcLSTM	0.9 s/51.4 s	1.5 s/82.0 s	2.3 s/134.2 s

shows the time consumed for different models to make predictions. The GPU used in the experiment is an Nvidia GTX 1080ti and the CPU is AMD Ryzen 7 2700X. We can find that the speed of GAN-rcLSTM is about the same as other models. Furthermore, even without GPU acceleration, the prediction time of GAN-rcLSTM is much lower than the period of radar update (6 min).

4. Conclusions

We investigate the problem of echo decay and blurring, which are the main drawbacks of LSTM-based radar extrapolation methods. This paper proposes a new recurrent neural network (rcLSTM) to overcome the degradation phenomenon of deep LSTM networks. We enhance the gradient transfer between multi-layer LSTMs by adding residual modules. Furthermore, we use rcLSTM as a generator and extend GAN-rcLSTM to solve the blurring problem in long sequence prediction. GAN-rcLSTM achieves the highest prediction accuracy score and the lowest ambiguity on the Jiangsu radar echo dataset.In order to further improve the prediction accuracy of radar echo generation and cancellation changes, we will also try to use an attention mechanism and an encoding–decoding structure to integrate multi-source observation data in the future.