DFFNet: A Rainfall Nowcasting Model Based on Dual-Branch Feature Fusion

Abstract

Timely and accurate rainfall prediction is crucial to social life and economic activities. Because of the influence of numerous factors on rainfall, making precise predictions is challenging. In this study, the northern Xinjiang region of China is selected as the research area. Based on the pattern of rainfall in the local area and the needs of real life, rainfall is divided into four levels, namely ‘no rain’, ‘light rain’, ‘moderate rain’, and ‘heavy rain and above’, for rainfall levels nowcasting. To solve the problem that the existing model can only extract a single time dependence and cause the loss of some valuable information in rainfall data, a prediction model named DFFNet, which is based on dual-branch feature fusion, is proposed in this paper. The two branches of the model are composed of Transformer and CNN, which are used to extract time dependence and feature interaction in meteorological data, respectively. The features extracted from the two branches are fused for prediction. To verify the performance of DFFNet, the India public rainfall dataset and some sub-datasets in the UEA dataset are chosen for comparison. Compared with the baseline models, DFFNet achieves the best prediction performance on all the selected datasets; compared with the single-branch model, the training time consumption of DFFNet on the two rainfall datasets is reduced by 21% and 9.6%, respectively, and it has a faster convergence speed. The experimental results show that it has certain theoretical value and application value for the study of rainfall nowcasting.

1. Introduction

Rainfall is an extremely important meteorological process in nature, which has a profound impact on human production activities, social economy, agriculture, water resource management, and urban planning. Therefore, rapid and accurate precipitation prediction in this region plays a critical role in the early warning of natural disasters, the protection of human lives and property security, and the prosperity and stability of society.

Researchers have divided the prediction technology based on different criteria. At the level of prediction time scale, rainfall prediction can be roughly divided into nowcasting, short-term prediction, medium-term prediction, and long-term prediction. Medium and long-term predictions have substantially enhanced in accuracy over extended periods of development, and the prediction technology has gradually matured. However, the development of nowcasting is obviously insufficient. Additionally, with the advancement of the economy, rainfall nowcasting is more and more in line with people’s needs. Therefore, rainfall nowcasting is in line with the development trend of rainfall prediction methods and the actual needs of human beings, and it has great room for improvement and application prospects.

According to the different types and quantities of meteorological elements used in the prediction model, the data used in rainfall nowcasting research can be divided into traditional meteorological elements, single-mode meteorological elements, and multi-mode meteorological elements. Although these prediction methods have certain advantages, they still have some shortcomings in the field of rainfall nowcasting. Since transformer and its variants overcome the drawbacks of RNN to a certain extent, Liu et al. [1] were inspired to propose a RLNformer model based on transformer and CNN to predict the rainfall levels in Northern Xinjiang. In contrast to the baseline models, the RLNformer model achieves the best prediction performance. However, due to the single-branch structure of the model, the extracted features are limited, resulting in a certain proportion of samples being wrongly predicted for other categories. Therefore, this paper improves the RLNformer model and proposes a rainfall nowcasting model named DFFNet based on dual-branch feature fusion.

The contributions of this study are summarized as follows:

(1)

Given the problem that the RLNformer model can only extract a single time dependency and cause the loss of some important information, we proposes a DFFNet model based on dual-branch feature fusion. The model can extract the time dependency and feature interactions in meteorological data, which avoids the loss of important information to a certain extent.

(2)

The public rainfall dataset of India, which is completely different from the climate in Northern Xinjiang, was selected for comparative experiments. Compared with the RLNformer, the training time consumption of the DFFNet model on this dataset was reduced by 6.9%, and it had a faster convergence speed, which verifies that the DFFNet model demonstrates a strong generalization capability.

(3)

The UEA dataset for multivariate time series classification was selected for comparative experiments. In contrast to the RLNformer model, the DFFNet model had different degrees of improvement in the prediction performance of each sub-dataset of the dataset, thus verifying that the DFFNet model is also applicable to the prediction of time series data in other fields.

2. Literature Review

2.1. Rainfall Nowcasting Based on Traditional Meteorological Elements

Rainfall nowcasting based on traditional meteorological elements mainly uses data extrapolation technology and statistical learning methods. Data extrapolation technology mainly uses radar echo maps as input data, which is the main technical means of precipitation nowcasting [2]. Ryu et al. [3] obtained the initial vector field by using the variational echo tracking algorithm, then they solved the Burgers equation to update the vector field. Nizar et al. [4] utilized satellite-derived curve relationships of cloud-top temperature and cloud effective radius at different time points. Then, they extracted feature values from these curves and employed logistic regression to predict the probability of extreme rainfall events occurring, enabling forecasts over 6 h in advance. Zhu et al. [5] proposed a rain-type adaptive pyramid optical flow method (A-PKLT) for radar echo extrapolation, which solves the problem that the PKLT optical flow algorithm is difficult to calculate the motion vector in mixed rainfall. The data extrapolation-based method has the advantages of a fast calculation speed, and it can meet the needs of real-time forecasting. However, this approach overlooks the nonlinear dynamics of large-scale air flow and thermodynamics, posing challenges in accurately describing complex rainfall processes. In the domain of statistical learning models, traditional time series methods have been extensively utilized and thoroughly researched, among which the ARIMA model [6] is more representative. Many studies have taken this model as a baseline and improved upon it. The ARIMA model plays an important role in the field of time series analysis and prediction. It consists of three parts: autoregressive (AR), integrated (I), and moving average (MA). The functions of these three parts are to predict the current value by using the linear relationship between the current value and several past values to transform non-stationary time series into stationary time series via differencing and to predict the current value by using the linear combination of the current value and several random disturbance terms in the past. De Luca et al. [7] proposed a hybrid model named PRAISE-MET, which integrates stochastic and numerical weather prediction models. It is designed to enhance rainfall forecasting at the watershed scale. Ray et al. [8] proposed a framework for predicting land surface temperature using the two-state Markov chain approach and the autoregressive approach, revealing a significant correlation between surface temperature and precipitation. Islam et al. [9] used a novel hybrid GEP-ARIMAX model for rainfall forecasting at 12 rainfall gauge sites in Western Australia, which had good rainfall forecasting capabilities compared to traditional linear and nonlinear models. The above studies are based on historical rainfall series for prediction, and these models are characterized by their low computational burden and high precision. Nevertheless, during severe convective weather, various meteorological elements change rapidly within a brief timeframe, and it is difficult to capture such changes using statistical learning methods of single meteorological elements. Therefore, the application scope of traditional statistical learning methods in rainfall nowcasting is limited.

2.2. Rainfall Nowcasting Based on Single-Modal Meteorological Elements

With the increase in the refinement of observation data, rainfall nowcasting based on single-modal meteorological elements has effectively made up for the shortcomings of the traditional meteorological elements by using machine learning and deep learning methods. Zhao et al. [10] put forward an hourly rainfall forecasting model (HRF) using a supervised learning algorithm, and they observed that this model exhibited excellent performance and superiority with regard to temporal resolution and prediction precision. Song et al. [11] introduced a novel summer hourly precipitation nowcasting model based on machine learning in the Eastern Alps, affirming the potential of machine learning methods in precipitation prediction. Maliyeckel et al. [12] implemented both LightGBM and SVR on the preprocessed dataset, subsequently integrating the predicted values from these models. Compared to using either model alone, the hybrid model achieved the lowest RMSE and provided more precise rainfall forecasting. Appiah-Badu et al. [13] used a variety of machine learning techniques for forecasting rainfall in various ecological zones of Ghana. They observed that the prediction performance of random forest, XGBoost and MLP was good, while the prediction performance of KNN was poor. Pirone et al. [14] performed probabilistic rainfall nowcasting at 19 weather stations in Southern Italy from 30 min to 6 h every 10 min in advance, and they found that the model could use only the current measured value as input to predict short-term rainfall using time and space information. Raval et al. [15] designed an optimized neural network to benchmark against various machine learning techniques. Their findings revealed that both traditional machine learning models and neural network-based approaches were capable of precisely forecasting precipitation. Adaryani et al. [16] evaluated the effectiveness of three rainfall forecasting techniques utilizing machine learning and deep learning, and they improved the performance of the slightly better PSO-SVR and LSTM methods by 3–15% and 2–10%, respectively. Rahman et al. [17] developed an advanced real-time rainfall forecasting system for smart cities, leveraging machine learning integration techniques and achieving more accurate predictions than other models. Despite the machine learning model’s ability to accurately capture the underlying nonlinear correlations linking the historical data of each meteorological element and the occurrence of precipitation events, several limitations persist, including vulnerability to local optima, the necessity for manual parameter tuning, and the tendency towards overfitting.

At present, the representative deep learning models for prediction using single-modal meteorological data mainly include LSTM [18], GRU [19], and MLP [20]. Both LSTM and GRU are models designed to solve the long-term dependence problem of standard RNNs. Among them, LSTM has a relatively complex structure and a large number of parameters, but it performs well on many tasks, while GRU has a relatively simple structure and a small number of parameters. It can achieve comparable performance to LSTM in some tasks, and the training convergence speed is faster. MLP is a common feedforward neural network model, which consists primarily of an input layer, a hidden layer, and an output layer. The existence of the hidden layer gives MLP a strong nonlinear fitting ability, and it can solve complex nonlinear problems. MLP is extensively utilized in various machine learning applications, including classification, regression, and clustering. Zhang et al. [21] applied the K-means clustering technique to segment the samples into four distinct categories, and they subsequently developed an LSTM model for each category. The findings indicated that the RMSE dropped by 0.65, while the threat scores for light rain and heavy rain increased. Khaniani et al. [22] implemented MLP and nonlinear autoregressive with exogenous input (NARX) models to forecast rainfall in Tehran. The NARX model exhibited superior accuracy compared to MLP in predicting both dry and rainy events. Fernández et al. [23] proposed an innovative architecture called Broad-Unet, derived from the core UNet model. In contrast to the core UNet model, the architecture stands out, with fewer parameters and superior forecasting precision. Yan et al. [24] substantiated the effectiveness of their TabNet-based rainfall prediction model using 5 years of meteorological data from 26 stations in the Beijing–Tianjin–Hebei region. Amini et al. [25] created some deep neural networks (DNNs) for rainfall nowcasting with an advance time of 5 min. The predictions from these deep neural networks were integrated with those from numerical weather prediction models using three ensemble models, thus significantly enhancing forecasting precision. Bhimavarapu et al. [26] introduced an improved regularization function (IRF) for rainfall prediction to mitigate bias, and they combined it with LSTM. The prediction performance of this method surpasses the comparison method. Despite the aforementioned research achieving fairly favorable outcomes, most models do not consider the temporality of rainfall data and the relationship between feature interactions, so they cannot extract deep features. As meteorological data are recorded sequentially, they can be classified as time series data. Several deep learning models dedicated to time series prediction [27,28,29] have been advanced in research, which are distinguished by their capability to capture the long-term dependencies of the series, thereby improving the prediction performance. At present, time series models are extensively employed in short-term load forecasting [30,31,32] and other fields, but they has relatively few applications in the field of meteorological forecasting. Therefore, they has broad development prospects to study rainfall data as time series data.

2.3. Rainfall Nowcasting Based on Multi-Modal Meteorological Elements

With the increasing richness of data description methods, the development of prediction technology based on multi-modal meteorological elements has a significant impact on the development of meteorological forecasting businesses. Multimodal fusion [33] refers to the process of integrating information from multiple resources. Each perceptual modality provides different types of information. Therefore, multimodal fusion can make up for the limitations between various modalities and obtain richer and more comprehensive expression and understanding ability, improving model prediction performance. Considering the success of multi-modal fusion technology in many fields, some researchers have applied it to the field of rainfall nowcasting. Zhang [34] et al. proposed a model called RN-Net that uses automatic weather station interpolation data and Doppler radar data as input data to forecast rainfall data for 0.5 h, 1 h, and 2 h in the future. Compared with related rainfall nowcasting studies, its threat scores increased by nearly four times, three times, and three times, respectively. Lyu et al. [35] proposed a dual machine learning model, DML, which fuses meteorological station data, satellite inversion data, and reanalysis data. It significantly enhanced the robustness and fusion accuracy of the algorithm. Although multi-modal fusion technology has made some progress, it is still in the exploratory stage. Its main challenges include data heterogeneity and low interpretability and reliability. Therefore, prediction methods based on single-mode meteorological elements still have certain advantages.

3. Study Area and Datasets

3.1. Study Area

Xinjiang belongs to a continental arid and semi-arid climate. The amount of long-term precipitation is less, and heavy precipitation mostly appears in the short-term. However, due to the fragile ecological environment, short-term heavy precipitation can easily lead to geological disasters and secondary disasters such as flash floods and debris flow, causing serious losses to social and economic construction in the region. There are significant differences in precipitation between Northern Xinjiang and Southern Xinjiang. According to statistics from 1961 to 2022, the average annual precipitation in Northern Xinjiang was 198.3 mm, while that in Southern Xinjiang was only 60.8 mm. Therefore, before the occurrence of short-term heavy rainfall, compared with the Southern Xinjiang region, the changes in meteorological elements in the Northern Xinjiang region are relatively small due to the relatively abundant precipitation, which makes it difficult for the prediction model to capture various meteorological factors. As a result, the probability of serious consequences caused by missing reports is greatly increased. Therefore, it is relatively difficult to conduct rainfall nowcasting in Northern Xinjiang. Based on the above reasons, we selected a total of six meteorological stations in Northern Xinjiang as the focus area for this research. The location and specific details of each meteorological station in this region are depicted in Figure 1 and

Table 1. Information about each meteorological station.

Station No.	Latitude (°)	Longitude (°)	Altitude (m)
51076	47.73	88.08	735.3
51087	46.98	89.52	807.5
51133	46.73	83.00	534.9
51156	46.78	85.72	1322.1
51243	45.62	84.85	450.3
51463	43.78	87.37	935.0

, respectively.

From Figure 1 and Table 1, it is evident that the latitude, longitude, and altitude vary significantly between each station in the research region, leading to diverse rainfall patterns between stations. Hence, predicting rainfall accurately and promptly using meteorological data from stations in northern Xinjiang poses a significant challenge.

3.2. Rainfall Datasets

The Northern Xinjiang rainfall dataset used in this paper was constructed by Liu et al. [1]. To validate the model’s capacity for generalization, the public rainfall dataset of India, which is completely different from the climate of Northern Xinjiang and has rich rainfall, was selected for comparative experiments. The temporal resolution of the two rainfall datasets is 3 h, and the detailed information is presented in

Table 2. Details of the two rainfall datasets.

Dataset Name	Feature Counts	Sequence Length	Category Counts	Sample Counts
Northern Xinjiang	80	8	4	36,411
India	80	16	4	46,112

.

The dimensions of both rainfall datasets are [N, L, C], where N represents the quantity of samples, L represents the number of time steps, and C represents the number of features.

According to the literature [36], Liu et al. divided the 3 h rainfall of the two rainfall datasets into four levels: ‘no rain’, ‘light rain’, ‘moderate rain’, and ‘heavy rain and above’. The classification standards are as shown in

Table 3. Three-hour precipitation levels.

Rainfall Levels	No Rain	Light Rain	Moderate Rain	Heavy Rain and Above
3 h precipitation (mm)	0	(0, 5]	(5, 10]	(10, )

.

After dividing the rainfall into four rainfall levels according to Table 3, ‘no rain’ was encoded as the integer 0, ‘light rain’ was encoded as the integer 1, ‘moderate rain’ was encoded as the integer 2, and ‘heavy rain and above’ was encoded as the integer 3.

3.3. UEA Dataset

The UEA dataset [37] is an important open-source dataset for multivariate time series classification research compiled by the University of East Anglia in the United Kingdom. The dataset covers a variety of fields and application scenarios, including a total of 31 sub-datasets from the fields of finance, medical care, and exercise physiology. The sequence length, category counts, and channel counts of each sub-dataset in the UEA dataset are quite different, and they need to be screened for comparison. After comparison, we finally selects four sub-datasets for auxiliary verification of the rainfall nowcasting model. The detailed information of the four sub-datasets of the selected UEA dataset is shown in

Table 4. Details of the four sub-datasets of the UEA dataset.

Dataset Name	Channel Counts	Sequence Length	Category Counts	Sample Counts
FaceDetection	144	62	2	9414
Epilepsy	3	206	4	275
NATOPS	24	51	6	360
PEMS-SF	963	144	7	440

.

In Table 4, the FaceDetection sub-dataset represents the time series changes in the magnetoencephalography generated by the subject when observing blurred or clear face images, which is used to determine whether the subject sees blurred or clear face images. The Epilepsy sub-dataset represents the motion time series data of six participants wearing a 3D accelerometer on their wrists, which is used to determine their four motion states (walking, running, repeated movement, and simulated seizures). The NATOPS sub-dataset represents time series data generated by sensors worn on the subject’s hands, elbows, wrists, and thumbs to determine the various gestures that control aircraft movement in naval aviation training. The PEMS-SF sub-dataset represents 15-month traffic flow time series data from different locations and is used to determine which day of the week each sample belongs to.

4. Theories and Methods

4.1. Convolutional Neural Networks

Convolutional neural networks (CNNs) are a foundational algorithm in the realm of deep learning. Due to the introduction of the convolution operator, it has the capability of local perception. At the same time, due to the existence of mechanisms such as parameter sharing, sparse connection, and the pooling layer, CNNs have fewer parameters, which makes them more efficient in the training process. According to the different data dimensions they can handle, CNNs are usually divided into 1D convolutional neural networks (1D-CNNs), 2D convolutional neural networks (2D-CNNs), and 3D convolutional neural networks (3D-CNNs). 2D-CNNs and 3D-CNNs are commonly employed in tasks such as image recognition and image classification, while 1D-CNNs are frequently employed in natural language processing as well as the analysis of time series such as speech and signals. The process of a single convolution kernel 1D-CNN with a convolution kernel size of 3 and a step size of 1 processing time series data with a feature number of 5 and a time step size of 3 is shown in Figure 2.

The dimension of the features formed after the original data are processed by the 1D-CNN can be calculated using the corresponding equation. Assuming that the quantity of time steps in the input time series data is $F_{in}$, the padding size is p, the convolution kernel size is k, the expansion coefficient is d, and the convolution kernel step size is s, the number of features $F_{out}$ formed after using the 1D-CNN to extract features from the original sequence can be expressed by the following equation.

$$F_{out}={\displaystyle \frac{F_{in}+2\times p-d\times (k-1)-1}{s}}+1$$

(1)

Suppose that the dimensions of the input time series data are [N, L, C], where N represents the number of samples in each training batch, L represents the number of time steps contained in each sample, and C represents the number of features contained in each time step. Based on Equation (1), the value of $F_{in}$ is the value of L, then the time series data are processed by a single convolution kernel 1D-CNN, and the data dimension becomes to [N, $F_{out}$, 1]. The output generated by the 1D-CNN with multiple convolution kernels will be superimposed together to form the final output, which is characterized by each convolution kernel learning different features so that the superimposed output has the advantage of capturing richer feature information. Assuming that the number of convolution kernels set by the 1D-CNN is M, according to Equation (1), after the data with input dimensions [N, L, C] are processed by the 1D-CNN, the data dimensions become [N, $F_{out}$, M].

4.2. Transformer

The transformer model was presented by the Google Machine Translation team in 2017 [38]. It uses the self-attention mechanism to replace the recurrent structure in the sequence-to-sequence model, it performs well in machine translation tasks, and it has gradually become an important model in the domain of natural language processing (NLP). As research progresses, the transformer has also become pivotal in domains like computer vision (CV) and long time series prediction due to its agile flexibility and powerful modeling capabilities.

The original transformer model is a Seq2Seq architecture built on a multi-head attention mechanism. The entire network architecture includes an encoder and decoder. The encoder consists of three primary components: multi-head attention, feed-forward network (FFN), and layer normalization (add and normalize). The decoder includes a cross-attention module based on the encoder module and integrates the multi-head attention mechanism with the mask mechanism. The diagram depicting the original transformer’s structure is shown in Figure 3.

The transformer model can be composed of multiple encoders and decoders. The classical transformer model consists of six encoders and six decoders. In the encoding module, the output of each encoder is used as the input of the next encoder, while in the decoding module, the output of each decoder is combined with the output of the entire encoding module as the input of the next decoder.

At the heart of the transformer lies the multi-head attention mechanism, which uses h groups of different attention linear mappings to replace the single-layer attention mechanism to process Q, K, and V. It inputs the transformed Q, K, and V into the h groups of attention mechanisms in parallel, then it concatenates the h groups of outputs obtained after the attention processing. Finally, it transforms them again through the linear layer to obtain the final output sequence. For each head, a scaled dot-product is used to implement the calculation of self-attention. Figure 4a,b are the structure diagrams of self-attention and multi-head attention, respectively.

The self-attention output is illustrated in Equation (2):

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(2)

where Q, K, and V are the query vector matrix, key vector matrix, and value vector matrix, respectively, and $d_k$ is the input dimension of vector K.

For the multi-head attention mechanism, it maps Q, K, and V through h different scaled dot products and then concatenates different attentions. Each attention head projects the input into a unique sub-representation space, enabling the model to concentrate on different aspects within these subspaces. The computational process is illustrated in Equation (3).

$$\begin{array}{cc}\hfill MultiHead(Q,K,V)& =Concat(hea{d}_1,hea{d}_2,\dots ,hea{d}_h)W^O,\hfill \\ \hfill hea{d}_i& =Attention(Q{W}_i^Q,K{W}_i^K,V{W}_i^V)\hfill \end{array}$$

(3)

where, $W_i^Q\in R^{d_{model}\times d_k}$, $W_i^K\in R^{d_{model}\times d_k}$, $W_i^V\in R^{d_{model}\times d_v}$, $W^O\in R^{d_{model}\times h{d}_v}$.

4.3. RLNformer Model

The RLNformer model [1] is composed of basic layers such as Conv1D and the transformer’s encoder structure, which are used to conduct rainfall level nowcasting in Northern Xinjiang. It has achieved excellent prediction performance.The model architecture is illustrated in Figure 5.

In Figure 5, the core of RLNformer is the improved encoder structure of the transformer, which is used to extract the time dependence in rainfall time series data. In this structure, the Conv1D layers is employed as a substitute for the word-embedding layer. The convolution kernel moves along the time direction of the rainfall time series to extract the relationship between different moments of each feature time series, form new features, and map the whole sequence of the same variable into a high-dimensional feature representation. The whole time change of the extracted feature is regarded as a token. After position embedding, the multi-head attention mechanism is used to capture the time dependence between its different time steps to improve the accuracy of time series prediction. Meanwhile, before the multi-head attention mechanism, the normalization layer is placed, and the feedforward layer is employed to improve the stability of the model.

The RLNformer model can perform parallel computing on the structure and can fully extract the dependencies of different time steps, which improves the efficiency of feature extraction to a certain extent.

4.4. Dual-Branch Feature Fusion Prediction Model Based on Improved RLNformer

In rainfall time series, there are two main dependence relationships, namely the time dependence relationship and feature interaction relationship. Time dependence refers to the correlation of data in time. For example, there is a close connection between the weather state at the current moment and the weather state at the next moment. Therefore, accurately extracting the time dependence relationship in meteorological data is crucial for weather prediction. The feature interaction relationship refers to the situation where there is interrelation and mutual influence between data features. When multiple features interact, it will lead to the occurrence of no rain and rainfall events. Since the RLNformer model adopts a single-branch structure, it is mainly used to extract the time dependence of time series data without considering the interaction between various meteorological elements, resulting in the omission of some critical features in feature extraction. Therefore, the RLNformer model cannot distinguish between the categories of some samples, resulting in some samples being mistakenly classified into other categories.

To tackle the challenge that the RLNformer model cannot extract the time dependence relationship and the interaction relationship of meteorological variables at the same time, we propose a rainfall nowcasting model named DFFNet based on dual-branch feature fusion on the basis of the RLNformer model. The model aims to capture the temporal dependence and feature interaction relationship in meteorological data more comprehensively.

The DFFNet model is mainly composed of two branch modules, which are the temporal dependency extraction module and the feature interaction relationship extraction module. Each branch module is composed of the Conv1D layer and the transformer’s encoder structure. At the same time, to better meet the needs of the rainfall nowcasting task, the internal structure of the original transformer encoder is also partially modified to form a new encoder structure. The model structure is shown in Figure 6.

4.4.1. Temporal Dependency Extraction Module

The left branch in Figure 6 is the temporal dependency extraction module (TDE), which is the RLNformer model. The dimensions of the input data are converted to [N, C, L], and the dimensions of each dataset are $X=[X_1;\cdots ;X_C]\in {\mathbb{R}}^{C\times L}$. Firstly, the convolution kernel of the Conv1D layer extracts temporal features along the time direction. Currently, the extracted feature dimension can be expressed as $X\in {\mathbb{R}}^{n\times d}$, where n is the number of filters in the convolutional layer and d is the output dimension. Then, the time series change in each feature is regarded as a whole, and the position embedding is obtained using sine and cosine functions of different frequencies. The feature dimension after position embedding is still $X\in {\mathbb{R}}^{n\times d}$. Finally, the position-embedded data are flipped, and the data dimension becomes $X\in {\mathbb{R}}^{d\times n}$ so that it can be input into the encoder structure. The multi-head attention mechanism is employed to extract the time dependence of the rainfall sequence. By stacking the multi-layer encoder structure, the expression ability of the model is increased. After encoder processing, the data dimension is $X\in {\mathbb{R}}^{d\times n}$. Now, the module completes the extraction of time dependence. The entire TDE module can be expressed in Equation (4):

$$TDE\left(x\right)=\mathrm{Encoder}{\left(\mathrm{Reshape}\left(\mathrm{Embedding}\left(\mathrm{Conv}1\mathrm{D}\left(x\right)\right)\right)\right)}_{\times n}$$

(4)

where n is the number of stacked encoder structures.

4.4.2. Feature Interaction Relationship Extraction Module

The right branch in Figure 6 is the feature interaction relationship extraction module (FIRE). Firstly, Conv1D is used to map the original features into low-dimensional feature representations, which reduces redundant information to a certain extent. Then, the normalized layer and ReLu activation function are carried out. The feature dimension after dimensionality reduction can be expressed as $X\in {\mathbb{R}}^{n\times L}$, where n is the number of filters in the convolution layer. Because the feature dimension of the data is relatively long, the idea of patching the image in the vision transformer [39] is introduced to segment the original data containing the feature dimension. It is assumed that $X\in {\mathbb{R}}^{n\times L}$ is divided into a sequence with K as the patch. When the feature dimension is not enough for a patch, padding filling is performed, and a total of $\lceil {\displaystyle \frac{n}{K}}\rceil$ patches can be divided. At this time, the data dimension can be expressed as $X\in {\mathbb{R}}^{L\times \lceil {\displaystyle \frac{n}{K}}\rceil \times K}$. Then, the data are dimensionally transformed to merge the time dimension and the batch size to facilitate the extraction of the feature interaction relationship. At this time, the dimensions of the input data are [$\mathrm{N}\times \mathrm{L}$, $\lceil {\displaystyle \frac{\mathrm{n}}{\mathrm{K}}}\rceil$, K], and the dimensions of each input dataset are $X\in {\mathbb{R}}^{\lceil {\displaystyle \frac{n}{K}}\rceil \times K}$. After patching, the feature dimensions are divided, which is beneficial for representing the local features of the sequence. Finally, the position embedding of the segmented features is input into the encoder structure, and the multi-head attention mechanism is used to extract the relationship between different slices. The extracted features are dimensionally transformed again so that the dimensions become [N, L, $\lceil {\displaystyle \frac{\mathrm{n}}{\mathrm{K}}}\rceil \times \mathrm{K}$].

The FIRE module divides the original feature dimensions, shortens the length of the sequence, helps to characterize the local features of the sequence, and improves the efficiency of extracting the relationship between features. The entire FIRE module is expressed by Equation (5):

$$FIRE\left(x\right)=\mathrm{Rearrange}\left(\mathrm{Encoder}{\left(\mathrm{Patching}\left(\mathrm{Reshape}\left(\mathrm{Conv}1\mathrm{D}\left(x\right)\right)\right)\right)}_{\times n}\right)$$

(5)

where n is the number of stacked encoder structures.

The main function of the feedforward layer in the transformer is to perform nonlinear transformation and feature interaction on the input features, but the parameters of the linear layer are related to the dimensions of the input and output. For high-dimensional input and output, the linear layer needs to learn many parameters, which makes the model more easy to overfit and greatly increases the training time and space consumption of the model. Therefore, this paper uses two Conv1D layers to replace the linear layers in the original feedforward layer. While ensuring the function of the original feedforward layer, the training efficiency is improved to a certain extent.

The temporal dependency extracted by the TDE module and the interactions between the features extracted by the FIRE module are spliced in the feature dimension, then the stitched data are transformed into dimensions, and the transformed data are predicted using the MLP model.

5. Experimental Design and Result Analysis

5.1. Experimental Design

5.1.1. Baseline Models

To gauge the forecasting effectiveness of the DFFNet model, in addition to selecting the RLNformer model for comparison, this paper also selected two recent studies on time series prediction to compare. The models proposed in these two studies are TimesNet [40] and iTransformer [41]. Among them, the theoretical results of TimesNet were published at the International Conference on Learning Representation (ICLR) in 2023, and the model achieved comprehensive leadership in five major tasks. iTransformer is the latest research result in the field of time series prediction in 2024, and it has achieved the best prediction performance in multivariate prediction tasks.

5.1.2. Experimental Process

In this paper, the experimental process is designed in detail. Figure 7 shows the overall experimental flow chart of the DFFNet model.

The entire experimental process mainly covers three key steps, including dataset partitioning, model training and prediction, and model evaluation. Below is a detailed description of each step.

(1)

Dataset partition. In this paper, different sliding windows are set for the two rainfall datasets. After completing the sliding window operation for each station’s data, 80% of the data are used for training, 10% are used for validation, and the remaining 10% are used for testing, all organized chronologically. Finally, these sets from each station are merged to form the consolidated training, validation, and testing datasets. The UEA dataset is divided in its default way. The division results of the rainfall datasets and the UEA dataset are shown in

Table 5. The division results of each dataset.

Dataset Type	Dataset Name	Training Set Size	Validation Set Size	Testing Set Size
Rainfall nowcasting	Northern Xinjiang	[29,127, 8, 80]	[3638, 8, 80]	[3646, 8, 80]
datasets	India	[36,888, 24, 80]	[4608, 24, 80]	[4616, 24, 80]
UEA dataset	FaceDetection	[5890, 62, 144]	[3524, 62, 144]	[3524, 62, 144]
	Epilepsy	[137, 206, 3]	[138, 206, 3]	[138, 206, 3]
	NATOPS	[180, 51, 24]	[180, 51, 24]	[180, 51, 24]
	PEMS-SF	[267, 144, 963]	[173, 144, 963]	[173, 144, 963]

.

(2)

Model training and prediction. In the training phase, for all datasets, the batch size is set to 64, the learning rate is set to 0.001, the loss function is set to the cross entropy, the optimizer is set to Adam, and the number of iterations is set to 1000. Due to the different time dependences and sequence lengths of each dataset, we sets separate hyperparameters for each dataset according to the actual situation. The specific parameter settings are shown in

Table 6. Different dataset parameter settings.

Dataset Type	Dataset Name	Slice Size	Channel Counts	Number of Encoder Layers	Number of Multi-Head Attention
Rainfall	Northern Xinjiang	80	4	4	4
nowcasting datasets	India	80	8	5	4
UEA dataset	FaceDetection	144	4	3	3
	Epilepsy	3	3	3	3
	NATOPS	24	4	3	3
	PEMS-SF	963	4	3	3

. In the prediction stage, we uses the trained model to predict the testing set of each dataset to verify the generalization ability of the model on unknown data.

(3)

Model evaluation. In this research, a range of assessment metrics are utilized to thoroughly evaluate the model’s predictive performance, ensuring its robustness. The multi-classification evaluation indicators are detailed in Section 5.2.1. At the same time, we also use training time consumption and verification loss to measure the efficiency of model training.

5.1.3. Experimental Environment

All the experiments in this study were conducted on a remote server. The server was equipped with Ubuntu 22.04.2 as its operating system and utilized an NVIDIA GeForce GTX 3090 graphics card for computational tasks. Visual Studio Code 1.81.7 served as the integrated development environment, with PyTorch employed as the computing framework. The Python version used was 3.8.17.

5.2. Experimental Result Analysis

5.2.1. Evaluation Metrics

This paper utilizes four evaluation metrics, all derived from the confusion matrix. The confusion matrix for multi-classes is displayed in

Table 7. Multi-class confusion matrix.

		Predicted Value
		0	1	2	3
True value	0	$T_0^0$	$P_0^1$	$P_0^2$	$P_0^3$
	1	$P_1^0$	$T_1^1$	$P_1^2$	$P_1^3$
	2	$P_2^0$	$P_2^1$	$T_2^2$	$P_2^3$
	3	$P_3^0$	$P_3^1$	$P_3^2$	$T_3^3$

.

In the multi-classification confusion matrix, $T_i^j$ denotes the number of samples correctly predicted as the i-th category, and $P_i^j$ denotes the number of samples that were incorrectly predicted the i-th category as the j-th category, where i is the label of the actual category to which the sample belongs, j is the label of the model prediction category, and $i,j\in \{0,1,2,3\}$.

The evaluation metrics used for multi-classification tasks are introduced in detail below.

(1)

Accuracy

$$Accuracy={\displaystyle \frac{{\sum }_{i=0}^3{T}_i^i}{Total}}$$

(6)

Here, $Total$ represents the number of all samples.

(2)

Precision

$${Precision}_i={\displaystyle \frac{T_i^i}{T_i^i+{\sum }_k{P}_k^i}},i=0,1,2,3;k=0,1,2,3;k\ne i$$

(7)

Precision is used to evaluate the local performance of the model prediction, focusing on the correctness of the model in the samples predicted as the i-th category while ignoring the prediction of the model for other categories of samples.

(3)

Recall

$${Recall}_i={\displaystyle \frac{T_i^i}{T_i^i+{\sum }_k{P}_i^k}},i=0,1,2,3;k=0,1,2,3;k\ne i$$

(8)

Recall is also used to evaluate the local performance of the model prediction, focusing on the statistics of real samples in a certain category.

(4)

${\mathrm{F}}_1$ score

$$F_1-{score}_i={\displaystyle \frac{2\ast {Precision}_i\ast {Recall}_i}{{Precision}_i+{Recall}_i}},i=0,1,2,3$$

(9)

${\mathrm{F}}_1$ score takes into account both precision and recall. It can be used to measure the comprehensive performance of the model when dealing with unbalanced datasets.

5.2.2. Experimental Results and Comparative Analysis of Rainfall Datasets

DFFNet is used to predict the test sets of two rainfall datasets, and a confusion matrix of the prediction results on the two datasets is obtained. Meanwhile, it is compared with the confusion matrix of the prediction results of the RLNformer model on the two rainfall datasets, and it is analyzed. The comparison results are shown in Figure 8 and Figure 9.

It can be seen from Figure 8 that compared with the RLNformer, the number of samples that the DFFNet incorrectly predicts in the ‘light rain’ category as being in the ‘no rain’ and ‘moderate rain’ categories is significantly reduced. For the prediction of the ‘no rain’ category, the two models are not significantly different. The number of samples correctly predicted as being in ‘no rain’ category by DFFNet is slightly lower than that predicted by RLNformer, but the samples in this category are not wrongly predicted as being in ‘moderate rain’ category. The number of samples correctly predicted by the DFFNet as being in ‘moderate rain’ category is slightly lower than that of those predicted by RLNformer, while for the prediction of the ‘heavy rain and above’ category, the prediction results of the two models are equivalent.

Based on the overall prediction results, it can be seen that the improvement effect of the DFFNet model is not very obvious. The reason is that the number of samples in the ‘moderate rain’ and ‘heavy rain and above’ categories in the training set is too small, and it is difficult for the model to fully learn the complex patterns during the training process. Although the overall performance is not greatly improved, the DFFNet model ensures that the prediction accuracy of the ‘light rain’ category is significantly improved without significantly reducing the prediction accuracy of the other categories, indicating that the dual-branch structure extracts the relationship between time dependence and feature interaction to a certain extent, which has a certain role in promoting the prediction performance of the model.

As can be seen from Figure 9, compared with the RLNformer, the DFFNet model improved the prediction of each category to varying degrees. For the ‘no rain’ category, the number of samples incorrectly predicted by the DFFNet as being in the ‘light rain’ category decreased by 39. For the ‘light rain’ category, the number of samples incorrectly predicted by the DFFNet model as being in the ‘no rain’ category decreased by 34, and the number of samples incorrectly predicted as being in the ‘moderate rain’ category decreased by 3. For the ‘moderate rain’ category, the DFFNet model only predicted some samples as being in the ‘light rain’ category, and it did not incorrectly predict them as being in the ‘heavy rain and above’ category like the RLNformer model did. For the ‘heavy rain and above’ category, the number of samples predicted by the DFFNet model for the correct category increased by four. Overall, the prediction performance of the DFFNet model on the India public rainfall dataset was relatively improved.

Based on the comparison results of the confusion matrices of the DFFNet model and the RLNformer model on the two rainfall datasets, it can be seen that the prediction performance of the DFFNet model using the dual-branch structure was improved to varying degrees on the two rainfall datasets. The improvement effect on the India public rainfall dataset is particularly obvious. The results from the confusion matrix clearly illustrate that the dual-branch structure within the DFFNet model effectively captured the complex interdependencies in meteorological data, demonstrating its robustness.

Next, the evaluation metrics of each category on the two rainfall datasets were calculated based on the results of the confusion matrix to further conduct a comparative analysis of the prediction performance of the model. Meanwhile, the TimesNet and iTransformer models were selected for comparison to comprehensively evaluate the prediction performance of the model. The comparison results of the selected models on the two rainfall datasets are shown in

Table 8. Comparison of evaluation metrics of each model on the two rainfall datasets. The bold represents the optimal value.

Models	Index	Northern Xinjiang				India
		0	1	2	3	0	1	2	3
TimesNet	Precision	0.976	0.880	0.455	1.000	0.937	0.807	0.340	0.786
	Recall	0.981	0.710	0.714	0.500	0.941	0.804	0.360	0.355
	${\mathrm{F}}_1$	0.979	0.786	0.556	0.667	0.939	0.805	0.350	0.489
iTransformer	Precision	0.972	0.883	0.733	1.000	0.938	0.822	0.550	0.818
	Recall	0.990	0.727	0.786	0.750	0.949	0.839	0.353	0.290
	${\mathrm{F}}_1$	0.981	0.798	0.759	0.857	0.943	0.831	0.416	0.429
RLNformer	Precision	0.970	0.897	0.846	1.000	0.937	0.812	0.553	0.909
	Recall	0.992	0.716	0.786	0.750	0.943	0.846	0.347	0.323
	${\mathrm{F}}_1$	0.981	0.796	0.815	0.857	0.940	0.829	0.426	0.476
DFFNet (Ours)	Precision	0.974	0.887	1.000	1.000	0.949	0.853	0.672	1.000
	Recall	0.991	0.756	0.714	0.750	0.956	0.872	0.507	0.452
	${\mathrm{F}}_1$	0.982	0.816	0.833	0.857	0.952	0.862	0.578	0.622

.

As can be seen from Table 8, for the Northern Xinjiang rainfall dataset, compared with the other models, the DFFNet model had the highest accuracy in the ‘no rain’ and ‘moderate rain’ categories, while the recall was lower than the RLNformer model. The reason is that the DFFNet model predicts the number of samples in the category corresponding to the other categories, but after combining the two metrics, the DFFNet model had the highest prediction performance for these two categories. The accuracy of the ‘light rain’ category prediction was lower than that of the RLNformer model, but the recall was higher, which led to the best prediction performance for ‘light rain’ category prediction. In addition to the TimesNet model, the prediction results of the three models for the ‘heavy rain and above’ category were consistent. For the India public rainfall dataset, since the number of correctly predicted samples in each category was more than that of the RLNformer model, the accuracy and recall of the DFFNet model in each category were improved, and they were also better than the other two comparison models.

On the two rainfall datasets, the accuracy of each model in individual categories is 1, because no samples of other categories were wrongly predicted as being in the category. Due to the limited sample size in the categories of ‘moderate rain’ and ‘heavy rain and above’, when the prediction results change slightly, the values of each evaluation index will change greatly, which is the reason why the DFFNet model greatly improved the prediction performance of the categories of ’moderate rain’ and ’heavy rain and above’ in the India rainfall dataset.

Since the DFFNet adopts a dual-branch structure, while the other three baseline models are all single-branch structures, the time consumption in the training process may be much higher than that of the baseline models in theory. Therefore, we compare the training time consumption of the selected models on the two rainfall datasets. The comparison results are shown in Figure 10.

It can be seen from Figure 10 that the training time consumption of each model on the two rainfall datasets showed the same trend; that is, the models with the highest and lowest training time consumption were TimesNet and DFFNet, respectively, while the training times of iTransformer and RLNformer were not significantly different, at a medium level. Compared with the RLNformer model, the training times of the DFFNet model on the Northern Xinjiang rainfall dataset and the India public rainfall dataset were reduced by 21% and 9.6%, respectively.

By comparing the changes in the validation set loss function of each model during the training process, we can evaluate the convergence of each model. The verification loss changes of each model on the Northern Xinjiang rainfall dataset are shown in Figure 11.

As can be seen from Figure 11, compared with the other models, given the same batch size and learning rate, the DFFNet model showed a faster convergence speed and required the least training epochs, which once again verifies the rationality of the model structure. For the TimesNet model, with the increase in the epoch, the oscillation amplitude of the verification loss was larger, indicating that there was a serious overfitting phenomenon in the training process, which led to the model learning noise or specific sample features on the training set rather than the overall regularity. For the iTransformer and RLNformer models, the convergence speed was slow, which required a large number of training epochs to fit the data, resulting in an increase in the training time consumption.

In summary, on the rainfall datasets, compared with the baseline models, the DFFNet model showed a higher prediction accuracy, shorter training time, and faster convergence speed, which fully verifies the excellent performance of the DFFNet model in the rainfall nowcasting task. This highlights its superiority in processing rainfall time series.

5.2.3. Ablation Experiment

The DFFNet model is composed of two branch modules: TDE and FIRE. In order to verify the rationality of the model structure, an ablation experiment was carried out. The TDE and FIRE single-branch modules were used to predict the two rainfall datasets, and the effectiveness of the model was evaluated. The outcomes of the ablation experiments are presented in

Table 9. Ablation experiment results. The bold represents the optimal value.

Dataset	Rainfall Level	TDE Branch Module			FIRE Branch Module			DFFNet (Ours)
		Precision	Recall	${\mathrm{F}}_1$	Precision	Recall	${\mathrm{F}}_1$	Precision	Recall	${\mathrm{F}}_1$
Northern Xinjiang	0	0.970	0.992	0.981	0.971	0.991	0.981	0.974	0.991	0.982
	1	0.897	0.716	0.796	0.887	0.713	0.791	0.887	0.756	0.816
	2	0.846	0.786	0.815	0.538	0.500	0.519	1.000	0.714	0.833
	3	1.000	0.750	0.857	0.429	0.750	0.545	1.000	0.750	0.857
India	0	0.937	0.943	0.940	0.900	0.960	0.929	0.949	0.956	0.952
	1	0.812	0.846	0.829	0.822	0.764	0.792	0.853	0.872	0.862
	2	0.553	0.347	0.426	0.571	0.312	0.403	0.672	0.507	0.578
	3	0.909	0.323	0.476	0.000	0.000	0.000	1.000	0.452	0.622

.

Based on the results of the ablation experiments, it can be found that the prediction performance of the TDE module for the ‘light rain’ and ‘moderate rain’ categories was lower than that of the DFFNet model, and the prediction accuracy of the FIRE module for two minority categories, namely ‘moderate rain’ and ‘heavy rain and above’, was significantly lower than that of DFFNet and TDE for these two categories. On the India rainfall dataset, the DFFNet model had a better prediction performance for each category than the two branch modules, showing certain advantages for the prediction of two minority categories, while the FIRE module could not predict the category of ‘heavy rain and above’.

The results of the ablation experiments show that the feature relationships extracted by the two branches in DFFNet complement each other, and they work together to improve the prediction performance of each category of samples.

5.2.4. Experimental Results and Comparative Analysis of the UEA Dataset

In this paper, the DFFNet model is trained by using the four sub-datasets of the UEA dataset, and the trained DFFNet model is used to predict the corresponding testing set. The confusion matrix of the prediction results of each sub-dataset is obtained and compared with the prediction results of the RLNformer model. The specific comparison results are shown in Figure 12, Figure 13, Figure 14 and Figure 15.

As can be seen from Figure 12, in contrast to the RLNformer model, the DFFNet model achieved significant improvements in the prediction accuracy of the second category, which indicates that the DFFNet model achieved relatively good results in accurately capturing data features and improving prediction performance.

As can be seen from Figure 13, in contrast to the RLNformer model, the DFFNet model achieved significant improvements in the prediction accuracy of the second and fourth categories. Although its prediction performance for the first and third categories decreased slightly, the overall performance of the DFFNet model was still better than the RLNformer model. This is due to the comprehensive consideration of the time dependence and feature interaction of the DFFNet model, which better distinguishes the complex relationships between different categories.

As can be seen from Figure 14, in contrast to the RLNformer model, the DFFNet model significantly improved the prediction accuracy of the third and sixth categories. Although its prediction accuracy for the first two categories was slightly lower, the overall accuracy was still better than the RLNformer model.

From Figure 15, it can be seen that compared with the RLNformer model, the DFFNet model did not significantly improve the prediction accuracy of each category. The reason is that the RLNformer model correctly predicted most samples for the corresponding category, and the deep relationship hidden in the data is difficult to re-extract.

In summary, the DFFNet model improved the prediction performance of the four sub-datasets of UEA to varying degrees, which indicates that it has a good generalization ability in different data scenarios. Therefore, the DFFNet model has certain versatility and excellent prediction performance when dealing with meteorological datasets and other time series datasets.

6. Conclusions

In this paper, rainfall prediction in Northern Xinjiang is studied in depth, and the existing RLNformer model is improved to improve the accuracy of prediction. The model is tested and analyzed in detail. The main work of this paper is summarized as follows.

(1)

Given the problem that the RLNformer model can only extract a single time dependence and cause the loss of some important information, we proposes a DFFNet model based on dual-branch feature fusion to extract the relationship between time dependence and feature interactions in meteorological data. The extracted two parts of the information are fused in the feature dimension, which not only avoids the loss of important information to a certain extent but also reduces the training time consumption on the Northern Xinjiang rainfall dataset by 21%.

(2)

The public rainfall dataset of India, which is completely different from the climate in Northern Xinjiang, is selected for comparative experiments. Compared with the RLNformer model, the training time consumption of the DFFNet model on this dataset is reduced by 6.9%, and it has a faster convergence speed, which verifies that the DFFNet model has a strong generalization capability.

(3)

The UEA dataset for multivariate time series classification is selected for comparative experiments. In contrast to the RLNformer model, the DFFNet model has different degrees of improvement in the prediction performance of each sub-dataset of the dataset, which verifies that the DFFNet model is also suitable for time series prediction research in other fields.

Although the DFFNet model proposed by us has achieved relatively good prediction performance on the two rainfall datasets and four sub-datasets of UEA, there are still some shortcomings that need to be improved and perfected. Therefore, the planned future work is as follows.

(1)

The DFFNet model still incorrectly predicts individual categories as other categories in the two rainfall datasets, indicating that some important information in the time series of each meteorological element is still not extracted. Therefore, in a follow-up study, it is proposed to improve the feature extraction method, such as converting the time series from the time domain to the frequency domain, then extracting information to avoid excessive information loss to the greatest extent.

(2)

In this paper, only the ground meteorological station data are used for rainfall prediction. This form of meteorological data is too limited, which may be the reason why the model is not ideal in some categories. Therefore, it is worth exploring the method of multi-mode fusion rainfall prediction using multi-source meteorological data such as radar images, high-altitude detectors, and satellite cloud images in future research.