Wildfire Spread Prediction Using Attention Mechanisms in U2-NET

Abstract

Destructive wildfires pose a serious threat to ecosystems, economic development, and human life and property safety. If wildfires can be extinguished in a relatively short period of time after they occur, the losses caused by wildfires will be greatly reduced. Although deep learning methods have been shown to have powerful feature extraction capabilities, many current models still have poor generalization performance when faced with complex tasks. To this end, in this study, we considered introducing attention modules both inside and outside the nested U-shaped structure and trained a neural network model based on the U2-Net architecture to enable the model to suppress the activation of irrelevant areas. Compared with baseline models such as U-Net, our model has made great progress on the test set, with an F1 score improvement of at least 2.8%. The experimental results indicate that the model we proposed has certain practicality and can provide a significant scientific basis for forest fire management and emergency decision-making.

1. Introduction

Due to the rising frequency of wildfires caused by climate change and human activities, natural ecosystems such as forest ecosystems and wetland ecosystems are facing serious wildfire threats. Wildfires are a natural disaster that cause large amounts of soot emissions, thereby affecting the global carbon cycle and slowing the achievement of carbon neutrality [1,2,3]. For example, over the past decade, wildfires in the United States have burned more than 68 million acres of land, causing enormous economic losses. The fires release large amounts of greenhouse gases and air pollutants, and the accumulation of greenhouse gases leads to global warming, which in turn triggers weather extremes such as unusual rainfall, drying of vegetation, strong winds, high temperatures, and high-frequency lightning strikes. Therefore, there is an urgent need for effective predictive models to curb this destructive phenomenon.

Traditional wildfire spread models consist of two main categories: the first is based on physical and chemical processes, or physical models of physical processes only, such as the Wildland–Urban Fire Dynamics Simulator (WFDS) [4]. It is based on Computational Fluid Dynamics (CFD) principles to simulate phenomena such as air flow, heat transfer, and wildfire combustion in wildfires. The characteristic of this type of model is that it physically models the frontier part of the wildfire based on the principle of the conservation of energy and analyzes the impact of the wildfire on unburned combustibles through radiation and heat transfer. The second category is empirical or semi-empirical models based on statistical analysis of experimental data, such as Rothermell [5] in the U.S. and Canadian Forest Fire Behavior Prediction (CFBP) [6] in Canada, etc. The characteristic of this type of model is that it does not take into account the physical and chemical processes in the spread of wildfires. It is basically based on a large amount of historical fire data or experimental data to establish the model, or to establish an expression for the wildfire spread rate using some form of physical framework [7]. From the earliest Huygens–Fresnel principle models (such as FARSITE [8] and graph theory models) to Cellular Automata (CA) [9,10], the prediction principles of most of these prediction models require some prior knowledge and empirical formulas, which often have certain limitations on environmental conditions, making it difficult to guarantee their accuracy in the prediction of large-scale fires.

Conventional wildfire spread models struggle to ensure their effectiveness in large-scale fires, and with the emergence of large amounts of data and higher computing power, deep learning (DL)-based methods have been used for wildfire detection, segmentation, and classification tasks and have shown good generalization capabilities. The most widely used and successful image classification model for wildfire classification tasks is the convolutional neural network (CNN). Typically, a CNN accepts an input image and predicts the presence or absence of wildfire as the output of the model [11,12]. While the wildfire classification model predicts the presence of wildfires in the input image, we are unable to determine the location of the wildfire, whereas the wildfire detection algorithm determines the class of the detected object in the image as well as the boundaries of the detected object [13,14]. The wildfire segmentation task aims to classify the parts of the input image that belong to the “FIRE” class, and the output is a binary mask that can highlight the shape and location of the wildfire through visualization techniques [11].

Widely used convolutional neural network models for wildfire segmentation tasks mainly include Fully Convolutional Network (FCN) and encoder–decoder models. The FCN extracts the features of the input image through convolution and pooling operations while the output is a binary mask. FCN models for semantic segmentation significantly improve accuracy through pre-trained classifier weights, fusion of different layer representations, and end-to-end learning over the entire image [15]. Raw FCNs may result in the loss of detailed information when capturing contextual information across long distances [16]. The encoder–decoder model comprises an encoder and a decoder, where the encoder gradually reduces the dimensionality of the data and extracts useful features, while the decoder is designed to upsample the compressed features and transform them into a meaningful binary firemask [17,18].

Despite the great feature extraction capabilities of deep learning methods, many current models still struggle to effectively learn information-rich contextual features, resulting in poor generalization performance when facing complex tasks. Therefore, in this paper, we present a U2-Net model incorporating an attention mechanism that introduces an attention mechanism both inside and outside the nested U-shaped structure, aiming to solve the problems of U2-Net such as retaining invalid features and losing edge information in the connection.

The rest of this paper is arranged as follows: Section 2 presents previous related work. In Section 3, we introduce our deep learning model. Section 4 introduces the data source, data processing, and model results. The last section presents our conclusions.

2. Related Work

U-Net is a typical encoder–decoder architecture that performs well in semantic segmentation tasks [19,20]. Encoder and decoder features are combined by employing jump connections to better catch finer details and generate more accurate results. Akhloufi [3] used the U-Net model for input features to segment the wildfire region, an encoder to extract wildfire features from the fire input image, and a decoder to decode the compressed feature map and output it as a binary mask. Many researchers have improved some of the modules of U-Net. Zhang [21] and Huot [22] proposed a model that combines the advantages of deep residual learning and the U-Net architecture by using residual units instead of ordinary neural units, simplifying the training process of the model. Li et al. [23] proposed an improved U-Net segmentation structure in which the encoder is a densely connected CNN structure such that each feature extractor layer accepts as input all the features obtained from all the previous layers, which achieves feature reuse so that as many features as possible can be extracted without adding more parameters. Khryashchev et al. [24] used U-Net with ResNet-34 as the backbone for detecting and segmenting wildfire areas, increased the amount of training data by using multiple data enhancement techniques, and optimized the training process with an adaptive moment estimation algorithm. Wang et al. [25] proposed smoke-U-Net, a network model for smoke segmentation based on an improved U-Net. The model combines the attention mechanism and the residual module by adding the ResBlock, a residual block with a jump connection structure, and the SEBlock module based on the attention mechanism in the network structure, respectively.

In addition to improving the modules of U-Net, there are many research projects aiming to improve the U-Net architecture. Zhou et al. [26] proposed an improved U-Net architecture, U-Net++, which allows the optimizer to solve the optimization problem in a simpler way by reducing the semantic gap between the encoder and the decoder. Bochkov and Kataeva [27] proposed the wideUU-Net concatenative (wUU-Net) connection model based on the U-Net architecture. The wUU-Net consists of a modernization of U-Net, and UU-Net consists of two U-Net architectures with an additional skip between the decoder of the first U-Net and the encoder of the second U-Net connections, with improved accuracy relative to the U-Net model. Qin et al. [28] proposed a simple yet powerful two-level nested U-structure, U2-Net, consisting of residual U-blocks (RSUs) and external U-structures that come to connect the residual U-blocks, increasing the depth of the overall network architecture while yielding low computational cost [29].

Vaswani et al. [30] demonstrated the effectiveness of the attention mechanism in improving model performance, a landmark for a large number of subsequent studies and applications. Zhang et al. [31] proposed a U-shaped method ATT Squeeze U-Net (attention U-Net and SqueezeNet) with attention modules. The method removes the conventional encoder and uses eight fire modules with SqueezeNet as the backbone to extract wildfire features, while the decoder employs 3 × 3 and 1 × 1 convolutional layers, a ReLU activation function, and three corresponding DeFire modules, a framework that significantly reduces the parameter size of the model. Shirvani et al. [32] added attention gate units (AU-Net) and residual blocks and attention gate units (RAU-Net) to the U-Net architecture, which improves feature extraction by integrating residual blocks and incorporates an attention mechanism at jump junctions to remove irrelevant and invalid information.

3. Methodology

We categorize this wildfire spread prediction task as a semantic segmentation problem in image segmentation, where each region is predicted to contain a fire or not based on the previous day’s fire location as well as various influencing factors. We first introduce the basic principles of the U-Net model and then introduce the U2-Net model based on U-Net. At the same time, considering the characteristics of multi-channel input features of remote sensing images, we propose a U2-Net model that incorporates an attention mechanism to handle this task.

3.1. Attention U2-Net

The entire U-Net framework consists of contraction paths and expansion paths. The former follows the classical architecture of convolutional networks and consists mainly repeated applications of two 3 × 3 convolutions (unpadded convolutions), each of which is followed by a rectified linear unit (ReLU) and a 2 × 2 max-pooling operation (with stride 2) for downsampling. Furthermore, at each downsampling step, U-net’s contraction path doubles the number of feature channels. The expansion path will first upsample the feature map and then perform a 2 × 2 convolution (“up-convolution”) to halve the number of feature channels. This is followed by concatenation with the correspondingly cropped feature map in the contraction path as well as two 3 × 3 convolutions, with a ReLU and cropping after each convolution. At the end of the decoder, U-Net uses a 1 × 1 convolutional layer to transform the feature map into the final segmentation map [19,33].

The design of U-Net is the basis of the design of U2-Net. U2-Net comprises a two-layer U-structure. It inherits the encoding and decoding ideas of the U-Net network model but no longer utilizes a simple convolutional layer or deconvolutional layer for every sample. Instead, it embeds a whole U-shaped residual block structure (RSU). Specifically, the outer structure of U2-Net consists of a saliency map fusion module, a five-level decoder, and a six-level encoder. Each level between the encoder and the decoder contains the newly proposed RSU.

The U2-Net architecture represents each sample from the Wildfire Dataset as an input to the contraction operation and generates a probability list as output based on the number of classes. When the input data are processed by each RSU, each downsampling/upsampling path in the corresponding RSU consists of a 3 × 3 convolution layer, a batch normalization layer, a ReLu activation function, and a 2 × 2 maximum pooling/upsampling layer. The six sets of saliency probability maps obtained by the six different RSU modules De_5, De_4, De_3, De_2, De_1, and En_6 are fused to obtain the final fused feature map. Due to the different depths of the RSU modules, the final fused feature map contains rich content. Finally, the model prediction results and loss function values are obtained, and the value of the loss function is reduced through continuous iteration to improve the generalization ability of the model [34].

The U2-Net network structure is sophisticated and deep, and deeper layers can be used to acquire high-resolution feature maps without considering the computing cost. However, the model is prone to retaining invalid features and losing edge information in the connection [35]. The attention mechanism was first proposed by Bahdanau et al. [36] to address the problem of precisely sequentially encoding long sentences in text to a constant length [37]. Adding an attention mechanism to the model can actively suppress the activation of irrelevant areas at jump connections, thereby reducing redundant information [38,39]. Therefore, we propose a U2-Net model that incorporates an attention mechanism. The attention mechanism is introduced both inside and outside the nested U-shaped structure. The model structure is shown in Figure 1. Figure 2 shows how the attention mechanism can be integrated into the U2-Net model.

3.2. Loss Function

In the training process, we use deep supervision similar to Holistically nested Edge Detection (HED), whose effectiveness was verified in HED and Deeply Supervised Salient object detection (DSS). The computation of the loss function includes the loss of the side output feature maps and the loss of the final fused output feature maps, and the loss function is defined as follows [28,35]:

$$\mathcal{L}oss=\sum _{s=1}^S{\omega }_{side}^{\left(s\right)}{\mathcal{l}oss}_{side}^{\left(s\right)}+{\omega }_{fuse}{\mathcal{l}oss}_{fuse}$$

(1)

where S = 6 denotes the number of side output feature maps, ${\mathcal{l}oss}_{side}^{\left(s\right)}$ indicates the loss of the side output feature map, ${\mathcal{l}oss}_{fuse}$ denotes the loss of the final output feature map, ${\omega }_{side}^{\left(s\right)}$ and ${\omega }_{fuse}$ denote the weight of each loss term, and ${\mathcal{l}oss}_{side}^{\left(s\right)}$ and ${\mathcal{l}oss}_{fuse}$ are binary cross-entropy loss functions. The formula for BCELoss is as follows:

$$\mathcal{l}oss=-\sum _{(h,w)}^{(H,W)}\left[P_{GT(h,w)}\mathit{log}{P}_{pred(h,w)}+(1-P_{GT(h,w)})\mathit{log}(1-P_{pred(h,w)})\right]$$

(2)

where $P_{GT(h,w)}, P_{pred(h,w)}$ indicates the pixel values where the predicted map’s ground truths are located, respectively. This loss term is computed with all the pixel points in that image. We tried to minimize the loss between the predicted value and ground truth during the training process and chose the final fused output feature map as our predicted value.

4. Experiments

4.1. Datasets and Experimental Setup

Data were collected from the work of the Google Research team [22]. The researchers presented the Wildfire Masks for the Next Day Wildfire Dataset. As shown in Figure 3, the dataset collects satellite data on wildfires in six areas: Elevation Data, Weather Data, Drought Data, Vegetation Data, Population Density Data, and Wildfire Masks. Specifically, the input data include elevation, wind direction (th), wind speed (vs), minimum temperature (tmmn), maximum temperature (tmmx), humidity (sph), precipitation (pr), drought index (pdsi), vegetation (NDVI), population density (population), energy release component (ERC), and firemask at time t (pfm), while the firemask at time t + 1 (fm) is used as the output. The resolution of a 64 × 64 size image is 1 km, and for each 1 km × 1 km area, the firemask consists of both ‘fire’ and ‘no fire’. We used 0 to fill the missing pixel points in each image. The dataset contains a total of 18,545 samples, and we divided the data into training, validation, and test sets in a ratio of 8:1:1.

For the task of forecasting the spread of forest fires, we consider the influences and the firemask at moment t as input features (denote ‘previous firemask’) and the firemask at moment t + 1 as labels (denote ‘firemask’). In addition to firemask, we performed clipping and normalization operations on the remaining input features, the purpose of which is to remove the effects of extreme values and magnitudes, respectively. Through Figure 3, it can be found that factors such as elevation and wind direction show great variations in the area of 64 km × 64 km, and the last two columns represent the firemask at the moment of t and the firemask at the moment of t + 1, respectively, where gray denotes “NO FIRE” and red denotes “FIRE”.

The network architecture was implemented in Python and executed on an NVIDIA GeForce RTX 3070 graphics processing unit (GPU) with 16 GB of memory. The initial learning rate was set to 0.001 and was adaptively decreased during training using the Adam optimizer. Besides that, we conducted comparative experiments with U-Net [20], Attention U-Net [20], ResUNet [22], U2-Net [28], and Attention U2-Net (Our Method). For the first four models, we ran their trained models on their proposed environments, while for our method, the hyperparameters are optimally sought using a grid-tuned parameterization method.

4.2. Model Evaluation Metrics and Baselines

There are several indicators used to evaluate the generalization ability of the model, including Precision, Recall, and Overall Accuracy (OA). Precision denotes the ratio of samples predicted to be positive that are indeed positive. Recall denotes the ratio of the actual number of positive samples in the predicted positive samples to the total number of positive samples. F is the binary feature map of the final fusion output and T is the true value. It is used for combined assessment of Precision and Recall. OA denotes the proportion of correctly predicted pixels among all pixels, denotes correctly classified pixels, and denotes the total number of pixels. The neural networks in this study were trained using Python’s Pytorch 2.2.2 and Tensorflow 2.16.1 libraries.

$$Precision={\displaystyle \frac{\left|F\cap T\right|}{\left|F\right|}}$$

(3)

$$Recall={\displaystyle \frac{\left|F\cap T\right|}{\left|T\right|}}$$

(4)

$$F_{\beta }={\displaystyle \frac{(1+{\beta }^2)\times Precision\times Recall}{{\beta }^2\times Precision+Recall}}$$

(5)

$$OA={\displaystyle \frac{P_{True}}{P_{total}}}$$

(6)

4.3. Results and Discussion

In this section, the generalization capabilities of the U-Net [20], Attention U-Net [20], ResUNet [22], U2-Net [28], and Attention U2-Net models were analyzed. The effects of the baseline model on the test set and the Attention U2-Net model on the test set are shown in Figure 4 and Figure 5, respectively. From Figure 4b, c, we can see that the prediction results of the Attention U-Net model and the ResUNet model tend to be smoother. From Figure 5, we can see that the Attention U2-Net model tends to connect small fires in the wildfire spread prediction, which is more in line with the actual wildfire spread trend, but its recognition of small fires is low.

In

Table 1. Comparison of modeling effects.

	OA	Precision	Recall	F1
U-Net [20]	0.977	0.275	0.563	0.37
Attention U-Net [20]	0.978	0.294	0.574	0.389
ResUNet [22]	0.985	0.4	0.389	0.394
U2-Net [28]	0.986	0.377	0.358	0.367
Attention U2-Net	0.982	0.521	0.331	0.405

, we can see that the accuracy of all models has reached more than 97%, and because our training data are labeled as unbalanced, accuracy is easily exaggerated, so we prefer to use F1 score to evaluate the advantages and disadvantages of the model. For the U-Net model, the introduction of the attention module improves the F1 score of the model by 5.1%. Interestingly, the F1 score of the U2-Net model is the lowest among all models, indicating that simply increasing the depth of the model architecture cannot improve the generalization performance and may cause overfitting. Compared with the U2-Net model, the F1 score of the model is improved by 10.3% by introducing the attention module both inside and outside the nested U-shaped structure, indicating that it is correct to introduce the attention module into the U2-Net structure. Compared with the F1 score of the ResUNet model, the F1 score of our model is improved by 2.8% because the Precision index is improved by 30.3%. For the Attention U2-Net model, the difference between Precision and Recall is 0.19, indicating that our model pays more attention to the prediction accuracy of the fire area.

4.4. Ablation Study

In order to analyze the impact of various input features on the prediction results, we perform ablation experiments using the model with the best F1 score, the Attention U2-Net model. We consider deleting each input feature and retraining the model using the remaining features. The metrics of the model are shown in

Table 2. Ablation study.

Removed Feature	Accuracy	Precision	Recall	F1
Elevation	0.98	0.519	0.343	0.413
ERC	0.982	0.474	0.364	0.412
PDSI	0.98	0.526	0.33	0.406
tmmn	0.981	0.492	0.343	0.404
sph	0.982	0.474	0.351	0.403
th	0.983	0.48	0.345	0.401
Population	0.981	0.504	0.332	0.400
pr	0.978	0.508	0.330	0.400
vs	0.979	0.523	0.323	0.399
Tmmx	0.981	0.492	0.336	0.399
NDVI	0.98	0.48	0.338	0.397
Prefiremask	0.982	0.424	0.325	0.368

. It can be found that the F1 score of the model decreases the most when the Prefiremask feature is deleted. Firemask has a large correlation with Prefiremask, indicating its importance as a predictor variable of the model. There is a small increase in the model’s F1 score when the Elevation and ERC features are removed, which we attribute to the stochastic nature of the model.

5. Conclusions

Based on the U-Net architecture, we propose an Attention U2-Net network for predicting the spread of wildfires, which introduces an attention module both inside and outside the nested U-shaped structure, allowing the model to suppress activation in irrelevant regions and thus reduce redundant information. The optimal hyperparameter combination determined through the experiments contributed significantly to the F1 score of the model in predicting wildfire spread. The experiments show that the Attention U2-Net network improves the F1 score in the test set of wildfire spread by at least 2.8% compared to benchmark models such as U-Net, and the experimental results indicate that the model can provide an important reference for fire managers to formulate effective suppression strategies. In addition, through ablation experiments, we found that the key influencing factor of the model was Prefiremask, indicating its importance as a predictor variable of the model.