1. Introduction
A growing issue as a result of the intensifying effects of global climate change is forest fires which are acknowledged as a major disruption in the world’s forest ecosystems. Major risks to biodiversity, ecological balance, and human settlements are posed by their growing frequency and severity. Rising temperatures, extended droughts, and shifting weather patterns provide favorable conditions for the ignition and rapid spread of fires.
Many studies have reported about the effects and issues of forest fires in many countries of the world. According to Evelpidou, N. [
1], Greece’s summer 2021 wildfires ranked among the worst forest fire incidents the nation had seen in the previous ten years. The forest fire lasted for a 20-day period (from 27 July 2021 to 16 August 2021) and left almost 3600 km
2 completely destroyed. Meier, S. et al. [
2] conducted an extreme value study using geospatial data from the European Forest Fire Information System (EFFIS) spanning from 2006 to 2019. After a 10-year analysis, they concluded that Portugal has the highest risk of forest fire with 50,338 ha, followed by Greece 33,242 ha, Spain 25,165 ha, and Italy 896,630 ha. Pang, Y. [
3] stated that 111,446 forest fires occurred, consuming a sizable 3,289,500 hectares of land, between 2003 and 2018 in China. It can be seen from previous studies that forest fires have consumed many lands and are on the increase. There has been a need to adopt methods to detect forest fires early. A forest fire begins small before it increases in size and becomes uncontrollable. Mehta, K. [
4] claims that based on data gathered over the years, 75–80% of the many disasters brought on by forest fires may have been avoided had the event been recognized and addressed sooner.
Reducing the uncontrolled spread of forest fires and protecting wider regions from their disastrous effects requires early detection and a timely response. Furthermore, it is important to recognize the additional difficulty of optimizing energy efficiency. Drone operations are made more complex by the requirement to control energy usage which calls for creative ways to balance the usage of sustainable resources with early detection accuracy. The main objective is to create all-encompassing solutions that maximize energy efficiency and improve fire detection accuracy hence reducing the ecological damage caused by these fires globally. The pressing need to solve the forest fire issue is the requirement for early detection that is both proactive and technologically advanced.
There are some tools that help in the early detection of forest fires and which help in making the right decision. Using a wireless sensor network, Dampage, U. [
5] proposed a system and approach that can be utilized to identify forest fires in the early stages. Kang, Y. [
6] notes that while sensors of active fires are effective in detecting forest fires, early detection response has not received as much attention due to generalization issues with basic threshold approaches based on contextual statistical analysis. Satellite images later emerged that provide a visual picture of the fire site. The image can be analyzed to detect the existence of a forest fire. Kalaivani, V. [
7] used a dataset from the Landsat satellite for forest fire early detection. Then drones appeared, which provided more facilities for the early detection of forest fires. According to Yandouzi, M. [
8], drones are distinguished by their ability to fly at low altitudes, collect more and accurate data, and have a low cost of use. Yandouzi, M. also adds that satellites are unable to detect small fires and can send photos every few days or weeks, whereas drones can send images every day. Drones are therefore considered to have the potential to assist in the early detection of forest fires. Many studies have proved the success of using drones for the early detection of forest fires. Mashraqi, A. M. [
9] proposed a model for the early detection of forest fires using RGB images taken from drones, while Wang, Y. [
10] proposed a model using IR images. Chen, X. [
11] tested various models in which the inputs for the forest fire detection model are RGB and IR images.
In order to create a system for the early detection of forest fires, a solution must address the problem of the unstable battery power consumption of drones. Energy-efficient routing protocols have the ability to manage a group of drones so that they use energy as efficiently as possible by organizing the work between them. Many studies have used energy-efficient routing protocols for drones. The Low-Energy Adaptive Clustering Hierarchy (LEACH) protocol is designed to save energy by applying the cluster head method, where the rest of the drones transmit information for the cluster head only, and the cluster head changes during a specific time frame [
12,
13,
14]. Optimized link state routing (OLSR) is an energy-saving protocol that relies on finding the shortest route to reduce energy consumption [
15,
16]. The ECP-LEACH protocol is a recently proposed energy-efficient protocol based on two components (a threshold monitoring module and sleep scheduling module) [
17].
The strategic integration of advanced detection models and energy-efficient protocols is the focal point of early forest fire detection. The effectiveness of detection algorithms in early fire detection is dependent on their ability to provide accurate results. These algorithms are essential for the reliable operation of early fire detection since they guarantee the accurate detection of forest fires. At the same time, energy-efficient measures must be put in place in order to achieve early detection and add a long-term and strategic aspect to monitoring activities. Accurate detection algorithms and energy-efficient protocols cooperate in order to maximize monitoring capabilities and enhance fire detection skills. This paper essentially goes beyond individual research studies by introducing a system that synchronizes the accuracy of forest fire detection with a deliberate dedication to energy efficiency.
In
Section 2, the protocol used for drones in the system is presented. In
Section 3, the Intermediate Fusion VGG16 model is proposed for forest fire detection. In
Section 4, the Forest Defender Fusion System is proposed. In
Section 5, a simulation is described. The results are displayed and discussed in
Section 6. In
Section 7, the conclusion of the paper is presented.
2. Routing Protocol for Drones
In our previous work [
17] we demonstrated that the ECP-LEACH protocol has significant energy savings by comparing it with the LEACH protocol. The comparison was made at energy consumption rates in three cases (50, 100, and 150 nodes) with different duty cycles, where each node starts with an initial energy equal to 18,720 joules. The ECP-LEACH protocol showed significant energy savings.
Table 1 shows the average energy consumed when using the default LEACH protocol.
Table 2 shows the average energy consumed when using the Enhanced LEACH protocol (ECP-LEACH).
To detect forest fires early, drones must be in operation for most of the day sending images to detect the fire. This process consumes a lot of battery power for the drone. In this paper, the use of the ECP-LEACH protocol is adopted to significantly save battery power. The ECP-LEACH protocol includes two basic parts: the threshold monitoring unit and the sleep scheduling unit. The threshold monitoring module plays a crucial role by constantly tracking the individual power consumption rates of each drone. The drone’s sleep mode is intelligently initiated by the ECP-LEACH protocol when it exceeds a specified threshold. The accurate use of battery power is ensured through strategic decision making that prevents waste.
The energy resources which are available for drones are limited. Energy conservation and extending the network’s lifetime are the goals. Drones can be placed into a sleep mode to conserve energy when they are not needed for data transmission, moving, or taking photos. Nevertheless, it might be difficult to judge when to put drones into sleep mode. To overcome this difficulty, the ECP-LEACH protocol includes a threshold unit. The threshold unit compares the drones’ energy consumption to a preset threshold and puts the drones into sleep mode if the threshold is exceeded.
Drones are signaled to go to sleep if their energy consumption exceeds the predefined threshold which is monitored by the threshold monitoring unit during each round. Preventing energy waste is aided by this strategic decision-making process based on individual power consumption rates. The sleep scheduling unit uses an algorithm that takes into account variables like energy levels and communication history to schedule the drones’ sleep modes based on signals from the threshold monitoring unit. The protocol successfully preserves energy which increases the network lifetime and lowers overall energy consumption by employing a threshold to put drones into sleep mode.
This approach ensures energy conservation without compromising network connectivity which is especially useful in applications like environmental monitoring and forest fire detection. It is crucial that the threshold and sleep scheduling unit is only intended for drones that are not cluster heads because cluster heads are responsible for network traffic and data distribution. The implementation of a threshold and sleep scheduling unit offers major advantages as it prevents potential connectivity issues and ensures the network operates smoothly and effectively.
To create an early detection system for forest fires, the energy consumed by the drone must be taken into account in order to obtain the longest possible time for the drone to operate. Therefore, the ECP-LEACH protocol is good for the early detection of forest fires.
3. Intermediate Fusion VGG16 Model for Detecting Forest Fires
The lack of well-annotated and high-quality forest fire datasets has hampered the development of drone-based fire monitoring systems partly because of drone flight restrictions for planned wildfires. In November 2021, the FLAME 2 dataset was created by Fule, P. F., et al. [
18] using drones to record streams of visible and IR spectrum video pairs together during a planned open canopy fire in northern Arizona. The main dataset and the supplemental dataset are the two main components of the FLAME 2 dataset. The main dataset consists of seven unlabeled raw RGB and IR video pairs, a set of 254p × 254p RGB/IR frame pairs, and a collection of original resolution RGB/IR frame pairs. The seven raw video pairs are the source of both sets of frame pairings. Both sets of frame pairs were labeled frame by frame. The “Fire/NoFire” label for each pair indicates if there is a fire or no fire in the RGB or IR image. The “Smoke/No Smoke” label for the RGB or IR image frame indicates if the smoke occupies at least 50% of the RGB image. The supplementary dataset consists of weather data, raw pre-burn videos, a geo-referenced pre-burn point cloud, a burn plan, an RGB pre-burn orthomosaic, and other data. The supplementary dataset gives context for the aerial photography in the main dataset. A set of 254p × 254p RGB/IR frame pairs was considered for training the forest fire detection model. The set contained 25,434 images for Fire/Smoke label as shown in
Figure 1, 14,317 images for Fire/No Smoke label as shown in
Figure 2, and 13,700 images for No Fire/No Smoke label as shown in
Figure 3.
The first stage of preprocessing involves transforming the images in order to improve their appropriateness. This transformation entails converting the photos to the BGR (blue, green, red) from the commonly used RGB (red, green, blue). Then, every color channel in the BGR representation undergoes a zero-centering process. A technique known as zero-centering is achieved by modifying each pixel’s intensity values so that the mean value drops to zero while maintaining the original pixel value scale. One of the methods in the case of two or more inputs to the deep learning model architecture is the fusion method. Stahlschmidt, S. R. [
19] explained that fusion methods were divided according to the condition of the fusion layers input into three categories: early, intermediate, and late. Early fusion involves concatenating the original input data and treating the resultant vector as unimodal input which means that the network architecture does not distinguish between the origins of the modality features. Intermediate fusion learns from marginal representations in the form of feature vectors rather than fusing the original multimodal input. Intermediate fusion can be divided into two parts: homogeneous and heterogeneous. In the homogeneous part, the neural networks of the same sort (convolutional neural network, fully connected, etc.) can be used to learn marginal representations while in the heterogeneous part, the peripheral representations are acquired through various network types. In late fusion, judgments from distinct unimodal sub models are integrated into a final conclusion rather than merging the original data or learned features.
One of the most popular deep learning models specialized in image classification is VGG16. VGG16 is divided into two parts as shown in the
Figure 4. The first section is a convolutional section and the second section is a dense section. The convolutional section consists of thirteen convolutional layers and five max-pooling layers. The dense section consists of three fully connected dense layers. The simple architecture of the VGG16 gives it the ability to produce outputs quickly, which is important for forest fire detection [
20].
The proposed model for fire detection belongs to the intermediate fusion method of homogeneous parts and is called Intermediate Fusion VGG16. The early detection model comprises two VGG16 models, with the first VGG16 receiving an IR image as input and the second VGG16 receiving an RGB image as input. Two feature vectors are created after the inputs are fed to the two VGG16 models. Those vectors are merged to have a single vector and then fed into a fully connected network with 64 neurons and RELU activation function. The output is fed into a fully connected network with 32 neurons and a RELU activation function. In the final layer, the output is fed in a fully connected layer with three neurons and a SOFTMAX activation function to obtain the predictions.
Figure 5 depicts the model architecture of detecting forest fire that is used in the Forest Defender Fusion system.