Forest Fire Identification in UAV Imagery Using X-MobileNet
Abstract
:1. Introduction
- We propose a novel deep learning method based on the existing MobileNetV2 that is suitable for fast execution and low-resource devices such as drones.
- The pre-trained weights for all layers of the MobileNetv2 architecture are used in the proposed method, which reduces the number of trainable parameters needed. Our model is now well suited for use on drones with limited resources as a result.
- Unlike standard machine learning techniques, the proposed model may be trained and deployed end to end, eliminating the need for separate feature extraction and classification phases.
- To ensure that our model is robust, it was validated using two different fire datasets. The results of our experiments demonstrate that our method consistently outperformed other recent DL-based algorithms on fire datasets.
- A lightweight hexagon drone was designed at low cost and with flight control.
2. Related Works
3. Materials and Methods
3.1. UAV and Its System Architecture
3.2. Dataset
3.3. Data Pre-Processing
Data Augmentation
- Rotation range: It is a value in the range of (0–180) degrees, a range within which the picture is rotated randomly.
- Width shift and height shift: To shift an image means moving all pixels of the image in one direction, such as horizontally or vertically, while keeping the same image dimensions.
- Zoom range: It is used for zooming the pictures.
- Horizontal flip: It flips half of the image in the horizontal direction.
- Fill mode: It fills the newly produced pixels that appear after rotation and width or height shift.
4. Proposed Enhanced MobileNet Model
4.1. Pre-Trained Model Usage
4.2. Customized Output Layer
4.3. Global Average Pooling
4.4. Depth-Wise Separable Convolution
- Depth-wise convolution: This technique performs layer filtering by applying a single filter to each input channel. In Figure 7, it can be seen that the filter is depth-wise separated with a single filter and applied to each layer. With this, 9 parameters in the filter can be represented with 6 parameters, as shown in Figure 7.
- Point-wise convolution: This technique uses 1X1 convolution to create linear collective output.
4.5. Fire Classification
5. Experimental Results and Discussion
5.1. Training the Model
5.2. Evaluation Measures
5.3. Forest Fire Detection Results
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Sherry, J.; Neale, T.; McGee, T.K.; Sharpe, M. Rethinking the maps: A case study of knowledge incorporation in Canadian wildfire risk management and planning. J. Environ. Manag. 2019, 234, 494–502. [Google Scholar] [CrossRef]
- Bytnerowicz, A.; Arbaugh, M.J.; Andersen, C.; Riebau, A.R. Chapter 26 Integrating Research on Wildland Fires and Air Quality: Needs and Recommendations. Dev. Environ. Sci. 2008, 8, 585–602. [Google Scholar] [CrossRef]
- Mahdipour, E.; Dadkhah, C. Automatic fire detection based on soft computing techniques: Review from 2000 to 2010. Artif. Intell. Rev. 2014, 42, 895–934. [Google Scholar] [CrossRef]
- Pineda, N.; Rigo, T. The rainfall factor in lightning-ignited wildfires in Catalonia. Agric. For. Meteorol. 2017, 239, 249–263. [Google Scholar] [CrossRef]
- Linares, C.; Carmona, R.; Salvador, P.; Díaz, J. Impact on mortality of biomass combustion from wildfires in Spain: A regional analysis. Sci. Total Environ. 2018, 622–623, 547–555. [Google Scholar] [CrossRef]
- Yan, F.; Xu, X.; Han, N. A kind of identification method of geometric parameters for forest fire. In Proceedings of the ICSPS 2010—2nd International Conference on Signal Processing Systems, Dalian, China, 5–7 July 2010; Volume 1, pp. 91–94. [Google Scholar] [CrossRef]
- Tang, L.; Shao, G. Drone remote sensing for forestry research and practices. J. For. Res. 2015, 26, 791–797. [Google Scholar] [CrossRef]
- Casagli, N.; Frodella, W.; Morelli, S.; Tofani, V.; Ciampalini, A.; Intrieri, E.; Raspini, F.; Rossi, G.; Tanteri, L.; Lu, P. Spaceborne, UAV and ground-based remote sensing techniques for landslide mapping, monitoring and early warning. Geoenvironmental Disasters 2017, 4, 9. [Google Scholar] [CrossRef]
- Ayhan, B.; Kwan, C. On the use of radiance domain for burn scar detection under varying atmospheric illumination conditions and viewing geometry. Signal Image Video Process. 2017, 11, 605–612. [Google Scholar] [CrossRef]
- Dos Santos, J.a.F.C.; Gleriani, J.M.; Velloso, S.G.S.; de Souza, G.S.A.; do Amaral, C.H.; Torres, F.T.P.; Medeiros, N.D.G.; dos Reis, M. Wildfires as a major challenge for natural regeneration in Atlantic Forest. Sci. Total Environ. 2019, 650, 809–821. [Google Scholar] [CrossRef]
- Lollino, G.; Manconi, A.; Guzzetti, F.; Culshaw, M.; Bobrowsky, P.; Luino, F. Engineering Geology for Society and Territory-Volume 5: Urban Geology, Sustainable Planning and Landscape Exploitation; Springer: Cham, Switzerland, 2015; Volume 5, pp. 1–1400. [Google Scholar] [CrossRef]
- Sherstjuk, V.; Zharikova, M.; Sokol, I. Forest Fire Monitoring System Based on UAV Team, Remote Sensing, and Image Processing. In Proceedings of the 2018 IEEE 2nd International Conference on Data Stream Mining and Processing, DSMP 2018, Lviv, Ukraine, 21–25 August 2018; pp. 590–594. [Google Scholar] [CrossRef]
- Sherstjuk, V.; Zharikova, M.; Sokol, I. Forest Fire-Fighting Monitoring System Based on UAV Team and Remote Sensing. In Proceedings of the 2018 IEEE 38th International Conference on Electronics and Nanotechnology (ELNANO), Kyiv, UKraine, 24–26 April 2018; Volume 7, pp. 663–668. [Google Scholar] [CrossRef]
- Collins, L.; Griffioen, P.; Newell, G.; Mellor, a. The utility of Random Forests for wildfire severity mapping. Remote Sens. Environ. 2018, 216, 374–384. [Google Scholar] [CrossRef]
- Badia, A.; Pallares-Barbera, M.; Valldeperas, N.; Gisbert, M. Wildfires in the wildland-urban interface in Catalonia: Vulnerability analysis based on land use and land cover change. Sci. Total Environ. 2019, 673, 184–196. [Google Scholar] [CrossRef]
- Hamilton, D.; Myers, B.; Branham, J.B. Evaluation of Texture as an Input of Spatial Context for Machine Learning Mapping of Wildland Fire Effects. Signal Image Process. Int. J. 2017, 8, 1–11. [Google Scholar] [CrossRef]
- Kim, S.; Lee, W.; Park, Y.S.; Lee, H.W.; Lee, Y.T. Forest fire monitoring system based on aerial image. In Proceedings of the 2016 3rd International Conference on Information and Communication Technologies for Disaster Management—ICT-DM 2016, Vienna, Austria, 13–15 December 2016; pp. 5–10. [Google Scholar] [CrossRef]
- Yuan, C.; Liu, Z.; Zhang, Y. Fire detection using infrared images for UAV-based forest fire surveillance. In Proceedings of the 2017 International Conference on Unmanned Aircraft Systems—ICUAS 2017, Miami, FL, USA, 13–16 June 2017; pp. 567–572. [Google Scholar] [CrossRef]
- Ganesan, P.; Sathish, B.S.; Sajiv, G. A comparative approach of identification and segmentation of forest fire region in high resolution satellite images. In Proceedings of the IEEE WCTFTR 2016—2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare, Coimbatore, India, 29 February–1 March 2016; pp. 1–6. [Google Scholar] [CrossRef]
- Dao, M.; Kwan, C.; Ayhan, B.; Tran, T.D. Burn scar detection using cloudy MODIS images via low-rank and sparsity-based models. In Proceedings of the 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Washington, DC, USA, 7–9 December 2016; pp. 177–181. [Google Scholar]
- Sayad, Y.O.; Mousannif, H.; Al Moatassime, H. Predictive modeling of wildfires: A new dataset and machine learning approach. Fire Saf. J. 2019, 104, 130–146. [Google Scholar] [CrossRef]
- Schepaschenko, D.; Chave, J.; Phillips, O.L.; Lewis, S.L.; Davies, S.J.; Réjou-Méchain, M.; Sist, P.; Scipal, K. A global reference dataset for remote sensing of forest biomass. In The Forest Observation System approach; IIASA: Laxenburg, Austria, 2019. [Google Scholar]
- Byrne, K. Digital Morphometry Applied to Geo-Hazard Risk Assessment: A Case Study from Germany. Technische Universitat Dresden. Faculty of Environmental Sciences. Inst. Cartogr. Master Sci. 2018, 1, 79–97. [Google Scholar]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 4510–4520. [Google Scholar]
- Kamilaris, A.; Prenafeta-Boldú, F.X. Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning. arXiv 2018, arXiv:1807.11805. [Google Scholar]
- Shahbazi, M.; Théau, J.; Ménard, P. Recent applications of unmanned aerial imagery in natural resource management. GISci. Remote Sens. 2014, 51, 339–365. [Google Scholar] [CrossRef]
- Zhang, Q.; Xu, J.; Xu, L.; Guo, H. Deep Convolutional Neural Networks for Forest Fire Detection; Atlantis Press: Guangzhou, China, 2016. [Google Scholar] [CrossRef]
- Wardihani, E.D.; Ramdhani, M.; Suharjono, A.; Setyawan, T.A.; Hidayat, S.S.; Helmy, S.W.; Widodo, S.; Triyono, E.; Saifullah, F. Real-time forest fire monitoring system using unmanned aerial vehicle. J. Eng. Sci. Technol. 2018, 13, 1587–1594. [Google Scholar]
- Jiao, Z.; Zhang, Y.; Xin, J.; Yi, Y.; Liu, D.; Liu, H. Forest Fire Detection with Color Features and Wavelet Analysis Based on Aerial Imagery. In Proceedings of the 2018 Chinese Automation Congress—CAC 2018, Xi’an, China, 30 November–2 December 2018; pp. 2206–2211. [Google Scholar] [CrossRef]
- Yuan, C.; Liu, Z.; Zhang, Y. Aerial Images-Based Forest Fire Detection for Firefighting Using Optical Remote Sensing Techniques and Unmanned Aerial Vehicles. J. Intell. Robot. Syst. Theory Appl. 2017, 88, 635–654. [Google Scholar] [CrossRef]
- Yuan, C.; Liu, Z.; Zhang, Y. Vision-based forest fire detection in aerial images for firefighting using UAVs. In Proceedings of the 2016 International Conference on Unmanned Aircraft Systems—ICUAS 2016, Arlington, VA, USA, 7–10 June 2016; pp. 1200–1205. [Google Scholar] [CrossRef]
- Yuan, C.; Ghamry, K.a.; Liu, Z.; Zhang, Y. Unmanned aerial vehicle based forest fire monitoring and detection using image processing technique. In Proceedings of the CGNCC 2016—2016 IEEE Chinese Guidance, Navigation and Control Conference, Nanjing, China, 12–14 August 2016; pp. 1870–1875. [Google Scholar] [CrossRef]
- Römer, H.; Kersten, J.; Kiefl, R.; Plattner, S.; Mager, A.; Voigt, S. Airborne near-real-time monitoring of assembly and parking areas in case of large-scale public events and natural disasters. Int. J. Geogr. Inf. Sci. 2014, 28, 682–699. [Google Scholar] [CrossRef]
- Lee, J.; Wang, J.; Crandall, D.; Selma, S.; Selmašabanovic, S.; Fox, G. Real-Time Object Detection for Unmanned Aerial Vehicles based on Cloud-based Convolutional Neural Networks. In Proceedings of the First IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan, 10–12 April 2017. [Google Scholar] [CrossRef]
- Pastor, E.; Barrado, C.; Royo, P.; Santamaria, E.; Lopez, J.; Salami, E. Architecture for a helicopter-based unmanned aerial systems wildfire surveillance system. Geocarto Int. 2011, 26, 113–131. [Google Scholar] [CrossRef]
- Colomina, I.; Molina, P. Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote Sens. 2014, 92, 79–97. [Google Scholar] [CrossRef] [Green Version]
- Sun, H.; Song, G.; Wei, Z.; Zhang, Y.; Liu, S. Bilateral teleoperation of an unmanned aerial vehicle for forest fire detection. In Proceedings of the 2017 IEEE International Conference on Information and Automation—ICIA 2017, Macao, China, 18–20 July 2017; pp. 586–591. [Google Scholar] [CrossRef]
- Merino, L.; Caballero, F.; Martínez-de dios, J.R.; Maza, I.; Ollero, A. Automatic Forest Fire Monitoring and Measurement using Unmanned Aerial Vehicles. In Proceedings of the VI International Conference on Forest Fire Research, Coimbra, Portugal, 15–18 November 2010; p. 15. [Google Scholar] [CrossRef]
- Pajares, G. Overview and Current Status of Remote Sensing Applications Based on Unmanned Aerial Vehicles (UAVs). Photogramm. Eng. Remote Sens. 2015, 81, 281–330. [Google Scholar] [CrossRef]
- Pádua, L.; Vanko, J.; Hruška, J.; Adão, T.; Sousa, J.J.; Peres, E.; Morais, R. UAS, sensors, and data processing in agroforestry: A review towards practical applications. Int. J. Remote Sens. 2017, 38, 2349–2391. [Google Scholar] [CrossRef]
- Petrides, P.; Kolios, P.; Kyrkou, C. Smart Cities in the Mediterranean; Progress in IS; Springer: Berlin/Heidelberg, Germany, 2017. [Google Scholar] [CrossRef]
- Çetin, A.E.; Dimitropoulos, K.; Gouverneur, B.; Grammalidis, N.; Günay, O.; Habiboǧlu, Y.H.; Töreyin, B.U.; Verstockt, S. Video fire detection–Review. Digit. Signal Process. 2013, 23, 1827–1843. [Google Scholar] [CrossRef]
- Chung, A.; Kim, D.Y.; Kwok, E.; Ryan, M.; Tan, E.; Gamadia, R. Cloud computed machine learning based real-time litter detection using micro-uav surveillance. In Proceedings of the 2018 IEEE MIT Undergraduate Research Technology Conference (URTC), Cambridge, MA, USA, 5–7 October 2018; pp. 1–4. [Google Scholar]
- WEI, X.; WU, S.; WANG, Y. Forest fire smoke detection model based on deep convolution long short-term memory network. J. Comput. Appl. 2019, 39, 2883. [Google Scholar]
- K Mohammed, R. A real-time forest fire and smoke detection system using deep learning. Int. J. Nonlinear Anal. Appl. 2022, 13, 2053–2063. [Google Scholar]
- Park, M.; Jeon, Y.; Bak, J.; Park, S. Forest-fire response system using deep-learning-based approaches with CCTV images and weather data. IEEE Access 2022, 10, 66061–66071. [Google Scholar]
- Zhang, L.; Wang, M.; Fu, Y.; Ding, Y. A Forest Fire Recognition Method Using UAV Images Based on Transfer Learning. Forests 2022, 13, 975. [Google Scholar]
- Vani, K. Deep learning based forest fire classification and detection in satellite images. In Proceedings of the 2019 11th International Conference on Advanced Computing (ICoAC), Chennai, India, 18–20 December 2019; pp. 61–65. [Google Scholar]
- Ghali, R.; Akhloufi, M.A.; Mseddi, W.S. Deep learning and transformer approaches for UAV-based wildfire detection and segmentation. Sensors 2022, 22, 1977. [Google Scholar] [CrossRef] [PubMed]
- Hu, Y.; Zhan, J.; Zhou, G.; Chen, A.; Cai, W.; Guo, K.; Hu, Y.; Li, L. Fast forest fire smoke detection using MVMNet. Knowl.-Based Syst. 2022, 241, 108219. [Google Scholar] [CrossRef]
- Khan, S.; Khan, A. FFireNet: Deep Learning Based Forest Fire Classification and Detection in Smart Cities. Symmetry 2022, 14, 2155. [Google Scholar] [CrossRef]
- Yandouzi, M.; Grari, M.; Idrissi, I.; Moussaoui, O.; Azizi, M.; Ghoumid, K.; Elmiad, A.K. Review on forest fires detection and prediction using deep learning and drones. J. Theor. Appl. Inf. Technol. 2022, 2, 4565–4576. [Google Scholar]
- Akay, M.; Du, Y.; Sershen, C.L.; Wu, M.; Chen, T.Y.; Assassi, S.; Mohan, C.; Akay, Y.M. Deep learning classification of systemic sclerosis skin using the MobileNetV2 model. IEEE Open J. Eng. Med. Biol. 2021, 2, 104–110. [Google Scholar] [CrossRef] [PubMed]
- Shahi, T.B.; Sitaula, C.; Neupane, A.; Guo, W. Fruit classification using attention-based MobileNetV2 for industrial applications. PLoS ONE 2022, 17, e0264586. [Google Scholar] [CrossRef]
- Patel, R.; Chaware, A. Transfer learning with fine-tuned MobileNetV2 for diabetic retinopathy. In Proceedings of the 2020 international conference for emerging technology (INCET), Belgaum, India, 5–7 June 2020; pp. 1–4. [Google Scholar]
- Nambur, A.; Sankalp Saxena, M.S.S.; Natarajan, M.G. Fire and Non Fire Image Dataset. 2022. Available online: hhttps://kaggle.com/datasets/f7517a19d918cae42ac1222937d07096179e663d7b8ed0a4c66deae33073b21d (accessed on 9 January 2023).
Sl. No. | Name of Component | Number of Components |
---|---|---|
1 | Flight controller (Pixhawk) with accessories | 1 |
2 | Brushless motor, 870 KV | 6 |
3 | Propeller, 11 × 4.7 | 12 |
4 | Electronic speed control, 30a | 1 |
5 | Receiver FS 1A10B (fly Sky) | 1 |
6 | Transmitter i65 (fly Sky) | 1 |
7 | Battery, 10,000 mAh | 1 |
8 | Video transmitter and receiver (Eachine) | 1 |
9 | Telemetry, 915 megahertz | 1 |
10 | GPS (ublox) | 1 |
11 | 3DR power module | 1 |
12 | Connection wires 18,16,14 AWG | 1 |
13 | JST, RMC, and XT-60 connection | 1 |
14 | Carbon fiber frame for hexacopter, 650 mm | 1 |
15 | Camera (Mobius) | 1 |
16 | Double tape, zip ties, and elastic tape | 1 |
17 | Heat shrink tubes | 4 |
18 | Gimble | 1 |
Hyperparameters | |
---|---|
No_of_Epochs | 30 |
Minimun_Batch_Size | 32 |
Initial_learning_rate | 0.0001 |
Optimizer | Adam |
Data Augmentation Parameters | |
rotation_range | 20 |
zoom_range | 0.15 |
width_shift_range | 0.2 |
height_shift_range | 0.2 |
shear_range | 0.15 |
horizontal_flip | True |
fill_mode | nearest |
Dataset | Loss Value | Accuracy |
---|---|---|
Training | 0.0630 | 0.9793 |
Validation | 0.0826 | 0.9722 |
Dataset | TP | FN | FP | TN | Total |
---|---|---|---|---|---|
Training | 4109 | 11 | 21 | 547 | 4688 |
Test | 1023 | 7 | 6 | 136 | 1172 |
Model | ACC | Precision | Recall | Specificity | F1-Score |
---|---|---|---|---|---|
VGG [47] | 0.732622 | 0.775316 | 0.631871 | 0.498264 | 0.696282 |
Inception [47] | 0.766392 | 0.799119 | 0.63189 | 0.478887 | 0.705733 |
ResNet50 [47] | 0.774748 | 0.805662 | 0.630917 | 0.476558 | 0.707661 |
Proposed X-MobileNet | 0.9889 | 0.9941 | 0.9932 | 0.9510 | 0.9889 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Namburu, A.; Selvaraj, P.; Mohan, S.; Ragavanantham, S.; Eldin, E.T. Forest Fire Identification in UAV Imagery Using X-MobileNet. Electronics 2023, 12, 733. https://doi.org/10.3390/electronics12030733
Namburu A, Selvaraj P, Mohan S, Ragavanantham S, Eldin ET. Forest Fire Identification in UAV Imagery Using X-MobileNet. Electronics. 2023; 12(3):733. https://doi.org/10.3390/electronics12030733
Chicago/Turabian StyleNamburu, Anupama, Prabha Selvaraj, Senthilkumar Mohan, Sumathi Ragavanantham, and Elsayed Tag Eldin. 2023. "Forest Fire Identification in UAV Imagery Using X-MobileNet" Electronics 12, no. 3: 733. https://doi.org/10.3390/electronics12030733