Deep Learning-Based Surface Fuel Type Classification from Forest Stand Photographs and Sentinel-2 Time Series ^†

Surface fuel composition and structure are important drivers of fire behaviour and information on the spatial arrangement of surface fuel types is valuable for fire risk assessment in forest ecosystems. However, field assessment of surface fuel types is very time consuming and mapping using airborne or spaceborne sensors is hindered by the tree canopy. We, therefore, suggest a novel remote and proximate sensing-based approach for mapping surface fuel types, while developing a surface fuel type classification for central European forests based on field inventories and expert opinion. Our approach combines in-forest RGB photographs with time series of multispectral satellite data in a deep learning framework: a convolutional neural network (CNN) based on the VGG-16 architecture and a bidirectional long short-term memory network (LSTM) are trained to classify seven different fire-relevant understory and six litter fuel types. Results are compared between different input data sources: horizontal forest stand photos, forest floor photos, Sentinel-2 time series and combinations thereof. The first results show that understory fuel types can be classified with a cross-validated accuracy of 0.78 and litter fuel types with an accuracy of 0.68 using multiple data sources. In addition, we investigate how majority votes using multiple photos in combination with class probabilities can be used to increase the reliability of the predictions. Our algorithm allows one to validate and improve fuel-type maps derived from remote sensing products with GNSS-located photos of the forest stands. The latter can, for example, be obtained by local forest managers or through citizen science. Our approach can, thus, help create more detailed and reliable surface fuel information for central European forests, which forest practitioners and fire fighters urgently need.