A Learning Strategy for Amazon Deforestation Estimations Using Multi-Modal Satellite Imagery

Abstract

Estimations of deforestation are crucial as increased levels of deforestation induce serious environmental problems. However, it is challenging to perform investigations over extensive areas, such as the Amazon rainforest, due to the vast size of the region and the difficulty of direct human access. Satellite imagery can be used as an effective solution to this problem; combining optical images with synthetic aperture radar (SAR) images enables deforestation monitoring over large areas irrespective of weather conditions. In this study, we propose a learning strategy for multi-modal deforestation estimations on this basis. Images from three different satellites, Sentinel-1, Sentinel-2, and Landsat 8, were utilized to this end. The proposed algorithm overcomes visibility limitations due to a long rainy season of the Amazon by creating a multi-modal dataset using supplementary SAR images, achieving high estimation accuracy. The dataset is composed of satellite data taken on a daily basis with relatively less monthly generated, ground truth masking data, which is called the many-to-one-mask condition. The Normalized Difference Vegetation Index and Normalized Difference Soil Index bands are selected to comprise the datasets. This yields better detection performance and a shorter training time than datasets consisting of RGB or all bands. Multiple deep neural networks are independently trained for each modality and an appropriate fusion method is developed to detect deforestation. The proposed method utilizes the distance similarity of the predicted deforestation rate to filter prediction results. The elements with high degrees of similarity are merged into the final result with average and denoising operations. The performances of five network variants of the U-Net family are compared, with Attention U-Net observed to exhibit the best prediction results. Finally, the proposed method is utilized to estimate the deforestation status of novel queries with high accuracy.

1. Introduction

Forests occupy more than 30% of the global terrestrial surface area and serve as habitats for numerous species of plants and animals [1]. Additionally, they play a significant role in the global ecosystem by preventing soil erosion and reducing the effects of climate change via a process known as carbon cycling, which stores large amounts of carbon in the soil and releases it back into the atmosphere [2]. In particular, the Amazon rainforest covers an area of 5.5 million km², making it the largest tropical forest, accounting for 40% of all tropical rainforest areas on Earth [3]. However, forested regions have decreased by 11,568 km² in 2022 according to the Legal Amazon Deforestation Satellite Monitoring Project (PRODES) and the National Institute for Space Research (INPE) [4], and this decline is considered to be one of the most serious environmental problems at the present moment. The degradation of forests induces a loss of biomass and the devastation of natural resources, eventually disrupting the ecological balance and causing far-reaching changes across the globe [5]. In particular, deforested regions in the Amazon rainforest have increased from less than 100,000 km² to more than 730,000 km² over the past 40 years [6] according to satellite monitoring data collected by the Brazilian National Institute for Space Research (INPE). To make matters worse, the deforestation rate is increasing yearly, having increased by more than 13% in 2022 compared to the previous year [4]. This could create landslides or collapses due to soil erosion and may cause alarming animal and plant habitat losses [7]. At this rate, the role of the Amazon rainforest as a carbon sink may end by 2035, accelerating global warming significantly [8].

Analyses of changes in time-series data, necessary for understanding deforestation trends, require continuous and extensive monitoring [9]. The INPE has been monitoring the deforestation rate of the Amazon region via the PRODES and the Real-time Deforestation Detection System (DETER) since the 1980s [5]. The DETER project is based on medium-resolution satellite images and serves as an early detection tool to minimize the damage caused by deforestation. Additionally, the PRODES project provides accurate deforested area maps and calculates the annual deforestation rate [10].

Several deforestation monitoring studies have been proposed based on remote-sensing data. For the monitoring of forest cover using optical data, the Normalized Difference Vegetation Index (NDVI) is used to maximize the contrast between the background and greenery to ascertain the characteristics of green vegetation or to estimate the amount of plant biomass present on the surface via a spectral signal analysis [1,11]. Schultz et al. [12] evaluated time-series Landsat performance outcomes through various vegetation indices, including NDVI, to identify deforestation in tropical regions. DeVries et al. [13] constructed an NDVI map based on Landsat data to identify deforestation and forest degradation in Ethiopia. Alternatively, the pixel-based Break For Additive Seasonal Trends (BFAST) method can be used to detect changes based on the prediction modeling of acquired time-series data [14] for deforestation detection. BFAST detects pixels that deviate significantly from breakpoints via a time-series data analysis and expresses the type of change using different intensity levels [15]. It can be used to detect deforestation in nearly real time and can help to identify breakpoints in long-term pattern data [14,16]. Nelson [17] studied band ratios and image differencing to detect changes in the forest canopy and confirmed that vegetation index differencing yields the best performance. Meanwhile, a number of studies on deforestation characteristics using SAR as well as optics have been conducted. Field research in the Madre de Dios region of Peru [18] revealed a marked reduction in the (L-band, HV) backscatter of SAR in portions of the forest that had undergone deforestation. In addition, deforested dry forests in Bolivia exhibited a clear reduction in cross-polarized backscattering values in both the L- and C-bands [19].

The multi-sensor data fusion strategy combining SAR and optical sensors such as Landsat has improved the accuracy of forest mapping [20,21,22]. However, there are limitations to the use of a fusion method to detect forest changes, such as discrepancies in collection dates and the misalignment of multiple forms of data [23,24].

Forest research, which has classically focused on normalized vegetation indices or individual tree classification [25], has been extended to include the integrated monitoring of large regions by incorporating deep learning technology [26,27,28,29]. This state-of-the-art technology detects deforestation with the best performance, using optical and SAR datasets [30,31,32,33]. Mazza et al. [34] conducted a study on the detection of forests with high-resolution images of 5 m using various CNN networks, with the best detection performance exhibited by U-Net [35]. Using Landsat imagery, Maretto et al. [36] modified the U-Net architecture to enable the integration of both spatial and temporal contexts to detect deforestation. Torres et al. [37] evaluated the performance of different networks to detect deforestation in the Amazon rainforest based on Landsat 8 and Sentinel-2 data. Through this, it was confirmed that high-resolution Sentinel-2 images increase the detection accuracy, suggesting that high deforestation detection is possible when PRODES and deep learning technology are combined. In addition, this deep learning technology was hypothesized to detect changes in Ukrainian forests by applying Sentinel-2 data to the U-Net in Isaienkov et al. [38], and John and Zhang [39] proposed an attention U-Net [40] segmentation network to detect deforestation in the Amazon region and South America. In the Amazon and Cerrado biomes, Ortega Adarme et al. [41] evaluated patch-wise classification algorithms for the automatic detection of deforestation, finding that deep-learning-based approaches surpassed the SVM (support vector machine) [42] baseline on all performance metrics. In the context of the change detection of deforestation tasks, Soto et al. [43] proposed a domain adaptation approach that takes into account several locations in the Amazon and Brazilian Cerrado biomes to increase the accuracy of cross-domain deforestation detection. De Andrade et al. [44] extended the original DeepLabv3+ to solve class-imbalanced problems in deforestation detection in the Amazon using Landsat OLI-8 images. Zhang et al. [45] used U-Net with LSTM to detect the deforestation change in China with Sentinel-2 imagery. Islam et al. [46] analyzed forest cover changes in the Sundarbans in India with a transfer learning method using Sentinel-2A imageries from 2016 and 2022.

In general, the training strategy used for deep learning is determined by the dataset modality features. Learning based on a single modality dataset is called single-view single-network learning. On the other hand, multi-view learning strategies are applied to learn a common functional space or shared patterns with consistent high-level semantics from multi-view data obtained from multiple modalities, sources, and formats [47,48]. To provide more universal representations, the multi-view CNN architecture combines multi-view data from several sources. In this study, we propose a novel multi-view learning strategy to identify deforestation using a multi-modal dataset consisting of three different modalities.

The main contributions of this research are summarized below.

• We propose a dataset selection strategy based on a spectral index that can achieve high performance with less learning time compared to when large-capacity initial multimodal datasets are used.
• A novel multi-view learning strategy for a multi-modal satellite dataset is proposed, and it applies to diverse conditions regardless of the availability of modalities.
• The proposed fusion strategy, which combines multiple outputs from three networks, achieves high accuracy for deforestation estimations in the Amazon area.

2. Dataset

This study was planned around the region and dataset provided by the Multimodal Learning for Earth and Environment workshop (MultiEarth) [49] and consists of Sentinel-1, Sentinel-2, and Landsat 8 satellite imagery sets. The region comprises a portion of dense tropical Amazon rainforest in Pará, Brazil, containing thousands of species of broad-leaved evergreen trees, and approximately 7.2 million hectares of forest have been lost in Pará over the past 20 years [50]. The region is bounded by (4.39°S, 55.20°W), (4.39°S, 54.48°W), (3.33°S, 54.48°W), and (3.33°S, 55.20°W), as depicted in Figure 1b [51,52]. The region measures approximately 9500 km² in area and is 80 km wide and 120 km across. The satellite images and ground truth data in the dataset covers the whole study area depicted in Figure 1.

The dataset contains synthetic aperture radar (SAR) images alongside optical imagery. Some studies have reported reduced backscatter measurements in SAR images following a deforestation event. For instance, Bouvet et al. [53] discovered a brief (5–6 months) decline in C–band vertical transmit/vertical receive (VV) values corresponding to deforested regions, consistent with patterns observed in deforested tropical forests in Indonesia [19,54].

Figure 1. (a) Deforestation map of Brazil [55], and (b) study area in the Amazon in Brazil (2021).

Figure 1. (a) Deforestation map of Brazil [55], and (b) study area in the Amazon in Brazil (2021).

In this study, Sentinel-1 (C-band) imagery is used, which leverages the characteristics of SAR to detect deforestation, using two polarization bands (Vertical-Vertical (VV) and Vertical-Horizontal (VH)) with a 10 m resolution. Sentinel-2 consists of twelve bands ranging from visible to shortwave infrared (SWIR) with spatial resolutions of 10 m, and Landsat 8 has eleven bands with spatial resolutions of 30 m.

All collected satellite images were divided into (256 × 256) size patches, and a single patch covers an area with a latitude and longitude range of 0.02°. The raw dataset is accessible on the MultiEarth workshop [49] webpage in the form of patches with the specified latitude and longitude range. Final patches are converted so that they have a spatial resolution of 10 m according to the masking data format. There are 69,977, 68,272, and 32,642 Sentinel-1, Sentinel-2, and Landsat 8 daily patches included in the dataset, respectively, and 17,514 monthly mask patches are used, significantly fewer than the number of daily satellite image patches.

The deforestation mask patches are generated based on Planet satellite data [56] with a spatial resolution of 3.7 m on the RGB band. In this study, we refer to the ground truth image as a ‘mask patch,’ as the deforestation estimation task is a binary classification problem (deforestation or not) [57]. The mask data consist of data captured between 2016 and 2021, excluding those captured between October and April each year, which corresponds to the rainy season in the region. The masking maker establishes the following criteria to identify deforestation areas [58]: (1) Areas that underwent the intentional felling of trees by humans are deemed to be deforestation regions. (2) Forests with continuous forest canopies exceeding 1 ha are not marked. (3) The remaining unlabeled areas where rivers with areas exceeding 1 ha pass through deforested regions are marked as deforestation regions. Finally, each pixel of the raw data is converted to 0, corresponding to forested areas, or to 1, corresponding to deforested areas. Figure 2 shows examples of three types of satellite patches and their corresponding mask patches.

A special feature of the dataset in this study is that multiple patches, taken at irregular interval days from three different satellites, correspond to a single mask patch generated on a monthly basis. Three types of satellite data are collected at different time resolutions for each month and region, and the observation frequency is irregular, meaning that the collected volume from those satellites differs even for each month. Such a problem, where multiple patches for the same area are matched to one mask patch, is known as ‘many-to-one labeling’ [59]. Given that we decided to refer to the ground truth image as ‘mask’, this problem is referred to as ‘many-to-one mask’ for the duration of this paper.

The dataset is structured with dates, locations, and corresponding mask and multi-modal source patches. Let us define the set of dates (D), locations (L), deforestation mask patches (M), patches of Sentinel-1 (S1), patches of Sentinel-2 (S2), and patches of Landsat 8 (L8), as shown in Equations (1)–(6):

$$D=\left\{d_1,d_2,\dots ,d_{n_D}\right\}=\left\{yyyymm\dots \right\}=\left\{201608,201707,\dots ,202108\right\},$$

(1)

$$\begin{gathered} L=\left\{l_1,l_2,\dots l_{n_L}\right\}=\left\{\left({lon}_1,la{t}_1\right),\left({lon}_2,la{t}_2\right),\dots \left({lon}_{n_L},{lat}_{n_L}\right)\right\} \\ =\left\{\left(54.48,3.33\right),\left(54.48,3.35\right),\dots \left(55.20,4.37\right),\left(55.20,4.39\right)\right\}, \end{gathered}$$

(2)

$$M=\left\{M^{l_1,d_1},M^{l_2,d_2},\dots ,M^{l_{n_L},d_{n_D}}\right\},$$

(3)

$${S1}^{l,d}=\left\{{s1}_1^{l,d},{s1}_2^{l,d},\dots ,{s1}_i^{l,d}\right\},\hspace{1em}\forall d\in D,\forall l\in L,$$

(4)

$${S2}^{l,d}=\left\{{s2}_1^{l,d},{s2}_2^{l,d},\dots ,{s2}_j^{l,d}\right\},\hspace{1em}\forall d\in D,\forall l\in L,$$

(5)

$${L8}^{l,d}=\left\{{l8}_1^{l,d},{l8}_2^{l,d},\dots ,{l8}_k^{l,d}\right\},\hspace{1em}\forall d\in D,\forall l\in L,$$

(6)

where n_D is a set number representing a date, n_L is a set number representing a location, and (lon, lat) refers to longitude and latitude, respectively. In addition, i, j, and k, respectively, represent the number of patches for Sentinel-1, Sentinel-2, and Landsat 8 included at the given location l and on date d.

Figure 3 shows an example of the May 2018 dataset. The entire set of data for the month consists of one mask patch representing the eleven patches from three different satellites.

The entire dataset is divided into the training, validation, and test sets on a regional basis, given the strong relationship between the trait and the location. Because the pattern or aspect of deforestation can change depending on the regional characteristics, the dataset was constructed to learn regional features and prevent overfitting and erroneous learning, as opposed to using of random selection.

3. Proposed Training Strategy

3.1. Optimized Data Selection

For high-performance deforestation detection for arbitrary regions during arbitrary seasons, we constructed a multi-modal dataset in this study. Deforestation changes the reflectance of vegetation in satellite images such that suitable band selection from the different satellites is crucial to detect this phenomenon efficiently. Deforestation induces a large change in the reflectance in the shortwave infrared, vegetation red edge, and red bands [60]. Thus, B4 (Red), B7 (Vegetation Red Edge), B11 (Shortwave Infrared), and B12 (Shortwave Infrared) bands are selected for Sentinel-2, and B4 (Red), B6 (Shortwave Infrared), and B7 (Shortwave Infrared) bands are selected for Landsat 8.

The NDVI is frequently used as an indicator to extract seasonal data based on the relationship between spectral reflectance and vegetation characteristics [61]. As given by Equation (7), the RED band represents the spectral absorption by the vegetation, and the near-infrared (NIR) band accounts for the majority of change in reflectance in this case. In addition, the Normalized Difference Soil Index (NDSI, Equation (8)) uses SWIR and NIR band information, and SWIR accounts for the majority of change in the reflectance values of soil areas. Thus, the NDSI could be a good indicator for distinguishing vegetated areas from soil-degraded areas [62], and study [60] developed a model that effectively detects forests even in areas with low reflectivity caused by clouds or shadows, by combining multiple satellites and indexes. Therefore, to compose the two aforementioned spectral indices, the B8 (Near Infrared) band of Sentinel-2 and the B5 (Near-Infrared) band of Landsat 8 are additionally added to configure the dataset. Finally, eleven bands in total are selected for this study based on the aforementioned indices and related works. These are described in

Table 1. Final data specifications of the multi-modal dataset.

Type	Selected Bands	Patch Size	Pixel Resolution
Sentinel-1	VV, VH	2 × 256 × 256	10 m
Sentinel-2	B4, B7, B8, B11, B12	5 × 256 × 256	10 m
Landsat 8	B4, B5, B6, B7	4 × 256 × 256	10 m (resized with bicubic interpolation from 30 m resolution)

.

$$NDVI=\frac{\left(NIR-RED\right)}{\left(NIR+RED\right)},$$

(7)

$$NDSI=\frac{\left(SWIR-NIR\right)}{\left(SWIR+NIR\right)},$$

(8)

3.2. Learning Strategy

As described in the previous section, images from three different satellites are utilized in this study, and because each type of satellite data has unique characteristics, each can be recognized as a single “view”. The single-view learning strategy, described in Figure 4a, refers to training and inferring each modality’s output with only one view. In this study, there are three views that must be acquired in what is referred to as a “multi-view learning” strategy. Multi-view architectures are usually divided into multi-view-one-network and one-view-one-network strategies [63], as depicted in Figure 4.

The multi-view-one-network strategy feeds various types of data into a single network, and fused results are generated through the network. Most studies that use two or more satellite datasets utilized this method [37,39]. On the other hand, another network is employed for each view in the one-view-one-network strategy and multiple inference results are then fused during the next step.

The multi-view-one-network strategy is inappropriate for this study because the numbers of training compositions and volumes in the training set vary irregularly from month to month. Therefore, we used the one-view-one-network strategy to utilize the unique characteristics of each satellite image and to ensure applicability even when there are very few specific views that can be utilized. One of the basic principles of one-view-one-network learning is the complementary usage of information contained in multi-source data. For example, missing optical information (Sentinel-2, Landsat 8) due to harsh weather conditions can be supplemented using SAR (Sentinel-1) images, and insufficient information in Landsat 8 images owing to low image quality levels can be supplemented using Sentinel-2. As depicted in Figure 4, an independent network is trained for each view, and each trainer adopts a network that yields the optimal performance. In this study, U-Net is selected as the backbone network due to its simple network structure and good performance for segmentation. Four more U-Net variants, specifically R2U-Net [64], Attention U-Net, Attention R2U-Net [40], and Nested U-Net [65], are applied and the performance capabilities of each are compared.

3.3. Fusion Strategy

This section describes how to convert the multiple outputs from the one-view-one-network into one final result. As shown in Figure 4, the proposed training strategy consists of three steps: (1) pre-processing and input into distinct views, (2) training using the individual network, and (3) multi-output fusion and post-processing. After the pre-processing step, the three datasets were entered into the three learning models to learn the features of each view. After the training step, view-specific features were integrated into the following fusion method to obtain a compact, discriminative shape feature. That is, the fusion step aims to combine the reconstruction of multi-view features by minimizing reconstruction errors and changing the set of multiple predicted results into one representative prediction (Y).

Figure 5 depicts the entire process of deriving the final deforestation detection result through the learning strategy and the fusion method with a multi-modal dataset proposed in this study.

The problems that originate from the number of inputs from each satellite differ depending on the location and date of the patch. Therefore, the number of prediction result sets (n) used in the final output also differs. Commonly used methods in the fusion layer include deriving the average value of all multiple predicted elements (Average), adopting the only the maximum value among all output elements (Maximum), and using the trimmed mean method after limiting the maximum and minimum values of elements (Trimmed Average) [63]. These are described in

Table 2. Different types of fusion strategies.

Fusion Types	Mathematical Expression
Average	$Y_{Avg}=\frac{\sum P_x}{n}$
Maximum	$Y_{Max}=max\left(P_x\right)$
Trimmed Average	$Y_{TA}=\frac{\sum \left(\frac{P_x-Y_{min}}{Y_{Max}-Y_{min}}\right)}{n}$
Distance Similarity	$Y_{DS}=\left|R_x-\overline{R_x}\right|$, $\overline{R_x}=\frac{\sum R_x}{m}$

and the subscript x appearing in all formulas in Table 2 means the index of the daily data set used to predict deforestation in a specific region and month, as described in Figure 3.

All of the other fusion methods are generally calculated in a pixel-wise method and finally expressed as one 2D image. In this study, we suggest the rate of deforestation (R), which represents the deforestation rate within one predicted result (P), as described in Equation (9). If all pixels are predicted to be deforestation, the value is 100%, whereas if all pixels are predicted to be forest, the value is 0%. All of the predicted 2D images ($P^{l,d}$) are converted into a collection of elements representing the deforestation rate ($R^{l,d}$).

$$\mathbf{R}\left(P^{l,d}\right)=\frac{numberofdeforestationpixelsof{P}^{l,d}}{totalnumberofpixelsof{P}^{l,d}}=\frac{{\sum }_{1\in P^{l,d}}^{}1}{N}\times 100\left(\%\right)=\left\{R_1,\dots ,R_n\right\},\forall R\in R$$

(9)

In addition, we proposed a method of calculating the ‘distance similarity’ between the predicted elements (R) and mean ($\overline{R}$) of the elements set, as described in Equation (10).

$$Y_{DS}=\left|R_x-\overline{R_x}\right|,$$

(10)

After certain elements with the lowest degree of similarity are eliminated according to the criteria of Equation (11), the distance similarity is recalculated with the remaining elements. Afterwards, only the elements corresponding to the condition in Equation (12) remain, indicating that the degree of similarity has become stable. In this step, eliminated elements, which have a low degree of similarity, may indicate outliers. For example, inference images with particularly low or high deforestation rates compared to other elements could have heavy cloud cover, which results in erroneous prediction results that would be removed with this procedure.

$${Y_{Dis}^{l,d}}_1=\left\{-3\sqrt{\frac{1}{n}{{\sum }_{i=1}^n\left(R_i-\frac{{\sum }_{i=1}^n{R}_i}{n}\right)}^2}<Y_{Dis}^{l,d}<+3\sqrt{\frac{1}{n}{{\sum }_{i=1}^n\left(R_i-\frac{{\sum }_{i=1}^n{R}_i}{n}\right)}^2}\right\},$$

(11)

$${Y_{Dis}^{l,d}}_2=\left\{-\sqrt{\frac{1}{m}{{\sum }_{i=1}^m\left(R_i-\frac{{\sum }_{i=1}^m{R}_i}{m}\right)}^2}<Y_{Dis}^{l,d}<+\sqrt{\frac{1}{m}{{\sum }_{i=1}^m\left(R_i-\frac{{\sum }_{i=1}^m{R}_i}{m}\right)}^2}\right\},$$

(12)

Here, n is the total number of elements constituting the initial prediction result, and m is the total number of remaining elements after Equation (11) is applied. The difference between Equations (11) and (12) is that the filtering range for the degree of similarity is different. The filtering range of Equation (11) is three sigma of the deforestation rate, which can eliminate outliers in 99.7% of the data range with the assumption of normal distribution. With this equation, we can eliminate certain outlier prediction results. Afterward, Equation (12), which has one sigma range for the rate, would wipe out additional elements with low degrees of distance similarity.

Only elements with high degrees of similarity remain after applying the proposed method, and the predicted images of $P_x$ matching the same index as these elements, $R_x$, are averaged and regenerated as a single image. Finally, an opening operation (Equation (13)) is applied to denoise the obtained image. The opening operation (∘) involves applying an erosion operation ($\ominus$) that removes small objects effectively, and then applying a dilation operation ($⨁$) that strengthens the shapes in the resulting image.

$$I\circ M=\left(I\ominus M\right)⨁M,$$

(13)

In this equation, I and M denote the image and the structuring element, respectively.

4. Results

In Section 4.1, the effects of the band selection results are presented, and the results of the proposed multi-view strategy, indicating a meaningful improvement in the prediction results, are compared to those of the single-view method in the next subsection. Finally, the effectiveness of the proposed fusion method is evaluated in the last subsection. To validate the proposed method and compare the performance capabilities from a variety perspectives, two evaluation metrics—the F1-score and Intersection over Union (IoU)—are used.

4.1. Band Selection Results

As described in Section 3.1, eleven spectral bands from three satellites were selected to comprise the dataset. We conducted experiments by altering the dataset configuration to evaluate the appropriateness of the proposed band selection method. These results are shown in Figure 6 and Figure 7. Figure 6 shows an example of the prediction results of different band selection methods for Sentinel-2. Each column depicts (a) an RGB patch of Sentinel-2 data, (b) the prediction result based on the RGB band, (c) the prediction result based all bands, and (d) the prediction result based on selected bands. True positives (deforestation) and true negatives (background), which are correct prediction results, are highlighted in green and gray, respectively. An unexpected prediction result when using only RGB bands is the high rate of false negatives, where the model failed to detect actual deforestation. In addition, when utilizing all bands, the prediction result demonstrated similarly high performance. However, training with all bands required an additional 6–7% more time and computation power in the same environment (NVIDIA 2080 Ti GPU).

Figure 7 shows the detection performance (F1-score, IoU) for different band configurations in Sentinel-2 and Landsat 8. In both cases, it can be seen that the highest performance arose when learning with the proposed selected band (green).

4.2. Multi-View Learning Effect

In this section, the results of the proposed learning strategy (one-view-one-network) are compared with those of the single-view method, which utilizes only data from a single satellite.

Table 3. Deforestation estimation performance outcomes of different networks with different dataset configurations.

Network	Metric	Single-View Learning			Multi-View Learning
		Sentinel-1	Sentinel-2	Landsat 8	Intersection Dataset (751 Queries)	Expanded Dataset (1355 Queries)
U-Net	F1-score IoU	0.808	0.711	0.758	0.889	0.871
		0.703	0.634	0.671	0.812	0.785
R2U-Net	F1-score IoU	0.703	0.538	0.630	0.752	0.758
		0.568	0.402	0.540	0.631	0.642
Attention U-Net	F1-score IoU	0.841	0.719	0.773	0.897	0.883
		0.743	0.642	0.690	0.823	0.801
Attention R2U-Net	F1-score IoU	0.704	0.582	0.729	0.786	0.789
		0.576	0.450	0.6180	0.677	0.680
Nested U-Net	F1-score IoU	0.811	0.708	0.760	0.890	0.874
		0.705	0.629	0.673	0.814	0.789

presents the effectiveness of the multi-view learning strategy. As described in Table 3, when the proposed multi-view learning strategy is applied, the evaluation metrics are significantly improved for every network type, meaning that the proposed multi-view learning method exhibits a superior learning performance compared to the single-view results. In addition, these performance improvements, compared to the single-view result, are identical in the U-Net family test overall.

Meanwhile, as mentioned in Section 2, the total number of mask patches in the test set is 1355. Therefore, 1355 novel queries are available for testing the proposed deforestation estimation algorithm. However, only 751 queries contain all three types of satellite imagery. These queries are known as the ‘Intersection dataset’, indicating the availability of all three types of modalities. The entire test dataset, consisting of 1355 queries, is called the ‘Expanded dataset’, indicating the availability of partial sets of modalities. Table 3 summarizes the evaluation results for single-view-based predictions and the proposed multi-view strategy-based predictions for the five different networks. The best values among each criterion are highlighted in bold.

The three cases shown in Figure 8 depict single-view results using only data from a single satellite and the prediction results with multi-modal data when using the proposed training strategy. As shown, in the single-view results, if there are heavy clouds or poor detection conditions, the final performance is reduced. However, given that the proposed model was trained under all conditions, it shows a high and stable performance regardless of exceptional conditions.

4.3. Performance of the Proposed Fusion Method

Here, we evaluate the performance of the proposed fusion method. The performance of existing fusion strategies, specifically the Average, Maximum, and Trimmed average methods, were compared with the proposed Distance similarity method.

Table 4. Comparison of the detection performance capabilities of the different fusion methods.

Fusion Type Metric	Average	Max	Trimmed Mean	Proposed Fusion Method
F1-score IoU	0.89 (σ = 0.12)	0.84 (σ = 0.10)	0.87 (σ = 0.10)	0.90 (σ = 0.10)
	0.81 (σ = 0.14)	0.74 (σ = 0.14)	0.78 (σ = 0.13)	0.82 (σ = 0.13)

describes the detection performance of the four fusion strategies. The Maximum method shows the lowest performance, likely because only the maximum case was utilized without judgments of any other predicted results. In addition, the Maximum method tends to predict wider possible deforestation areas. On the other hand, the Average method shows high accuracy relative to that of the proposed model, though the high standard deviation implies that the true negative rate is very high, as shown in the first case in Figure 9. In contrast, with regard to Trimmed average, it is applied with outliers eliminated, with the result being that false positives are clearly reduced. However, as shown in the second and last examples, for Trimmed average, there is a considerable variation in accuracy depending on the image. This may have occurred because some false positives were not removed, depending on the range of the prediction results. In contrast, for the proposed method, only the prediction result that is as close to the ground truth as possible remains through the distance similarity evaluation, therefore showing similar performance outcomes in all cases.

4.4. Annual Deforestation Change Analysis

Using the proposed model, we extended the time scale to analyze the detected annual deforestation levels for the entire test area. Deforestation areas in 2016 are highlighted in yellow. The years 2016 to 2019, and 2019 to 2021, are shown in red and blue, respectively, as illustrated in Figure 10. From this color marking, it is evident that the deforested area in the test region has steadily increased over the past five years.

Similar results are observed in the annual forest loss output by Hansen et al. [66]. Hansen’s study provides various analysis results, including information on loss due to fire, as well as deforestation information for the annual global region. Since it encompasses this information, it appears to be a valuable comparative analysis tool when expanding the model to other regions in the future. Through this proposed learning strategy, this study confirmed that the model exhibits high performance and demonstrates the possibility of analyzing monthly and even annual changes, despite the limitations of daily images and monthly mask configuration.

5. Discussion

5.1. Cloudy Satellite Image Cases

In this study, a multi-modal satellite dataset containing images from three different satellites is proposed with an optimal learning strategy that can perform similarly in all environmental conditions. In this section, the model performance is discussed under various cloud conditions.

Cloudy images are classified into three types depending on the amount of cloud coverage. First, scattered clouds allow near-perfect surface visibility for humans, and, in this case, the proposed model predicted deforestation areas satisfactorily. Examples of this type and the corresponding prediction results are depicted in Figure 11 (1) under ‘Scattered Cases’. Second, more dense clouds are labeled as ‘Broken Cases’. In this case, the clouds are too thick for the surface to be observed by humans. However, the proposed method can estimate the deforestation status with high accuracy.

In the last case, the cloud cover is extremely dense, and the ground is not visible at all. These are labeled as ‘Overcast Cases’. Such cases are common in Amazonian forests, especially during the rainy season. Related research has focused on estimations of deforestation in the Amazon region [37] based on electro-optical (EO) images taken between July and August. If we trained the model with only a single optical satellite or with the single-view learning method, the model would not have detected deforested areas well. However, we can utilize multi-modal datasets, e.g., SAR images obtained from Sentinel-1 and clear images from another day, and an optimal dataset configuration with selected bands including the IR range to predict deforestation. Thus, even in the presence of overcast skies, highly accurate predictions can be obtained using other clear/SAR images.

5.2. Estimation Performance for Different Time Scales in the Many-to-One-Mask Condition

This study proposed an optimal learning strategy and fusion method for the many-to-one-mask dataset environment. The features of daily images are learned through the proposed one-view-one-learning approach, and through the fusion method it is confirmed that final high-performance monthly detection results are derived. In other words, results showing very high performances can be obtained without generating matching masks for all daily satellite images, confirming the possibility of constructing an efficient ground truth dataset. In particular, this method maintains high performance regardless of the amount of data or the composition, and it operates regardless of the satellite revisit time or in the event of data collection problems caused by weather conditions.

In addition, high detection accuracy for daily changes was confirmed by the proposed method. As shown in Figure 12, as a result of analyzing two cases with a difference of about two weeks, it was found that daily changes are clearly detected despite training based on monthly masks. Case 1 utilizes Sentinel-2 RGB patches captured on 2 July and 18 July 2017. Unlike the first result (yellow), the newly detected result (orange) identifies an expansion of approximately 6% of the deforestation area. Similarly, in case 2, it is observed that the deforestation area, which was about 11% as of 3 August 2017, increased to 17% as of 19 August 2017. This result shows that the proposed model defines deforestation areas as well as the exact ratio of the changed deforestation area.

6. Conclusions

In this study, a novel learning strategy for deforestation estimations using a multi-modal satellite dataset is presented. The dataset, which contains three different types of satellite data (Sentinel-1, Sentinel-2, and Landsat 8), consists of daily captured images and monthly masked ground truth deforestation information. The proposed multi-view learning strategy consists of three phases—optimizing data selection, multi-view learning, and result fusion. The detection performances with respect to varying input band configuration strategies—RGB, all, and selected bands, based on spectral indices (NDVI, NDSI)—are compared, and the proposed band selection was found to yield a better performance with lower computation costs. The neural networks for each satellite type are independently trained using the one-view-one-network strategy. The networks are then used to infer the individual deforestation estimation rates from each patch and the final estimation results are calculated using the proposed fusion method. It was confirmed that the optimal dataset configuration and learning strategy displayed a robust performance in all weather conditions. The prediction performances of diverse U-Net family networks (U-Net, R2U-Net, Attention U-Net, Attention R2U-Net, and Nested U-Net) were compared in terms of two evaluation metrics, revealing that Attention U-Net achieves the best predictions for both metrics. We expect that a more extensive investigation of time-series deforestation estimations could predict future deforestation risks effectively, with these results then used to formulate appropriate forest protection policies.