Evaluating Agreement Between Global Satellite Data Products for Forest Monitoring in Madagascar

Abstract

Producing high-quality local land cover data can be cost-prohibitive, leaving gaps in reliable estimates of forest cover and loss for environmental policy and planning. Remote sensing data (RSD) offer accessible, globally consistent layers for forest mapping. However, being able to produce reliable RSD-based land cover products with high local fidelity requires ground truth data, which are scarce and cost-intensive to obtain in settings like Madagascar. Global land cover datasets that rely on models trained mostly in well-studied regions claim to alleviate the problem of label scarcity. However, studies have shown that these products often fail to fulfill this promise. Given downstream studies focused on Madagascar still rely on these global land cover products, in this study we compared seven global RSD products measuring forest extent and change in Madagascar to explore levels of similarity across different forest ecoregions over multiple years. We also conducted temporal correlation analysis by checking the correlation between forest area from the different products. We found that agreement levels among the different data products varied by forest type and region, with higher disagreement levels in drier forest ecosystems (dry and spiny forests) than in more humid ones (moist forests and mangroves). For instance, if high agreement is defined as a pixel being classified as a forest by all or all but one product in a year, the average percentage of high-agreement pixels between 2016 and 2020 is just about 8% in the spiny forest and 16% in the dry forest region. These findings underscore the limitations of global RSD products and the importance of localized data for accurate forest monitoring, building justification for efforts to develop a local forest cover product for Madagascar. Our temporal similarity analysis indicates that, although pixel-level maps may show low agreement, temporal aggregates tend to be highly correlated in most cases. We synthesized these results with existing applications of global RSDs in Madagascar to propose practical recommendations for end-users of these products in Madagascar.

1. Introduction

Climate change mitigation solutions (conservation, restoration, and improved land management actions) can provide cost-effective contributions to reducing carbon emissions and keeping global warming below 2 °C [1]. Forests absorb large amounts of carbon from the atmosphere, so managing forests is crucial to reducing global warming. Deforestation leads to the release of greenhouse gases, contributing to climate change. As a result, multiple countries came together to establish the Reducing Emissions from Deforestation and Forest Degradation (REDD+) framework as an international effort to encourage developing countries to reduce emissions from deforestation, and to promote conservation, sustainable management of forests, and enhancement of forest carbon stocks [2]. Under REDD+, developing countries are eligible for results-based financial incentives to reduce emissions through enhanced forest management practices. This incentive system requires reliable and comprehensive data to accurately identify deforestation hot spots and implement effective conservation strategies. Remote sensing data (RSD) provide an objective, cost-effective, and scalable solution for monitoring forests due to their continuous observation and large geographical area coverage, providing critical insights into forest dynamics, deforestation trends, and the effectiveness of forest conservation measures [3].

Madagascar is home to a unique array of plant and animal species within its diverse ecosystems, of which more than 90% are not naturally found anywhere else in the world [4]. These ecosystems range across highlands, humid tropical rainforests, arid deserts, dry deciduous forests, and its unique spiny forests [4]. In recent decades, the country has experienced high deforestation [5], mostly driven by anthropogenic activities including swidden agriculture, mining [6], and conversion to pasture, leading to the loss of about 44% of its natural forest between 1953 and 2014 [7]. Much of Madagascar’s rich biodiversity is found within its tropical forests, categorized into four distinct types: the moist forests in the east, the dry forests in the west, the spiny forests in the south, and the mangroves along the coastline [7,8] (Figure 1).

Figure 1. (A) Forest ecoregions in Madagascar as presented in Vieilledent et al. (2018) [7]. (B) Boundaries of the 119 districts in Madagascar, provided by the United Nations Office for the Coordination of Humanitarian Affairs. The scale and north arrow provided in these figures can be applied in interpreting Figure 2.

Figure 1. (A) Forest ecoregions in Madagascar as presented in Vieilledent et al. (2018) [7]. (B) Boundaries of the 119 districts in Madagascar, provided by the United Nations Office for the Coordination of Humanitarian Affairs. The scale and north arrow provided in these figures can be applied in interpreting Figure 2.

In addition to carbon emission and biodiversity impacts, deforestation has local-scale human health impacts and there is a need for reliable data to quantify forest trends as a pathway to understanding their human health impacts. For example, communities may be exposed to higher temperatures due to deforestation as forested areas experience much cooler temperatures than deforested landscapes due to shade from trees and the role of evapotranspiration in reducing ambient heat [9]. The resulting heat stress from increasing local ambient heat can exacerbate cardiovascular illnesses, metabolic conditions like diabetes, and other severe health risks like dehydration and heatstroke [10].

Furthermore, the transmission mechanisms of infectious diseases like vector-borne and diarrheal diseases can be influenced by deforestation and forest fragmentation through impacts on vector populations, vector–human contact rates, and water sedimentation [11,12,13]. Deforestation has also been linked to water access, which has important health implications [14].

Currently, to our knowledge, there is no publicly accessible multitemporal land cover RSD product specifically produced and validated for all of Madagascar. Zhang et al. (2020) [15] applied a tile-based classification approach to Sentinel-2 satellite imagery to produce landcover classification for the year 2018 in Madagascar using 9220 training and 1278 validation samples. While the results of that study showed 89.2% overall accuracy, neither the resulting land cover map nor the training dataset has been released for public consumption to date.

In the absence of validated local products that are publicly accessible, global products are often used instead (e.g., [16,17,18]) under the blanket belief that they provide valid estimates. Studies have shown that these products exhibit varying levels of accuracy in different climate zones and locations, affecting how useful they can be for local-scale applications [19,20,21,22,23]. For forest extent analysis, these errors could include over- or underestimation of forest area, pixel misclassification, and differences in temporal deforestation trends across different RSD products in varying locations of the world [22]. Global RSD can provide very different estimates of forest extent over the same scene [22].

For the assumption that global RSD products can effectively compensate for the absence of national or sub-national RSD to hold true, multiple RSD products over the same area (Madagascar) should ideally demonstrate a high (or decent) degree of agreement with one another and be interchangeable for downstream analyses. Otherwise, there is the need for extensive validation to be carried out using ground truth data. In this study, we compare seven global RSD products measuring forest extent and change across multiple years. Our comparison focuses on assessing similarity [24] rather than conducting validation. The global data products we consider are Madagascar Forest Extent data combined with Global Forest Change data (MF-GFC), Dynamic World (DW), European Space Agency WorldCover (ESA), ESRI land cover (ESRI), Finer Resolution Observation and Monitoring—Global Land Cover (FROM-GLC), Copernicus Global Land Service Land Cover (CGLS), and PALSAR data, all resampled to a common spatial resolution. Specifically, our study aims are as follows:

• Assess pixel-level agreement between forest extent obtained from these different data products in Madagascar across multiple years.
• Evaluate how levels of agreement between these data products vary across Madagascar’s diverse forest ecoregions.
• Examine the temporal similarity of forest and deforested areas obtained from the different data products. We achieve this by computing the pairwise correlation between district-aggregated forested and deforested areas from all considered RSD products.
• Generate and release the pixel-level agreement maps for multiple years to enable further analysis by the broader scientific community interested in forests in Madagascar and tropical forests in general.
• Provide practical recommendations for users of existing global products in Madagascar while also proposing critical pathways to improving on existing available data (see Section 4.2).

Our similarity assessment is a useful first step and hints at important pathways forward. As documented in Fritz et al. (2012) [24], there are two broad ways to compare RSD products: accuracy assessment (or validation) and checking similarity/agreement. Accuracy assessment requires validation data. In our case, we do not have validation data, hence we rely on similarity assessment as a first step prior to validation data collection. As far as we know, no existing study has documented and compared existing global remote sensing data products in Madagascar. We contribute to conversations on remote sensing for forest monitoring in Madagascar from the standpoint of understanding possible flags in the data and identifying areas to focus validation efforts on. In Section 4.2, we comment on practical and principled ways to use existing data for downstream analyses requiring local forest-related metrics in Madagascar.

The primary contribution of this manuscript is an analysis of the agreement between seven remote sensing datasets for Madagascar, spanning multiple years. While the remote sensing datasets considered are currently used in policy-driven research for Madagascar [6,8], there has been no systematic evaluation of these datasets in any way. Our study makes a first attempt to cover this gap by quantifying similarities and discrepancies, enabling users to make informed choices about data selection. In addition, we translate our findings into actionable recommendations (Section 4.2) to help mitigate measurement errors and biases, ensuring more reliable applications in policy and research.

2. Methods

The detailed methodological workflow is illustrated in Figure 3.

2.1. Data Products

2.1.1. Spatial Boundaries: Forest Ecosystems and Districts

As presented in Vieilledent et al. (2018) [7], Madagascar is divided into four different forest ecoregions; namely, the moist forest in the East, the dry forest in the West, the spiny forest in the South, and the mangroves. Figure 1A presents the spatial boundaries of these four distinct forest ecoregions (we use the terms “ecosystems” and “ecoregions” interchangeably going forward from this point).

In Madagascar, forest management is conducted locally. Hence, we aggregated the forest area per year to the district level [25] to conduct temporal correlation analysis. Figure 1B shows the geospatial boundaries of Madagascar’s 119 districts.

2.1.2. RSD Forest Cover Products

For our analyses, we selected seven data products covering Madagascar (Table 1), each possessing a different spatial resolution (we resampled them to a common resolution of 30 $\mathrm{m}$), time coverage, and satellite sensors used. Among these datasets, three have a native resolution of 10 $\mathrm{m}$, three have a resolution of 30 $\mathrm{m}$, one has a resolution of 25 $\mathrm{m}$, and one dataset has a resolution of 100 $\mathrm{m}$. Collectively, these datasets map forest areas in Madagascar from 2000 to 2023, with each one covering either all or some of this time window. We recognize that other RSD mapping forest areas in Madagascar exist [26]. However, we chose these specific datasets for the following reasons: (i) the need for multitemporal records to track forest metrics trends over time; (ii) the accessibility and availability of the data at no cost; (iii) the inclusion of diverse sensors (both optical and radar) and methodological approaches; and (iv) comprehensive coverage of the entire region of Madagascar. While other datasets or cost-effective alternatives for monitoring tree cover extent exist, these datasets align best with our research objectives.

These RSD differ in their temporal coverage, spatial resolution, satellite sensors, and creation methodologies (e.g., machine learning methods and training datasets). To align the datasets for processing in a unified spatial resolution, we resampled each of them to a common resolution of 30 $\mathrm{m}$ using the nearest-neighbor method [27] and co-registered all the data to a common coordinate reference system (EPSG:4326). All image processing operations were performed using the Javascript interface of Google Earth Engine [28].

• Madagascar Forest Extent data combined with Global Forest Change data (MF-GFC): These data were created using the method outlined in Vieilledent et al. (2018) [7]. For the baseline year of 2000, the forest/non-forest map produced by Harper et al. (2007) [29] was used. This map was derived from the supervised classification of Landsat satellite images in Madagascar and includes 208,000 hectares of areas unclassified due to cloud cover, predominantly within the moist forest domain (4.17 million hectares). To classify these unclassified regions, Vieilledent et al. (2018) [7] applied a 75% tree cover threshold to the year 2000 Hansen et al. (2013) [30] tree cover layer percent. This threshold was optimal for defining forests, particularly in the moist forest ecosystem, where 88% of the unclassified areas are located. This process resulted in the baseline map for the year 2000.
  To produce an annual forest extent map for this dataset, we integrated the baseline map with the annual tree cover loss layer from Hansen et al. (2013) [30]. Specifically, we iteratively removed pixels identified as “tree loss” in the tree cover loss layer from each successive year to create the forest extent map for each year up to 2023. The annual deforestation map for this data product corresponds directly to Hansen et al.’s (2013) [30] tree loss layer. This methodology and data for measuring forest cover and deforestation in Madagascar are well documented in Vieilledent et al. (2018) [7].
  While these data do not include any measurements for gained forest/trees, it has been included in our analyses since it is the most prominently used dataset for forest analyses focusing on Madagascar. In addition, as far as we can report, and as reported by Vieilledent et al. (2018) [7], there is little evidence of major forest regeneration in Madagascar.
• Dynamic World (DW): Global 10 $\mathrm{m}$ near-real-time (NRT)-resolution land cover data, jointly developed by Google and the World Resources Institute. It produces probabilities per pixel for nine land cover types, making it useful for change detection (e.g., detecting deforestation). A neural network model, trained using a combination of hand-annotated imagery and unsupervised methods, is used to operationally generate NRT predictions of land use and land cover (LULC) class probabilities for all Sentinel-2 imagery using cloud computing on Google Earth Engine [28].
  To produce annual tree cover layers with these data, we combined all images within a given year into a mosaic by selecting the class with the highest occurrence (the mode class) for each pixel. We then binarized the annual image by setting the “Trees” class to 1 and all other classes to 0. Although these data are available from mid-2015, we only use them from 2016 to 2023, which are the years with full coverage at the time of our study.
  To produce annual deforestation data for this dataset, we applied a differencing approach to the binarized tree cover layers. This method identifies changed pixels by detecting where pixels with a value of 1 in the previous year’s layer have a value of 0 in the current year’s layer.
  To avoid overestimating forest extent, we did not include the “Flooded vegetation” class in these data as part of our classification of tree/forest extent because this class also includes grass, shrubs, emergent vegetation, and bare ground.
• European Space Agency WorldCover (ESA): This dataset was generated through the supervised classification of combined multi-sensor data from the ESA’s Sentinel-1 and Sentinel-2 constellations, offering a spatial resolution of 10 $\mathrm{m}$. Versions of the data are released annually, beginning with Version 100 in 2020, followed by Version 200 in 2021. The product achieves an overall global accuracy of 75% [31].
  To produce annual tree cover layers with these data, we binarized the annual image setting “Tree cover” class to 1 and other classes to 0. To produce annual deforestation data for this dataset, we applied a differencing approach to the binarized tree cover layers, as already described for the DW data.
• Global ESRI land cover (ESRI): This provides annual global maps of land use and land cover (LULC) from 2017 to 2021. These maps, derived from ESA Sentinel-2 imagery at a 10 $\mathrm{m}$ resolution, provide a representative snapshot of each year by compositing LULC predictions for nine classes throughout the year. This dataset was generated by training a deep learning model for land classification using billions of human-labeled pixels curated by the National Geographic Society. The model was applied to the Sentinel-2 annual scene collections to produce the global maps [32].
  Annual tree cover layers and deforestation maps are produced exactly as described for the ESA data. For this product, we binarize using the “Trees” class to select forested pixels. All other classes are selected as background (with label 0).
  For the same reason stated in the case of the DW data, we did not include the “Flooded vegetation” class as part of our definition of the forest/tree class for these data.
• Finer Resolution Observation and Monitoring—Global Land Cover (FROM-GLC): This dataset was developed to produce the 30 $\mathrm{m}$ global LULC dataset using a combination of satellite sensors including Sentinel-2, Landsat, MODIS, and AVHRR, using a combination of ensemble machine learning and post-processing methods including time consistency check and spatial filtering to produce fully optimized classification maps. The data provide 10 land cover classes including the “Forest” class [33].
  Also, for this data product, annual tree cover layers and deforestation maps were produced exactly as described for the ESA data. We binarized using the “Forest” class to select forested pixels, and all other classes were selected as background.
• Copernicus Global Land Service Land Cover (CGLS): The CGLS land cover data were developed using data from the PROBA-V sensor, processed for quality, and converted to surface reflectance. Integrating high-quality external data and machine learning, CGLS presents global land cover maps with 22 classes, including 12 closed- and open-forest classes, at 100 $\mathrm{m}$ spatial resolution. This makes it suitable for forest monitoring, which is why it has been included in our analysis despite the coarse resolution of the data compared to other data that have been used in this study. These data are available from 2015 to 2019 [34].
  For this data product, which was produced with multiple (12) tree/forest classes, we combined all the forest classes to form a single forest class for each year and selected the remaining classes as background. We also applied a differencing approach, as with DW, to produce annual deforestation data.
  Due to the much coarser resolution compared to the rest of the data (see Table 1), these data were only used for generating pixel-level agreement maps, not in the temporal correlation analyses.
• The Phased Array L-band Synthetic Aperture Radar forest/non-forest map (PALSAR): To obtain this dataset, time series at a global 25 $\mathrm{m}$ resolution of PALSAR-2/PALSAR synthetic aperture radar satellite data were classified by analyzing the backscattering coefficient, assigning strong backscatter pixels as “forest” and low backscatter pixels as “non-forest”. The classification used region-specific radar backscatter thresholds due to varying radar responses in different climate zones. The accuracy was verified with in situ photos and high-resolution optical satellite images [35].
  Two versions of this dataset have been released, covering different periods. The 3-class PALSAR data were applied for the years 2007–2010 and 2015–2017. The 4-class PALSAR data, which splits the “Forest” class into “Dense Forest” and “Non-dense Forest”, are available from 2017 to 2021. To create forest layers for each year, we binarized the data by combining all forest-related classes into a single forest class, with the remaining classes forming the non-forest background. The deforested area map for each year was created as already described for the DW and ESA data.

We use the terms tree, tree cover, and forest interchangeably, based on the definitions provided by the respective data sources for the relevant classes in all the remote sensing data (RSD) applied in our study. The data products presented here not only differ in algorithms, satellite sensors, and methodologies used in producing them, but also in their definitions of forest or tree cover classes.

Table A1. Summary of forest monitoring satellite image products with respective definitions of tree/forest land cover and label names and index(es) for tree/forest class in the data products, and temporal indices indicating coverage years. Each product is identified by a unique name and is associated with a specific spatial resolution and sensor source.

Product Name	Definition of Tree/Forest	Label Name(s)	Label Index(es)
MF-GFC	MF: Primary vegetation dominated by tree cover at least seven meters in height, with neighboring tree crowns touching or overlapping when in full leaf. GFC: Canopy closure for all vegetation taller than 5 m in height.	MF: forest GFC: loss year	MF: 1 GFC: 1–23. 1 is the year 2001, and 23 is the year 2023
DW	Any significant clustering of dense vegetation, typically with a closed or dense canopy. Taller and darker than surrounding vegetation (if surrounded by other vegetation).	Trees	1
ESA		Tree cover	10
ESRI	Any significant clustering of tall (15 feet or higher) dense vegetation, typically with a closed or dense canopy; examples: wooded vegetation, clusters of dense tall vegetation within savannas, plantations, swamp or mangroves (dense/tall vegetation with ephemeral water or canopy too thick to detect water underneath).	Trees	2
FROM-GLC	Broadleaf, needleleaf, mixed, orchard and forest in urban area.	Forest	2
PALSAR	Natural forest with an area larger than 0.5 ha and forest cover over 10%.	Forest	1, 2
CGLS	Closed forest and open forest: evergreen needleleaf, evergreen broadleaf, deciduous needleleaf, deciduous broadleaf, mixed, and others.	Closed forest, open forest	111, 112, 113, 114, 115, 116, 121, 122, 123, 124, 125, 126

presents the definition of forest/tree, class names, and class labels for each remote sensing dataset.

Table 1. Summary of forest monitoring satellite image products with respective spatial resolutions, temporal coverage years, and sensor sources.

Product Name	Spatial Resolution	Time Coverage	Satellite Sensor(s)	Citation
Madagascar Forest Extent data combined with Global Forest Change data (MF-GFC)	30 m	2000–2023	Landsat	[7,29,30]
Dynamic World land cover (DW)	10 m	2016–2024	Sentinel-2	[28]
European Space Agency WorldCover (ESA)	10 m	2020–2021	Sentinel-1, Sentinel-2	[36]
Global ESRI land cover (ESRI)	10 m	2017–2022	Sentinel-2	[32]
Finer Resolution Observation and Monitoring-Global Land Cover (FROM-GLC)	30 m	2009–2020	Sentinel-2, Landsat, MODIS, AVHRR	[33]
PALSAR-2/PALSAR forest/non-forest map (PALSAR)	25 m	2007–2010, 2015–2016, 2017–2020	ALOS PALSAR and ALOS-2 PALSAR-2	[35]
Copernicus Global Land Service Land Cover (CGLS)	100 m	2015–2019	PROBA-V	[34]

Table 1. Summary of forest monitoring satellite image products with respective spatial resolutions, temporal coverage years, and sensor sources.

Product Name	Spatial Resolution	Time Coverage	Satellite Sensor(s)	Citation
Madagascar Forest Extent data combined with Global Forest Change data (MF-GFC)	30 m	2000–2023	Landsat	[7,29,30]
Dynamic World land cover (DW)	10 m	2016–2024	Sentinel-2	[28]
European Space Agency WorldCover (ESA)	10 m	2020–2021	Sentinel-1, Sentinel-2	[36]
Global ESRI land cover (ESRI)	10 m	2017–2022	Sentinel-2	[32]
Finer Resolution Observation and Monitoring-Global Land Cover (FROM-GLC)	30 m	2009–2020	Sentinel-2, Landsat, MODIS, AVHRR	[33]
PALSAR-2/PALSAR forest/non-forest map (PALSAR)	25 m	2007–2010, 2015–2016, 2017–2020	ALOS PALSAR and ALOS-2 PALSAR-2	[35]
Copernicus Global Land Service Land Cover (CGLS)	100 m	2015–2019	PROBA-V	[34]

2.2. Comparing Different Remote Sensing Datasets

2.2.1. Pixel-Level Forest Extent Agreement Maps

Drawing from past studies [20,21,22,37,38], we employed a spatial consistency analysis method to rank pixels by the extent to which they are indicated as a forest across different RSD products. Because this approach relies on the combination of multiple RSD products, we limit our agreement analyses to years with at least five RSD products (2016 to 2020). There were five RSD products concurrently available in 2016 and six in the other years. The set of RSD used for each year changed based on what dataset was available for the year.

Table A2. List of remote sensing datasets used for generating agreement maps for each year.

Year	Available Remote Sensing Datasets	Total Number of Datasets
2016	MF-GFC, FROM-GLC, DW, PALSAR, CGLS	5
2017	MF-GFC, FROM-GLC, DW, CGLS, PALSAR, ESRI	6
2018	MF-GFC, FROM-GLC, DW, CGLS, PALSAR, ESRI	6
2019	MF-GFC, FROM-GLC, DW, CGLS, PALSAR, ESRI	6
2020	MF-GFC, FROM-GLC, DW, PALSAR, ESRI, ESA	6

shows the dataset used for agreement analyses for each considered year.

We implemented this component of our study using Google Earth Engine [39]. For each year, we stacked all available RSD and counted, for each pixel, the number of RSD that classified it as forest. We reported three forest agreement classes: high agreement (classified as forest by all or all but one RSD products), low agreement (classified as forest by only one or two RSD products), and medium agreement (in between low and high agreement). Low-agreement pixels indicate areas where only one or two products agree on the presence of forest cover, suggesting significant uncertainty or disagreement among the data sources. Medium agreement reflects a moderate level of confidence. High-agreement pixels denote areas where all or all but one product agree on the presence of forest cover, indicating high confidence and consistency across the data sources. We calculated the proportion of each agreement level annually for each year and then averaged these proportions across years for each ecosystem to observe the variation in consistency across forest ecosystems.

2.2.2. Temporal Correlation Analysis

To align with the local administrative level, we aggregated estimated areas to district boundaries for each RSD dataset. Specifically, for each measure (i.e., forested area and deforested area), we aggregated the data by counting all classified pixels within each of the 114 districts in Madagascar and then multiplied by the pixel area ( 30 $\mathrm{m}$ × 30 $\mathrm{m}$). We then computed similarity matrices comparing every data pair using two different evaluation metrics: the root mean square deviation (RMSD) and the Spearman correlation coefficient ($\rho$) [40]. The RMSD, as defined in Equation (1), quantifies the average magnitude of errors between pairs of observations, providing a symmetric measure of discrepancies across two sets of area measurements. The Spearman correlation coefficient ($\rho$) measures the strength and direction of the association between two ranked variables, serving as a non-parametric measure that does not assume a normal distribution of the data. While the RMSD measures the mean distance between district-level aggregated areas from data pairs, $\rho$ assesses the similarity in trends between two datasets by evaluating how consistently their ranks correspond to each other, indicating their collinearity.

$$\mathrm{RMSD}=\sqrt{{\displaystyle \frac{1}{mn}}\sum _{i=1}^n\sum _{y=1}^m{({\mathrm{area}}_{iy}^{\left(a\right)}-{\mathrm{area}}_{iy}^{\left(b\right)})}^2}$$

(1)

where i indexes the district, y denotes the year, and a and b represent two different RSD pairs. The RMSD is calculated over all districts (n) and all overlapping years (m) between a and b.

3. Results

3.1. Forest Extent Agreement Analysis Results

Sample comparisons of forest extent classification across multiple datasets for different forest types in Madagascar are shown in Figure A1. This comparison illustrates variation across RSD products in classifying forest extent over the same area.

We found that in 2016, the moist evergreen forest ecosystem in the east showed a higher density of high-agreement areas compared to the drier western and southern regions. The spiny forests in the deep south exhibited mostly medium-to-low-agreement areas. There are changes across years in the spatial allocation of agreements (Figure 2). For example, locations classified as high agreement in 2017 changed to medium and low agreement in 2019 (see inset in Figure 2).

We found that the proportion of low-agreement pixels is almost stable across the years but there is a reduction in high-agreement pixels between 2017 and 2018, leading to an increase in medium-agreement pixels (Figure 4a). In comparison to wet ecosystems (moist forest and mangroves), drier forest ecosystems (dry and spiny forests) show lower proportions of high-agreement pixels, suggesting moist and mangrove forests are more easily and consistently identified across RSD products than drier ecosystems like dry and spiny forests (Figure 4b).

3.2. Temporal Consistency Analysis Results

When aggregating forest area to the district level, DW exhibits the lowest correlation and highest (worst) RMSD consistently when paired with any of the rest of our considered RSD products, showing poor agreement (Figure 5d). We find that there is a strong correlation between ESRI and PALSAR (0.99), ESRI and ESA (0.95), PALSAR and ESA (0.94), and MF-GFC and ESA (0.88), among others (Figure 5c). Consistent with the high agreement measured by correlation, the lowest and best RMSD for forest area is between ESRI and PALSAR (238.32 km²) (Figure 5d).

After calculating the deforested area for each product and similarly aggregating to the district level, we found that there is strong agreement between MF-GFC and FROM-GLC (0.92), ESRI and ESA (0.90), ESRI and PALSAR (0.90), and MF-GFC and ESA (0.89). As with forest area, there are lower correlation values between DW and all the rest of the RSD products (Figure 5a). We also found that using the RMSD metric to measure the mean deviation between deforested area time series, DW consistently showed the highest deviation value from all the RSD products, indicating poor agreement. In contrast, MF-GFC and FROM-GLC demonstrated the best pairwise RMSD value at 33.22 km², reflecting a high level of agreement (Figure 5b). In summary, the MF-GFC and FROM-GLC pair showed the best correlation and RMSD values for deforestation.

4. Discussion

In this study, we have assembled and compared multiple remote sensing products estimating forest extent in Madagascar across multiple years. Forest monitoring and conservation in Madagascar relies on accurate observation of forest extent and disturbance in space and time, and remote sensing data provide an objective and efficient avenue for such monitoring, but have previously been challenging to use as global RSD products have not been compared specifically for the context of Madagascar. Furthermore, disturbances in forest ecosystems have significant impacts on humans and wildlife. Improving our understanding of agreements and disparities among different global forest extent data products can inform the key considerations we make in using the products.

4.1. Forest Extent Agreement

Previous studies in Mexico [22] and China [21] have conducted spatial and temporal consistency analyses to assess confidence in RSD for land cover mapping and monitoring. We introduced a multitemporal dimension to the spatial similarity analyses, hypothesizing that the level of agreement between multiple RSD products measuring forest extent and deforestation is different for different years. Our results show that the proportion of low-agreement areas stayed relatively unchanged from 2016 to 2020, and more areas were classified as high-agreement in 2016 and 2017 compared to later years. This means that when using global land cover products for local-level analyses across different time points, it is insufficient to understand the accuracy of the data at a single time point since it can change over time.

We found that the proportion of high-agreement forest pixels across the seven RSD products used in our study and across time was higher in wetter forest ecosystems (mangroves and moist forests) than in drier forest ecosystems (dry and spiny forests) (Figure 4b). Past studies [41,42] have established that dry forests have characteristics that pose challenges in mapping them using RSD. These characteristics include open canopies, leading to mixed pixels consisting of woody vegetation, herbaceous vegetation, bare soil, and a high share of deciduous woody species, which can be difficult to separate from the soil background. This difficulty in mapping drier forests could mean a lower reliability of global RSD for forested area mapping in dry and spiny forest ecosystems compared to moist forests and mangroves. Our findings here suggest that end-users of these RSD forest products focusing on Madagascar and similar climes need to carefully consider the impact of using one product over another, as results could vary considerably across heterogeneous ecosystems.

As documented in Table A1, the different data products result from different sensor technologies (e.g., optical and SAR), varying spatial resolutions, and differences in the definition of the “tree” or “forest” class and the mapping methodologies. These differences can also ripple into some of the systematic disagreements that have been observed across different forest ecoregions. Given Madagascar’s unique and diverse forest ecoregions, there is a need for representative samples to be generated to ensure tree structure variants (e.g., open and closed canopies) can be fully mapped from satellite data.

The agreement maps we have generated can also be used for automatic label generation in forest or land cover classification focusing on Madagascar. To train machine learning models to classify land cover classes in Madagascar, obtaining high-quality labels can be very costly. By leveraging the agreement maps released as part of this study, high-agreement pixels in these data can be used as weak or noisy prior labels [43], significantly reducing the cost and effort involved in generating high-quality training labels. These prior labels can then be used to augment ground truth labels collected over the rest of the area. A similar approach has already been applied in Tanzania [44]. A study has shown that agreement maps can be used to improve accuracy of future land cover mapping efforts [45]. Furthermore, given the lower agreement between data products in drier forest ecosystems, ground truth labeling can prioritize drier ecosystems to improve land cover map quality. Also, in other similar climes (e.g., African countries) where high-quality labels are lacking and resources are limited, our workflow can be used to generate similar prior labels.

While the spatial distribution of forested- or deforested-pixel locations could differ across RSD products, this does not imply that the data cannot be used for studying the downstream effects of deforestation in Madagascar, especially in cases where unit-level aggregated trends, directionality or gradients, are of interest. Even though pixel-level comparisons may show differences, trends in aggregates can still be correlated. We found that, apart from the DW data, the rest of the RSD products showed high pairwise correlation and acceptable RMSD values (Figure 5). This highlights the potential usefulness of these data when analyzing relative differences across time points and districts, rather than relying solely on the absolute magnitude of forest loss within a single district.

This study serves as a precursor to ground truth data collection, validation, and local forest cover data creation for Madagascar. With the current absence of any form of validation for global RSD products in Madagascar, based on our temporal similarity analyses, a set of suggestions can be prescribed for studies that rely on aggregated area trends rather than pixel-level classification. The temporal coverage of the data is key to selecting an appropriate RSD for a downstream study. Given that MF-GFC covers 23 years (2000–2023), it provides the longest temporal coverage for longitudinal studies. Our analysis shows that the MF-GFC data exhibit good temporal similarity with other RSD products, except DW, for both forested and deforested area measures. That said, it would be useful to conduct sensitivity analyses with another dataset with shorter time coverage including PALSAR, FROM-GLC, ESRI, and ESA to check the reliability of the estimates. Future studies could extend this temporal similarity analysis to explore variations across different forest types.

4.2. Practical Recommendations for Users of Existing Global Data Products in Madagascar

Pending interventions like ground truth data collection and/or development of local land cover and change datasets for monitoring forest and other land cover classes in Madagascar, studies continue to rely on the global land cover data products which have been compared in this manuscript. As a result, it is critical to provide practical recommendations based on our findings to support the user ecosystem of these data in and around Madagascar.

Multiple studies focused on Madagascar apply forest-related metrics (area or loss) as either an environmental exposure variable or a control variable in regression analyses for research purposes. This class of studies seeks to provide answers to specific health, socioeconomic, and policy questions. For example, Devenish et al. (2022) [6] investigated the effectiveness of offset activities in mitigating deforestation driven by major mining activities in the Ambatovy mine in Madagascar. The outcome variable of the regression model specified in that study was the deforestation rate obtained from MFGFC data. Vieilledent et al. (2016) [8] also used MFGFC data to predict the climate-change-driven decrease in tropical forest carbon stocks in Madagascar.

To support the growing number of these kinds of studies, we make the following recommendations:

• Robustness checks using multiple RSD products: Studies should incorporate sensitivity analysis to assess the consistency of regression estimates across multiple global datasets. Given that we find acceptable spatiotemporal correlation between FROM-GLC, ESRI, MFGFC, PALSAR, and ESA, we recommend these products for such robustness checks. The data and code provided with this publication offer users the tools needed to efficiently extract and integrate these datasets into their study. These checks could help discover potential bias in research results driven by the choice of RSD for forest metric measurement.
• Robustness checks across forest ecoregions: In response to findings from our forest extent agreement analysis showing that there is a significant decrease in data agreement in dry ecoregions (dry and spiny forests), regression estimate consistency should be tested for robustness to this effect. This can be achieved by performing sensitivity analyses that stratify the regression models by ecoregion (see Figure 1A) and/or including interaction terms or region-specific coefficients that help assess whether the estimated relationships hold consistently across different ecoregions. While the ecoregions may not directly and mechanistically affect the user’s outcome of interest, they could indirectly influence it by contributing to variations in measurement error levels in the RSD.

5. Conclusions

This study provides a comparison of multiple global remote sensing datasets to estimate forest extent and deforestation across Madagascar. By comparing multiple RSD products, we generated forest agreement maps, helping to identify consistent forest-classified areas and highlighting discrepancies among different datasets. These maps are valuable for various applications including automatic label generation for forest or land cover classification, which can significantly reduce the cost of obtaining high-quality training labels for machine learning models.

Notably, we found that agreement levels vary across time and forest ecoregion dimensions. In comparison to wetter forest ecoregions (moist forest and mangroves), drier forest ecoregions (dry and spiny forests) show lower proportions of high-agreement forest pixels, suggesting moist and mangrove forests are more easily and consistently classified across RSD products than drier ecosystems like dry and spiny forests. Since most monitoring efforts and remote sensing-based downstream analyses around forests in Madagascar rely on global data products, these results have significant implications. Such implications include the need for local land cover data and a more cautious interpretation of forest cover maps in drier ecoregions.

In our temporal correlation analysis, we find that all except one RSD product exhibit high pairwise correlation values with other data. This suggests that, despite pixel-level disagreements, downstream application of forest metrics in Madagascar might still be useful when the signal of interest for users lies in the temporal dimension. We recommend that future studies in Madagascar utilizing global RSD should incorporate robustness checks that test for bias in the study estimates driven by the choice of RSD. This approach will enhance the reliability and validity of their findings.

We have developed and presented a workflow for multitemporal forest map agreement analysis that is reproducible and can be applied to any part of the world, making it a valuable tool for global forest monitoring and environmental impact assessments. In addition, we have released our derived forest agreement map for Madagascar for the years 2016 to 2020. These data can serve the community of scientists interested in developing a more accurate forest cover map in Madagascar based on remote sensing data. Our practical recommendations to users of these data in Madagascar also provide the principles of ways to apply these data to mitigate or discover some of the potential bias in downstream application results.

The temporal and spatial agreement data and metrics presented in this study contribute significantly to improving the utilization of RSD for forest monitoring and related impact assessments in Madagascar. Future studies using global RSD for local applications in Madagascar should consider the temporal dynamics of RSD agreement and conduct sensitivity analyses across different ecosystems to enhance the reliability of their findings.