Consistency Analysis of Remote Sensing Land Cover Products in the Tropical Rainforest Climate Region: A Case Study of Indonesia

Abstract

Land cover changes in tropical rainforest climate zones play an important role in global climate change and the functioning of the Earth’s natural system. Existing research on the consistency of different land cover products has mainly focused on administrative divisions (continental or national scales). However, the ongoing production of large regional or global land cover products with higher resolutions requires us to have a better grasp of confusing land types and their geographical locations for different zoning (e.g., geographical zoning) in order to guide the optimization of strategies such as zoning and sample selection in automated land cover classification. Therefore, we selected the GlobeLand30-2010, GLC_FCS30-2015, and FROM_GLC2015 global land cover products with a 30-m resolution covering Indonesia, which has a tropical rainforest climate, as a case study, and then analyzed these products in terms of areal consistency, spatial consistency, and accuracy evaluation. The results revealed that (a) all three land cover products revealed that forest is the main land cover type in Indonesia. The area correlation coefficient of any two products is better than 0.89; (b) the areas that are completely consistent among the three products account for 58% of the total area of Indonesia, mainly distributed in the central and northern parts of Kalimantan and Papua, which are dominated by forest land types. The spatial consistency of the three products is low, however, due to the complex surface types and staggered distributions of grassland, shrub, cultivated land, artificial surface, and other land cover types in Java, eastern Sumatra, and the eastern, southern, and northwestern sections of Kalimantan, where the elevation is less than 200 m. Given these results, land cover producers should take heed of the classification accuracy of these areas; (c) the absolute accuracy evaluation demonstrated that the GLC_FCS30-2015 product has the highest overall accuracy (65.59%), followed by the overall accuracy of the GlobeLand30-2010 product (61.65%), while the FROM_GLC2015 exhibits the lowest overall accuracy (57.71%). The mapping accuracy of the three products is higher for forests and artificial surfaces. The cropland mapping accuracy of the GLC_FCS30-2015 product is higher than those of the other two products. The mapping accuracy of all products is low for grassland, shrubland, bareland, and wetland. The classification accuracy of these land cover types requires further improvement and cannot be used directly by land cover users when conducting relevant research in tropical rainforest climate zones, since the utilization of these products could lead to serious errors.

1. Introduction

Land cover classification and mapping is an important basic goal in global change research and provides a data source for many studies on global change [1,2,3]. The spatial distribution of land cover and its changes have certain effects on the material circulation, the dynamic balance of water and heat, and the structure and function of the ecosystem [4,5,6,7,8,9,10]. The traditional investigative method based on field surveys has been used for many years, requires a large amount of investment, and features limited precision. Recently, however, the acquisition of land cover information based on remote sensing has become an important means of quickly obtaining regional and global land cover information due to its ability to rapidly determine land surface cover distribution and dynamic change information in the region of interest [11,12,13,14,15,16].

At present, there are many sets of land cover products with different resolutions, such as the Global Land Cover Fine Surface Covering 30-2015 (GLC_FCS30-2015), produced by the Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences [17], the GlobeLand30-2010, produced by the National Geomatics Center of China [18], the Fine Resolution Observation and Monitoring of Global Land Cover (FROM_GLC), produced by Tsinghua University [19], the Moderate-resolution Imaging Spectroradiometer (MODIS) of Boston University [20], the GLC2000 of the European Union [21], and others. The emergence of these remote sensing products provides basic data for industry and academia that can be utilized to perform relevant production research [22]. At the same time, it should not be ignored that different products may use different classification systems, classification methods, and satellite images. Hence, the resulting land cover products themselves, as well as the series of follow-up studies derived from these land cover products, will vary [23,24]. The reason for this variety is that these remote sensing land cover products lack consistent benchmarks, and land cover users do not understand the spatial accuracy characteristics of these products in different regions and the advantages and disadvantages of specific field applications.

Driven by the comprehensive promotion of practical demand and theoretical research, scholars have conducted comparative analyses of multi-source remote sensing land cover products on regional and global scales [25,26,27]. At the regional scale, Song et al. [28,29] studied the classification accuracy and spatial distribution of different land cover products in China, concluding that these products had obvious errors and serious confusion in some local areas. The research of Kuenzer et al. [30] in the Mekong River Basin revealed that there is an obvious confusion phenomenon in the staggered distribution areas of different land cover types. Liang et al. [31] assessed the accuracy and consistency of the four global land cover products in the Arctic region in terms of land cover type distribution, spatial superposition, and validation samples. Their results showed that the overall accuracy of the Climate Change Initiative Land Cover (CCI-LC) 2000 is highest in the Arctic region, at 63.5%, while the overall accuracy of the MODIS data is lowest, at only 29.5%. Kang et al. [32] introduced the landscape index to evaluate the consistency of different land cover products in 2010, concluding that although the area ratio of land use types for these three products was more consistent in northern Laos, the spatial pattern characteristics of the different land use types were significantly different at a 30-m scale. On a global scale, Giri et al. [33] studied the consistency between Global Land Cover (GLC) 2000 and MODIS land cover products, finding that these products have higher overall consistency but lower consistency for finer land cover types. Herold et al. [34] analyzed the consistency of four global land cover datasets with a resolution of 1 km. Their results showed that the land cover types of evergreen broadleaf forest, snow, and bare land are more consistent, and the accuracy of producers and users is higher. The above studies demonstrate the importance and value of analyzing the consistency of multi-source remote sensing land cover products from different aspects.

However, most of these studies focused on administrative divisions (continental or national scales). In fact, there are some types of surface cover under different zoning (e.g., geographical or ecological zoning) with poor accuracy due to the cognitive standards and complexity of the spatial patterns of ground objects. The current production of land cover products with a high resolution of 30 m in large regions or globally requires a better grasp of the confusing land cover types and their geographical locations in different zones in order to guide the optimization of strategies such as zoning and sample selection during automated land cover classification. Hua et al. [35] performed consistency studies on different land cover data at global and continental scales through climate and altitude zoning. Their results revealed that the spatial consistency of Europe is high, at 66.57%, and the overall consistency of the frost climate is as high as 95%. Research on the consistency of different land cover products in tropical rainforest climate zones is lacking, however, even though land cover changes in tropical rainforest climate zones will impact the greenhouse effect, energy balance, and water transport, thereby affecting climate change on a regional or even global scale. Therefore, land cover and its changes in tropical rainforest climate zones have important implications for global climate change. In addition, existing data sources used in the consistency study of different land cover products focus mainly on products with lower spatial resolution. However, low-resolution products, such as MODIS, CCI-LC2000, and GLC2000 have some limitations when carrying out in-depth research on natural geography, the ecological environment, and global change. Hence, there is an urgent need to evaluate and analyze the consistency of current higher-resolution land cover products. Indonesia is located around the equator and is the largest archipelagic country in the world. Its territory spans Asia and Oceania, and it plays an important role in global strategy. Due to its typical tropical rainforest climate, Indonesia is rich in resources, and its land cover types are complex and diverse. The distributions of population, agriculture, forestry, mining, and other resources on each of its islands are unique.

Therefore, in order to make up for the shortcomings of the existing research, this study used Indonesia, with its typical tropical rainforest climate characteristics, as the research area, and the global land cover products GlobeLand30-2010, FROM-GLC2015, and GLC_FCS30-2015 as the data sources to carry out the analysis of areal consistency, spatial consistency, relative accuracy, and absolute accuracy evaluation methods. The study results can provide valuable reference material for research in the field of global ecological environment change and climate change.

2. Study Area and Data

2.1. Study Area

Indonesia (Figure 1) is located in Southeast Asia and is connected to countries such as Papua New Guinea, Timor-Leste, and Malaysia. It consists of 17,508 islands and is the largest archipelagic country in the world. The larger islands include Kalimantan, Sumatra, Sulawesi, and so on. The interior of the islands is mostly rugged mountains and hills, with only narrow plains along the coast. Kalimantan has mountains extending from its center to the west, with vast coastal plains and swampy southern regions. On Sumatra, the mountain range is oblique, extending from northwest to southeast, with hills and wider coastal alluvial plain in the northeast of mountain range. Sulawesi is mostly mountainous, with only narrow plains along the coast. Indonesia mainly has a tropical rainforest climate, and seasonal changes in precipitation due to monsoon. The annual rainfall in the plain areas ranges from 1780 to 3175 mm, while yearly totals in the mountain areas can reach 6100 mm.

2.2. Data and Preprocessing

We selected three datasets of higher-resolution global land cover products that are currently available free of charge for consistent research, i.e., the GlobeLand30-2010 (http://www.globallandcover.com/), FROM-GLC2015 (http://data.ess.tsinghua.edu.cn/), and GLC_FCS30-2015 (http://data.casearth.cn/sdo/detail/5d904b7a0887164a5c7fbfa0). There are differences among the data sources, classification systems, and classification methods used in the production of these products. The information regarding the main parameters of the three selected land cover products is listed in

Table 1. Main parameters of different products.

Name	Resolution (m)	Number of Categories	Time	Method	Overall Accuracy (%)	Production Institution	Sensor	Satellite	Area
GlobeLand30-2010	30	10	2010	POK (based on pixels, objects, and knowledge rules)	80.3	National Geomatics Center of China	TM/ETM+	Landsat/ HJ-1A/B	Global
FROM_GLC2015	30	26	2015	Random forest	77.3	Tsinghua University	TM/ETM+/ OLI	Landsat	Global
GLC_FCS30-2015	30	30	2015	Random forest	81.4	Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences	OLI	Landsat	Global

.

These products require preprocessing prior to accuracy evaluation and consistency analysis, including data clipping, projection conversion, and classification system merging. In this study, the Geographic Information System (GIS) software ArcGIS developed by the Environmental Systems Research Institute (ESRI) was used to preprocess the original data. We used Indonesia’s vector border cropping to obtain three land cover products encompassing the study area. The three original land cover products selected in this study are all in the same reference systems, and their geographic coordinate systems are all WGS84. In order to carry out subsequent accuracy evaluation and consistency analysis of different products, the coordinate system of three products is projected as World Mercator. Since the classification system types as well as the formulations of related standards were inconsistent among the three products (

Table 2. Original classification systems and codes for different land cover products.

Code	GlobeLand30-2010	Code	FROM_GLC2015	Code	GLC_FCS30-2015
10	Cropland	10	Cropland	10	Rainfed cropland
20	Forest	11	Rice paddy	11	Herbaceous cover
30	Grassland	12	Greenhouse	12	Tree or shrub cover (orchard)
40	Shrubland	13	Other	20	Irrigated cropland
50	Wetland	14	Orchard	50	Evergreen broadleaf forest
60	Water body	15	Bare farmland	60	Deciduous broadleaf forest
70	Tundra	20	Forest	61	Open deciduous broadleaf forest (0.15 < fc < 0.4)
80	Artificial surface	21	Broadleaf, leaf-on	62	Closed deciduous broadleaf forest (fc > 0.4)
90	Bareland	22	Broadleaf, leaf-off	70	Evergreen needleleaf forest
100	Permanent snow and ice	23	Needleleaf, leaf-on	71	Open evergreen needleleaf forest (0.15 < fc < 0.4)
		24	Needleleaf, leaf-off	72	Closed evergreen needleleaf forest (fc > 0.4)
		25	Mixed leaf, leaf-on	80	Deciduous needleleaf forest
		26	Mixed leaf, leaf-off	81	Open deciduous needleleaf forest (0.15 < fc < 0.4)
		30	Grassland	82	Closed deciduous needleleaf forest (fc > 0.4)
		31	Pasture	90	Mixed-leaf forest (broadleaf and needleleaf)
		32	Natural grassland	120	Shrubland
		33	Grassland, leaf-off	121	Evergreen shrubland
		40	Shrubland	122	Deciduous shrubland
		41	Shrubland, leaf-on	130	Grassland
		42	Shrubland, leaf-off	140	Lichens and mosses
		50	Wetland	150	Sparse vegetation (fc < 0.15)
		51	Marshland	152	Sparse shrubland (fc < 0.15)
		52	Mudflat	153	Sparse herbaceous (fc < 0.15)
		53	Marshland, leaf-off	180	Wetland
		60	Water	190	Impervious
		70	Tundra	200	Bare area
		71	Scrub and brush tundra	201	Consolidated bare area
		72	Herbaceous tundra	202	Unconsolidated bare area
		80	Impervious surface	210	Water body
		90	Bareland	220	Permanent ice and snow
		100	Snow/Ice	250	Filled value
		101	Snow
		102	Ice
		120	Cloud

), this study selected the GlobeLand30-2010 classification system as a reference to standardize the code of the FROM_GLC2015 and GLC_FCS30-2015 land cover types, ultimately forming a new classification system (

Table 3. Merged classification system and its correspondence with the original classification system.

Type	GlobeLand30-2010	FROM_GLC2015	GLC_FCS30-2015
10 Cropland	10	11,13,14,15	10,11,12,20
20 Forest	20	21,22,23,24,25	50,60,62,70,80
30 Grassland	30	32,33	130
40 Shrubland	40	41,42	120,121,122
50 Wetland	50	51,52,53	180
60 Water	60	60	210
80 Artificial surface	80	80	190
90 Bareland	90	90	150,200

). The spatial distributions of the three land cover products in Indonesia after preprocessing are shown in Figure 2.

3. Methods

3.1. Area Composition Similarity

The correlation coefficient of the land cover type area can quantitatively evaluate the similarity degree of the same land cover type between different data [36]. The formula is:

$$R_i=\frac{{{\displaystyle \sum }}_{i=1}^n\left(X_i-\overline{X}\right)\left(Y_i-\overline{Y}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(X_i-\overline{X}\right)}^2{{\displaystyle \sum }}_{i=1}^n{\left(Y_i-\overline{Y}\right)}^2}}$$

(1)

where $R_i$ is the area correlation coefficient of two land cover data, $i$ is the land cover type, $X_i$ is the total area of type $i$ in land cover dataset $X, Y_i$ is the total area of type $i$ in land cover dataset $Y, \overline{X}$ is the average of the total area of all types in land cover dataset $X, \overline{Y}$ is the average of the total area of all types in land cover dataset $Y, \mathrm{and} n$ is the total number of land cover types.

3.2. Spatial Pattern Consistency

In order to intuitively express the spatial pattern consistency of different land cover data, the spatial superposition method was employed to obtain the spatial correspondence of different land cover data pixel-by-pixel. Based on the number of matching coverage types of different land cover data determined pixel-by-pixel, the degree of consistency was classified into three levels, in descending order:

(1)

High consistency: the coverage types of the three land cover products are exactly the same at the same pixel;

(2)

Moderate consistency: any two products have the same coverage type at the same pixel;

(3)

Low consistency: the three land cover products all have different coverage types at the same pixel.

3.3. Consistency Distribution of Topographic Features

Topographic and geomorphic characteristics influence the precision of land cover products [37]. In this study, elevation data were introduced in order to analyze the consistency distribution of multi-source remote sensing land cover products at different elevations. The digital elevation model (DEM) data used in the study are Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model Version 2 (ASTER GDEM V2). The product is based on the calculation of ASTER data, and is the only high-resolution elevation image data covering the global land surface. We downloaded the 30 m resolution ASTER GDEM V2 data covering the study area from the Geospatial Data Cloud (http://www.gscloud.cn/). Based on the topographic and geomorphic characteristics of the study area, in order to provide the land cover types with obvious gradient characteristics in the elevation distribution, we divided the elevation into five grades (

Table 4. Elevation grades.

Grade	1	2	3	4	5
Elevation (m)	<200	200–500	500–1000	1000–2000	>2000

).

3.4. Precision Evaluation Based on the Confusion Matrix

3.4.1. Relative Precision Evaluation

The precision evaluation of land cover data based on the confusion matrix is the most common and important method used [38,39,40,41]. The precision evaluation indices obtained by the confusion matrix are producer accuracy (PA), user accuracy (UA), overall accuracy (OA), and kappa coefficient. The calculation formulas of each index are as follows [42]:

$$\mathrm{PA}=\frac{x_{ii}}{x_{+i}}\times 100\%$$

(2)

$$\mathrm{UA}=\frac{x_{ii}}{x_{i+}}\times 100\%$$

(3)

$$\mathrm{OA}=\frac{{{\displaystyle \sum }}_{i=1}^r{x}_{ii}}{n}\times 100\%$$

(4)

$$\mathrm{Kappa}=\frac{n·{{\displaystyle \sum }}_{i=1}^r{x}_{ii}-{{\displaystyle \sum }}_{i=1}^r\left(x_{i+}·x_{+i}\right)}{n^2-{{\displaystyle \sum }}_{i=1}^r\left(x_{i+}·x_{+i}\right)}$$

(5)

where $n$ is the total number of pixels, $r$ is the number of types, $x_{ii}$ is the number of pixels correctly classified, $x_{+i}$ is the number of pixels of a particular type in the reference data, and $x_{i+}$ is the number of pixels of a particular type in the data to be evaluated. We arbitrarily selected two of the three land cover products to calculate their confusion matrix and determine the relative accuracy evaluation results of the different products.

3.4.2. Absolute Precision Evaluation

In order to evaluate the absolute accuracy of the three land cover products, this study took the field survey samples from Indonesia in 2018 (Figure 3) as the reference data, and combined them using manual encryption in order to obtain the final accuracy evaluation verification samples. The field survey samples referenced are collected by GPS. The main steps include five parts: sample point selection, coordinate positioning, taking photos, information recording, and sample point correction. Sample point selection: the basic principles to be followed are that the collected sample points are typical and representative; the collected sample points should choose the larger type patches as far as possible; and the collected sample points try not to be close to the distribution boundary of the type to avoid the influence of the mixed type. Coordinate positioning: use GPS to locate geographical coordinates at selected points. Taking photos: take actual photos at each selected point and save these photos as original materials. Information recording: record the coordinates (latitude and longitude), type, and other related information of each sample point. Sample point correction: refer to high-resolution images (such as Google Earth) and other information to correct the coordinates of the sample points indoors. When obtaining verification samples, there is a certain time interval between the base year of the land cover products to be evaluated and the investigation time. In addition, the Google Earth high-resolution image repository was one of the main data sources for accuracy evaluation given its advantages of accurate positioning, abundant time phase, and wide coverage [43]. Therefore, we transform sample points into the Keyhole Markup Language format (KML) and imported into Google Earth, and then the sample points are interpreted online with Google Earth high-resolution images in 2010 and 2015. If there is no Google Earth historical image at a sample point during interpretation, the sample is discarded. In order to reduce the negative impact of positioning error and interpretation error on sample quality, the following principles were followed during sample selection and interpretation: (1) Since the positioning error of the high-resolution image samples of Google Earth is approximately 15 m, while the spatial resolution of the land cover products to be evaluated is 30 m, the center point of a 210 × 210 m homogenized area was selected as the sample in order to reduce the impact of positioning error [44]; (2) for samples that were difficult to interpret, reference was made to other auxiliary information, such as Geo-Wiki [45]; (3) the interpretation adopted the multi-person independent interpretation method, and samples were abandoned when the interpretation results could not be unified after negotiation. Based on the above principles, 2189 verification samples in 2010 (Figure 4a) and 2232 verification samples in 2015 (Figure 4b) were obtained.

4. Results and Analysis

4.1. Area Consistency Analysis

Figure 5 presents the area statistics of the three land cover products in Indonesia. This figure reveals that all of these products generated basically the same type of Indonesia land cover pattern, i.e., the distribution of forest land cover types in the study area predominates, while the cropland, grassland, shrubland, and other types have smaller distributions. The GlobeLand30-2010 and FROM_GLC2015 products produced highly consistent forest area coverage, with area proportions of 77.39% and 79.07%, respectively. The water and artificial surface types of the three products were highly consistent, with water area proportions of 1.31% (GlobeLand30-2010), 1.26% (FROM_GLC2015), and 0.71% (GLC_FCS30-2015), and the proportions of artificial surface area were 0.85% (GlobeLand30-2010), 1.13% (FROM_GLC2015), and 1.88% (GLC_FCS30-2015). The consistencies of cropland, grassland, shrubland, and other land cover types among the three products were relatively low. By calculating the correlation coefficient of area composition for any two of the three land cover products (

Table 5. Area correlation coefficients between different products.

Product	GlobeLand30-2010	FROM_GLC2015	GLC_FCS30-2015
GlobeLand30-2010	1.00	0.99	0.93
FROM_GLC2015	0.99	1.00	0.89
GLC_FCS30-2015	0.93	0.89	1.00

), the area composition correlation coefficient of the GlobeLand30-2010 and FROM_GLC2015 products was determined to be the highest, 0.99, while that of the GLC_FCS30-2015 and FROM_GLC2015 products was found to be the lowest, 0.89.

4.2. Consistency Analysis of Spatial Patterns

As the largest country in Southeast Asia, Indonesia is rich in natural resources due to its location in the contact zone between the Eurasian and Pacific plates, a region featuring frequent crustal movements and numerous volcanoes [46]. Figure 6 and Figure 7 illustrate the spatial consistency distributions and area statistics of several major cover types from the three land cover products in the study area. For cropland types (Figure 6a), the highly consistent regions of the three products were mainly distributed on northwestern Java, eastern Sumatra, and southwestern Sulawesi. The percentage of highly consistent area to the total area of the study area was 3.15% (Figure 7a). On eastern, western, and northwestern Kalimantan, central Sumatra, Java, and Nusa Tenggara, the consistency of cropland for the three products was low. For forest types (Figure 6b), all of the products exhibited high consistency on Papua, Maluku, central Sulawesi, and central and northern Kalimantan. From Figure 7b, the percentage of highly consistent area of forest to the total area of the study area was 54.21%. For grassland types (Figure 6c), the three land cover products differed greatly, with the percentage of highly consistent area to the total area of the study area only 0.004% (Figure 7c). The low consistency areas were distributed in the border areas of forest and cropland types on Java and other coastal areas. For the artificial surface type (Figure 6d), the consistency of the three products was found to be relatively low. The highly consistent area was mainly distributed in the Jakarta area of Java, and the consistency area accounted for 0.29% of the study area (Figure 7d).

Based on the consistency analysis of each land cover type, the consistency of all types for the three products in Indonesia was further analyzed (Figure 8 and Figure 9). The results revealed that the area ratios of the three land cover products for high-consistency areas, moderate-consistency areas, and low-consistency areas were 58.14%, 33.25%, and 8.61%, respectively. The high-consistency areas were mainly distributed on Papua Island, with forest land as the main cover type, and in the central and northern regions of Kalimantan island, with the proportions of the high-consistency areas on each island to the total study area totaling 18.05% and 17.05%, respectively. Meanwhile, low consistency of the three land cover products was found in the eastern, southern, and northwestern regions of Kalimantan, and the alluvial plains of eastern Sumatra and Java. The main cover types in these low-consistency areas were cropland, artificial surface, and grassland.

4.3. Spatial Consistency at Different Elevations

In order to analyze the influence of topographic and geomorphologic features on the precision of land cover products, the spatial consistency distribution results of the three land cover products obtained in Section 4.2 were superimposed with different elevations to obtain the spatial consistency distribution law under different elevations. Figure 10 shows the distribution law of low consistency for several major land cover types at different elevations. It was discovered that the low consistency of cropland, forest, grassland, and artificial surface types among the three products was mainly concentrated in the elevation range of <200 m. The proportions of the areas of low-consistency regions of each type to the total area of the corresponding low-consistency regions were 79.30%, 83.64%, 72.69%, and 85.25%, respectively. With reference to Figure 12, the consistency of each type of the three land cover products is low in the area with the elevation is <200 m which accounts for 61.19% of the total area of the study area. When the elevation is >200 m, however, the low-consistency area of each type decreases significantly, with the area ratios of the low-consistency areas <≈10%.

Figure 11 shows the distribution law of the overall low consistency area of the three products at different elevations. For all of Indonesia, the low-consistency area was mainly distributed at elevations < 200 m, accounting for 80.22% of the total low-consistency area. These areas were mainly located on Sumatra, Java, and Kalimantan. Therefore, as with the distribution law of low consistency of different types at different elevation levels, the overall consistency of the three land cover products was the lowest in the area with the elevation of <200 m which accounts for 61.19% (Figure 12) of the total area of the study area, and the land cover types varied greatly among the different products. The low consistency of the three products across the entire study area was significantly less when the elevation was >200 m, particularly when the elevation was >2000 m, where the low-consistency area only accounted for 1.50% of the total low-consistency area, and forest was the main land cover type.

From the above analysis, the low consistency of each type or total of the three land cover products decreases with the decrease of the percentage of elevation grade area. That is, the low consistency is highest in the range of elevation <200 m with the largest proportion of area, and decreases with the decrease of the percentage of elevation grade area.

4.4. Precision Comparison of Different Products

4.4.1. Relative Precision

Table 6. Confusion matrix between FROM_GLC2015 and GlobeLand30-2010 (FROM_GLC2015 provides the reference data).

	Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	63,765,335	82,319,844	6,118,721	1151	2,239,850	2,376,296	3,453,694	41,433	39.78
	20	145,564,881	1,373,581,262	49,231,240	5106	24,370,319	14,443,112	7,324,310	102,272	85.07
	30	44,575,022	102,117,491	16,319,969	12,836	1,889,055	2,687,942	2,468,208	65,208	9.59
	40	7,100,847	16,508,690	2,417,365	686	266,216	348,253	459,575	7977	0.00
	50	622,442	841,257	108,738	9	138,805	79,017	30,976	1070	7.62
	60	4,569,142	13,144,409	955,971	48	1,504,843	4,062,357	261,072	6504	16.58
	80	13,422,930	4,631,981	609,448	1936	110,888	375,785	3,470,854	3861	15.34
	90	65,905	1,705,584	42,124	0	9886	27,664	13,266	168	0.01
	PA (%)	22.80	86.13	21.53	3.15	0.46	16.65	19.85	0.07
	OA (%)	72.24
	Kappa	0.22

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

,

Table 7. Confusion matrix between FROM_GLC2015 and GLC_FCS30-2015 (FROM_GLC2015 provides the reference data).

	Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	136,155,874	17,841,006	7289	541,272	1,792,536	544,446	6,761,429	3,147,258	81.63
	20	393,567,421	1,186,613,850	116,801	2,788,981	56,464,435	975,728	7,536,838	22,571,822	71.03
	30	115,951,879	21,723,043	641,586	992,341	5,185,234	710,482	11,035,484	22,462,783	0.36
	40	16,181,077	6,198,069	19,559	125,207	1,183,178	44,034	1,082,450	3,192,584	0.45
	50	1,180,904	289,290	1120	18,980	293,114	92,955	41,846	74,183	14.71
	60	7,542,240	4,372,744	45,605	150,341	1,411,568	12,313,442	317,371	183,700	46.75
	80	10,098,892	372,187	27,295	25,016	95,181	87,961	12,771,650	610,762	53.02
	90	496,409	1,116,041	15,672	5873	12,790	11,733	227,407	53,583	2.76
	PA (%)	19.99	95.81	73.33	2.70	0.44	83.31	32.11	0.10
	OA (%)	64.28
	Kappa	0.29

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

and

Table 8. Confusion matrix between GLC_FCS30-2015 and GlobeLand30-2010 (GLC_FCS30-2015 provides the reference data).

	Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	191,298,294	400,714,223	33,572,497	18,719	7,580,681	8,267,857	10,263,784	144,523	29.35
	20	49,454,791	1,117,265,055	23,515,060	610	7,639,140	8,253,571	1,919,939	51,036	92.48
	30	3483	781,586	83,371	0	1712	3796	318	91	9.53
	40	320,146	3,428,270	552,875	0	95,032	73,608	10,913	2035	0.00
	50	3,959,528	29,880,664	6,070,136	14	13,438,485	2,595,123	195,365	15,380	23.93
	60	2,720,871	6,033,115	519,312	25	1,009,879	3,447,387	118,337	2199	24.89
	80	20,053,445	10,881,463	1,068,983	0	275,915	817,506	4,336,331	2532	11.58
	90	12,011,530	26,485,064	9,972,700	2642	395,965	827,048	639,151	10,876	0.02
	PA (%)	68.36	70.03	0.11	0.00	44.15	14.20	24.80	4.76
	OA (%)	65.74
	Kappa	0.29

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

are the confusion matrices between combinations of two of the three products, showing that the overall accuracy of any two of the three products was <73%, and the kappa coefficient was less than 0.30. The overall accuracy and kappa coefficient between the FROM_GLC2015 and GlobeLand30-2010 products were the highest, with respective values of 72.24% and 0.22, which is consistent with the conclusion regarding the area correlation coefficients of the different product types obtained in Section 4.1. The analysis of the various land cover types revealed that the consistency of the forest land cover type between the FROM_GLC2015 and GLC_FCS30-2015 products was the highest, and the mapping accuracy was 95.81%, while the mapping accuracy of the forest between the GLC_FCS30-2015 and GlobeLand30-2010 was the lowest, with a value of 70.03%. The consistency levels of the grassland and water types between the FROM_GLC2015 and GLC_FCS30-2015 products were high, with mapping accuracies of 73.33% and 83.31%, respectively. For cropland, the consistency between the GLC_FCS30-2015 and GlobeLand30-2010 products was high, and the mapping accuracy was 68.36%, but the user accuracy was only 29.35%.

4.4.2. Absolute Precision

The absolute accuracy of the three land cover products was evaluated using the sample points obtained by field investigation and artificial encryption. The results revealed that the overall accuracy and kappa coefficient of the GLC_FCS30-2015 product were the highest, with values of 65.59% and 0.55, respectively (

Table 9. GLC_FCS30-2015 confusion matrix.

	Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	475	96	120	72	12	126	1	71	48.82
	20	14	555	7	64	2	24	0	3	82.96
	30	0	0	7	0	0	0	0	4	63.64
	40	8	1	1	10	0	5	0	1	38.46
	50	9	14	0	0	33	16	0	2	44.59
	60	5	0	0	0	1	260	0	0	97.74
	80	24	0	1	0	0	0	106	13	73.61
	90	22	1	21	0	5	1	0	16	24.24
	PA (%)	85.28	83.21	4.46	6.85	62.26	60.19	99.07	14.55
	OA (%)	65.59
	Kappa	0.55

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

), followed by the GlobeLand30-2010 product, with overall accuracy and kappa coefficient values of 61.65% and 0.49, respectively (

Table 10. GlobeLand30-2010 confusion matrix.

	Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	335	16	42	60	2	18	17	15	66.34
	20	147	633	73	43	16	135	5	64	56.72
	30	35	12	36	6	19	22	0	25	23.23
	40	0	0	0	26	0	0	0	0	100.00
	50	5	4	0	0	7	14	0	1	22.58
	60	4	1	1	0	9	228	1	0	93.44
	80	14	0	2	7	0	2	84	2	75.68
	90	0	0	0	0	0	0	0	0	0.00
	PA (%)	62.04	95.05	23.38	18.31	13.21	54.42	78.50	0.00
	OA (%)	61.65
	Kappa	0.49

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

). The overall accuracy and kappa coefficient values of the FROM_GLC2015 product were the lowest, with respective values of 57.71% and 0.46 (

Table 11. FROM_GLC2015 confusion matrix.

Reference Data
Evaluation data	Type	10	20	30	40	50	60	80	90	UA (%)
	10	182	16	8	4	0	17	1	13	75.52
	20	156	642	73	127	29	64	4	9	58.15
	30	151	6	63	9	19	40	14	51	17.85
	40	6	0	7	6	3	1	0	2	24.00
	50	0	0	0	0	0	2	0	0	0.00
	60	10	2	1	0	5	296	1	14	89.97
	80	51	1	5	0	0	12	87	10	52.41
	90	0	0	0	0	0	0	0	12	100.00
	PA (%)	32.73	96.25	40.13	4.11	0.00	68.52	81.31	10.81
	OA (%)	57.71
	Kappa	0.46

Note: 10: Cropland; 20: Forest; 30: Grassland; 40: Shrubland; 50: Wetland; 60: Water; 80: Artificial surface; 90: Bareland.

). For the various land cover types, the mapping accuracies of the cropland, forest, and artificial surface types of the GLC_FCS30-2015 product were high, with levels ranging from 83.21% to 99.07%, while the mapping accuracies of the grassland, shrubland, and bareland types were <15%. For the FROM_GLC2015 and GlobeLand30-2010 products, the mapping accuracies of the forest and artificial surface types ranged from 78.50% to 96.25%, while the grassland and shrubland types were lower. In addition, the accuracy of the bareland for GlobeLand30 and the wetland for FROM_GLC is 0, indicating that these two types have not been correctly classified.

5. Discussion

Given that the tropical rainforest climate zone is the world’s largest biological gene bank, changes in its land cover not only reflect its surface natural environment, but also directly affect the global environment, especially human living conditions. Remote sensing mapping technology can now provide data support for related research. In this study, we performed a comprehensive evaluation and analysis of three freely available global land cover products, in general, there is low consistency among these products, possibly as a result of the following:

(1) Data sources. Differences in remote sensing images will affect the consistency between different products. The GlobeLand30-2010 product is classified using single Landsat TM/ETM remote sensing images combined with HJ-1A/B remote sensing images, the FROM_GLC2015 product is classified using only single Landsat TM/ETM/OL remote sensing images, while the GLC_FCS30-2015 product is classified using time series Landsat OLI remote sensing images. Our absolute accuracy evaluation revealed that the mapping accuracy of the land cover products obtained with time series Landsat imagery (Table 9) was higher than that obtained with single-period Landsat imagery (Table 10 and Table 11). At the same time, the fact that the complex cloudy and rainy weather conditions in the tropical rainforest climate area seriously impact the acquisition of high-quality remote sensing images cannot be ignored, which directly affects the accuracy of subsequent classification. Moreover, we found that the area composition correlation coefficient between the GlobeLand30-2010 and FROM_GLC2015 products (Table 5) as well as the relative accuracy were highest (Table 6). It shows that the inconsistency caused by the dynamic change of land cover is much smaller than that caused by different institutions, different data sources, and different classification methods.

(2) Classification systems. The global land cover classification system was established for global classification by taking full account of global land cover characteristics, which inevitably leads to the limitations of its application to unique geographical locations such as tropical rainforest climate zones [47]. The GlobeLand30-2010 product classification system only includes 10 first-class categories, while the classification systems of the FROM_GLC2015 and GLC_FCS30-2015 products are more refined. There are differences in the definitions of some land cover types among the classification systems of these products. For example, for the shrubs of various vegetation type in the tropical rainforest climate zone, the GlobeLand30-2010 product clearly defines land with shrub coverage > 30% and desert shrub coverage > 10% in desert areas as shrubs, while the GLC_FCS30-2015 and FROM_GLC2015 products only define evergreen shrubland and deciduous shrubland, and do not explicitly provide the vegetation coverage values. Differences in the definitions of these vegetation types resulted in low consistencies of the grassland, shrubland, and bareland types among the three products.

(3) Classification strategies and methods. There are some differences in the classification strategies and methods used in the three global land cover products, which will have an impact on consistency. The GlobeLand30-2010 product is constructed using a single-type classification, followed by an integrated classification strategy. The classification method based on “pixel-object-knowledge” is used to classify each type one-by-one, including pixel-based classification, object-based filtering, and human–computer interaction verification to fully utilize the advantages of various classification algorithms and make full use of knowledge and human experience in order to improve classification quality. In addition, this method effectively reduces the classification errors caused by the same objects having different spectra and different objects having the same spectrum [18]. However, the classification strategies and methods adopted by the GlobeLand30-2010 product not only create a large workload, but also require a great deal of human input. The FROM_GLC2015 and GLC_FCS30-2015 products are generated using random forest classification algorithms that are rated as the most robust in global land cover mapping [48,49]. The FROM_GLC2015 product first divides the Earth into 16 regions, and then DEM data, Slope data, Landsat images, and the first global multi-season sample set [50] are used for training classification. Moreover, nighttime light data are introduced to improve the accuracy of impervious types. The FROM_GLC2015 product, however, is extracted scene-by-scene and stitched scene-by-scene during the entire extraction process. The final classification results exhibit severe banding, which seriously affects the overall accuracy. The GLC_FCS30-2015 product proposes a global spatial-temporal spectral library (GSPECLib) using temporal MCD43A4 [20] reflectance products and CCI_LC2015 [51] surface cover products. When constructing the GSPECLib, CCI_LC coverage products are utilized to provide category information for the spectral features satisfying the conditions. Then, based on the geographic location of the GSPECLib, along with the time series Landsat data and the auxiliary terrain data integrated by the Google Earth Engine (GEE) cloud platform, the random forest multi-temporal classification model is trained region-by-region, and the global 30-m high-resolution surface cover classification results are obtained. The classification technology of the GLC_FCS30-2015 product obtains a continuous spatial distribution pattern of ground objects, effectively eliminating the banding problem in the single-period classification results and achieving higher overall accuracy than the other two products.

(4) Other factors. The consistency of different products is high for land cover types with obvious spectral and texture features (such as water and artificial surfaces). For the various vegetation types in the tropical rainforest climate zone, such as forest, shrubs, and grassland, due to the small differences in spectral and textural characteristics and the similar life forms, different types often display the same spectrum, making it more difficult for optical remote sensing to yield accurate classifications. Therefore, the consistency of these confusing types is low.

Analysis revealed that these global land cover products exhibit better mapping accuracy for the dominant forest types in tropical rainforest climate regions such as Indonesia. Users can select one of the products as their data source for relevant research. However, the special climatic and geographical features in the tropical rainforest climate region, such as cloudy and rainy weather, as well as the common occurrence of the same object having different spectra and different objects having the same spectrum, increase the difficulty of remote sensing interpretation. Therefore, for remote sensing interpretation of the tropical rainforest climate region, only relying on a single optical remote sensing image for classification leads to a very large error. It is therefore necessary to integrate multiple data sources (such as SAR and LiDAR data) in order to avoid interference from clouds and rain. Furthermore, the introduction of auxiliary data (such as DEM data) to prior regional divisions of tropical rainforest climate zones may help to improve classification accuracy.

6. Conclusions

In order to provide a reference for the selection of suitable land cover data required by many studies in tropical rainforest climate regions, this study performed precision evaluation and consistency analysis of three global land cover products, reaching the following conclusions: (1) the performances of the three products on Indonesia’s overall land cover type are basically the same. There is a strong area correlation between different products, with a correlation coefficient better than 0.89. (2) The overall spatial consistency of the three products is low, with the highly consistent area accounting for 58% of the total area of Indonesia. This high-consistency area is mainly distributed on Papua Island, as well as the central and northern parts of Kalimantan, with forest as the main land cover type. The low-consistency area is mainly distributed in the more complex areas with elevations < 200 m, such as eastern Sumatra, Java, and the eastern, southern, and northwestern parts of Kalimantan. The grassland, shrubland, forest, cropland, artificial surface, and other land cover types of these areas are staggered and affected by human activities. Therefore, attempts to improve classification accuracy should focus on these areas. (3) The absolute accuracy evaluation experiment of three products shows that the GLC_FCS30-2015 product displays the highest overall accuracy in Indonesia (65.59%), the GlobeLand30-2010 product exhibits the second-highest overall accuracy (61.65%), and the FROM_GLC2015 product has the lowest overall accuracy (57.71%). For forests and artificial surfaces, the classification accuracy of the three products is higher and land cover users can choose any of these products when conducting related research on these land cover types. The GLC_FCS30-2015 product exhibits better classification accuracy for cropland types, with its mapping accuracy in Indonesia reaching 85.28%. The mapping accuracies of the three products are low for grassland, shrubland, bareland, and wetland types, however, indicating that land cover mappers should focus on ways to improve the accuracy of these types. In general, the accuracy levels of these three global land cover products in Indonesia are not ideal, especially for grassland, shrubland, bareland, and wetland land cover types. Hence, these products are not suitable for land cover change studies in Indonesia.

With the development of remote sensing technology, different production agencies and organizations are constantly introducing new and higher-resolution land cover products. In view of the differences between these multi-source remote sensing land cover products, the use of data fusion technology to improve the accuracy of these products will be the main development trend of global land cover mapping in the future.