Evaluation of the Simard et al. 2011 Global Canopy Height Map in Boreal Forests

Abstract

Light detection and ranging (LiDAR) provides a state-of-the-art technique for measuring forest canopy height. Nevertheless, it may miss some forests due to its spatial separation of individual spots. A number of efforts have been made to overcome the limitation of global LiDAR datasets to generate wall-to-wall canopy height products, among which a global satellite product produced by Simard et al. (2011) (henceforth, the Simard-map) has been the most widely applied. However, the accuracy of the Simard-map is uncertain in boreal forests, which play important roles in the terrestrial carbon cycle and are encountering more extensive climate changes than the global average. In this letter, we evaluated the Simard-map in boreal forests through a literature review of field canopy height. Our comparison shows that the Simard-map yielded a significant correlation with the field canopy height (R² = 0.68 and p < 0.001). However, remarkable biases were observed with the root mean square error (RMSE), regression slope, and intercept of 6.88 m, 0.448, and 10.429, respectively. Interestingly, we found that the evaluation results showed an identical trend with a validation of moderate-resolution imaging spectroradiometer (MODIS) tree-cover product (MOD44B) in boreal forests, which was used as a crucial input data set for generating the Simard-map. That is, both the Simard-map and MOD44B yielded an overestimation (underestimation) in the lower (upper) tails of the scatterplots between the field and satellite data sets. This indicates that the MOD44B product is the likely source of error for the estimation biases of the Simard-map. Finally, a field calibration was performed to improve the Simard-map in boreal forests by compensating for the estimation biases and discarding non-forest areas, which provided a more reliable canopy height product for future applications.

1. Introduction

Forest canopy height is regarded as one of the most important forest attributes, in terms of indicating forest biomass, carbon storage, species diversity, and other forest ecosystem functions [1,2,3]. Moreover, canopy height has also been used to facilitate the generation of global land-cover maps by separating the forests from other vegetation types in several widely used classification schemes, including the International Geosphere-Biosphere Programme (IGBP) and the Food and Agriculture Organization (FAO) [4,5].

Forest canopy height is traditionally measured through non-destructive methods in field surveys by using instruments such as a dendrometric table, Blume Leiss, and Bitterlich relascope. The rationale for measuring canopy height is based on the trigonometric relationships between the distance from the measurement point to a tree and the angle between the displacements from the measurement point to the top and base of the tree [6]. The field measurement of canopy height can be of high accuracy if clear visibility is available for both the top and the base of a target tree. However, it is usually very time-consuming, labor-intensive, and difficult to implement on a large scale over long-term periods.

In contrast, light detection and ranging (LiDAR) provides a state-of-the-art remote-sensing technique for measuring canopy height accurately and efficiently on a large scale [7,8]. As the first space-borne LiDAR system, the Geoscience Laser Altimeter System (GLAS) launched in 2003 has achieved unprecedented scientific successes in providing forest canopy height and biomass at regional-to-global scales in an operational manner [9,10]. Consequently, several successive space-borne LiDAR projects, such as the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) [11] and Global Ecosystem Dynamics Investigation (GEDI) [12] of the National Aeronautics and Space Administration (NASA) and the Multi-footprint Observation Lidar and Imager (MOLI) of the Japan Aerospace Exploration Agency (JAXA) [13], have been launched or are in the planning stage. However, space-borne LiDAR may miss some measurements of forest areas because of its spatial separation of individual spots. For example, the GLAS observations can only provide the footprints of 65 m in diameter spaced by 170 m along track and several tens of kilometers across tracks [14].

To overcome the limitation of the GLAS-based canopy height, many efforts have been made to generate wall-to-wall canopy-height products at a national-to-global scale in recent decades. For example, Lefsky [15] developed an empirical regression model to generate the first global forest canopy-height product based on a combination of GLAS and moderate-resolution imaging spectroradiometer (MODIS) data at 500 m spatial resolution. Simard et al. [14] produced the second global wall-to-wall canopy height product at 1 km resolution using a random forest algorithm based on the combination of GLAS data and seven ancillary variables of climatic and remote-sensing data. Los et al. [16] also generated a canopy height product between 60° S and 60° N with a coarse resolution (0.5° × 0.5°) using all the GLAS data collected from 2003 to 2009. Moreover, Wang et al. [17] generated the global distributions of mean forest canopy height at 500 m resolution using the GLAS data collected from 2005 to 2006 and 13 ancillary variables. Huang et al. [18] mapped vegetation heights in China using the slope correction GLAS data, Shuttle Radar Topography Mission (SRTM)-based, MODIS-based, and climatic data sets.

Among these wall-to-wall canopy height products, the one generated by Simard et al. [14] has been the most widely applied in many respects, possibly due to its easy availability and reliability in the existing validations. For example, the product has been used for estimating forest stand age in China [19,20], revealing the relationship between the spatial distribution of water availability and forest canopy height [21], and analyzing the aboveground carbon stocks of the global mangrove canopy [22]. However, it has also been pointed out that the canopy height product of Simard et al. [14] showed remarkable biases in various study areas, including the forests in China [18,19] and the tropical forests in central Africa [23]. More extensive evaluation efforts in different regions are needed to derive a more comprehensive assessment of this global canopy height product. This is helpful not only for improving the quality of this product but also for guaranteeing the reliability of related applications. Specifically, it is unclear how the product performs in boreal forests, which play a crucial role in global terrestrial carbon cycles and are encountering more extensive climate change than global averages [24].

Consequently, the objectives of this study were (1) to evaluate the performances of the global canopy product of Simard et al. [14] in boreal forests using the field measurements of tree height derived from literature surveys, and (2) to analyze the possible means to further improve the product in boreal forests.

2. Materials and Methods

2.1. Global Satellite Product of Canopy Height and Moderate-Resolution Imaging Spectroradiometer (MODIS) Data

We downloaded the global satellite product of canopy height in 2005 with a spatial resolution of 1 km developed by Simard et al. [14] (the Simard-map). The Simard-map was generated by a random forest-based algorithm using the combined datasets of the GLAS-based canopy height, climate and elevation data sets, and the MODIS percent tree-cover product in 2005 (i.e., MOD44B Collection 4 version) [25]. It provides an estimation of maximum canopy height within an area of 1 km × 1 km. The Simard-map yielded an acceptable accuracy with root mean square error (RMSE) and coefficient of determination (R²) of 6.1 m and 0.5, respectively, through the validation based on the in situ canopy heights measured at 66 FLUXNET sites mainly located in broadleaved deciduous and needleleaved evergreen forests.

We also downloaded the MODIS land-cover product (MCD12Q1) [26] and MODIS tree-cover product (MOD44B) [27] in 2005 from the Land Processes Distributed Active Archive Center (LP DAAC) for comparison analysis.

2.2. Collection of Field Canopy Heights from the Literature

We collected the field canopy heights measured in boreal forests (i.e., latitude higher than 40° N) from the literature to quantitatively evaluate the Simard-map. We conducted a literature survey based on online databases (e.g., the Institute for Scientific Information (ISI) Web of Science and Google Scholar) using the search terms “forest” and “tree/canopy height” with the filter for publication year set to later than 1995. Moreover, we also referred to the literature list provided by Iio et al. [28]. In total, we reviewed 448 literature resources (see Supplementary Materials for detail).

The minimum criteria for inclusion of canopy height data are as follows: (1) the latitude and longitude of the study plots were provided; (2) the investigation year and forest stand age were presented; and (3) the observed plots were nominally “homogenous” in the description of the literature. We extracted the following data: in situ measurements of canopy height, site location information (latitude and longitude), forest stand age, investigation year as well as other ancillary information including leaf area index (LAI), diameter at breast height (DBH), tree species, and tree density (see Table S1 in the Supplementary Materials). For sites with more than one measurement of canopy height, the maximum value was used for evaluation, to get the match-up with the estimation of maximum canopy of the Simard-map for any given 1 km × 1 km area. It is worth noting that the canopy heights in the reviewed literature were mainly measured by methods based on relascope, clinometer, flux tower observations, and DBH–height relationship derived from destructive measurements. These are all commonly used methods for measuring forest canopy height [6], which suggests that the assembled canopy height measurements are comparable.

2.3. Evaluation and Calibration Methods

Because the Simard-map of canopy height was generated for the year 2005, we first screened the field data measured in 2005. Correspondingly, there were only 17 sites retained for 2005. Therefore, we defined a forest age criterion to gather more applicable validation data as follows. Considering the asymptotic trend of canopy height increase to tree age [29], we applied the percentage ratio of the difference between the measurement year (Y_meas) and 2005 to the forest age (AGE) to define the selection criterion (SC):

$$SC=100\%\times \frac{\left|Y_{meas}-2005\right|}{AGE}$$

(1)

The sites with SC values lower than 5% were also collected and used as evaluation datasets. Through this, we finally obtained 88 observation sites in total (Figure 1 and Table S1).

The root mean square error (RMSE) was used in the accuracy assessment of the Simard-map:

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^N{\left(X_{field,i}-X_{product,i}\right)}^2}{N}}$$

(2)

where N is the number of field samples used for evaluation; X_field,i and X_product,i indicate the field measurements and the Simard-map of canopy height, respectively. The coefficient of determination (R²) between X_field,i and X_product,i was also computed.

Moreover, the derived linear regression relationship between X_field,i and X_product,i was applied to calibrate the Simard-map located in boreal forests (i.e., latitude higher than 40° N).

3. Results

3.1. Comparison between Field Measurements and Simard-Map of Canopy Height

Figure 2 shows the scatterplot between the field measurements of canopy height derived from literature surveys and the corresponding pixel values extracted from the Simard-map. It can be seen that the two data sets yielded a significant correlation with R² of 0.68 (p < 0.001). Although the RMSE (6.88 m) was comparable with the RMSE (~6.1 m) of the validations presented in Simard et al. [14], remarkable discrepancies were observed with the slope and intercept around 0.45 and 10.43, respectively. Specifically, the canopy height was noticeably overestimated when it was approximately lower than 10 m, and underestimated when it was approximately higher than 25 m (Figure 2). In addition to this study, the underestimation of the Simard-map was also observed in central African tropical forests when the canopy height was higher than 30 m (see Figure 12 in reference [23]).

3.2. Calibration of the Simard-Map in Boreal Forests

We carried out a simple calibration to the Simard-map in boreal forests using the relationship between the field and satellite canopy height:

$$H_{cal}=1.519\times H_{ori}-10.645$$

(3)

where H_ori and H_cal indicate the original and calibrated canopy height of the Simard-map, respectively. It should be noted that this calibration equation is different from the relationship as shown in Figure 2, because the independent variable was set as the satellite canopy height instead of the field measurement.

Figure 3 shows a comparison between the spatial distributions of the original and calibrated Simard-map of canopy height in boreal forests. It can be seen that some lower (higher) areas in the original Simard-map were decreased (increased) in the newly calibrated data. Moreover, the areas with the original canopy heights lower than 7 m were discarded because of the invalid calibrated values (i.e., with calibrated canopy height less than 0). We further checked the tree-cover values (i.e., MOD44B) and land-cover type (i.e., MCD12Q1) for the invalid calibration areas (Figure 4). It was found that these areas were generally with tree cover lower than 15%, and belonged to the land-cover types classified as crops, shrubs and savanna. That is, the discarded areas through the calibration process were highly prone to be non-forest areas. This indirectly verified the effectiveness of the simple calibration for the Simard-map based on field data derived from literature surveys. Nevertheless, this calibration is not applicable to the areas beyond boreal forests because it depends on the empirical relationship derived from the regression analysis based on field data sets.

4. Discussion

4.1. Consistency with the Performance of MOD44B Product in Boreal Forests

The MODIS tree-cover product (i.e., MOD44B) is a crucial input data set for generating the Simard-map. Therefore, we compared the consistency between our results and a validation study of the 2005 MOD44B product in boreal forests [30]. It was found that they presented an identical trend to that shown in Figure 2. That is, the tree cover was overestimated when it was approximately lower than 20%, and overestimated when it was approximately higher than 70% in the boreal forests (see Figure 8 in reference [30]). This indicates that the uncertainty of MOD44B in boreal forests is a likely reason for the discrepancies between the Simard-map and field measurements of canopy height presented in this study. However, improving the MOD44B is a daunting task because a large amount of training data of ground tree cover is required [30]. Therefore, we conducted a simple calibration of the Simard-map in boreal forests instead, as shown in Section 3.2.

4.2. Influences of Spatial Heterogeneity of the Validation Sites

Spatial heterogeneity of field sites is usually inevitable for validation of satellite products [31]. Removal of sites with high heterogeneity is often helpful for improving the validation results. For example, the accuracy of the Simard-map was increased (RMSE decreased from 6.1 m to 4.4 m) by excluding seven FLUXNET sites with land-cover heterogeneity, as presented in reference [14]. Unfortunately, we were unable to collect the field information on spatial heterogeneity of the validation sites utilized in this study.

Instead, we used the satellite data itself (i.e., the Simard-map) to analyze the possible influences of site heterogeneity on the evaluation of canopy height. At first, we calculated the standard deviation (SD) of the pixels of the Simard-map within a 3 × 3 window centered by each field site derived from the literature surveys. Then, we separated the validation sites into two groups: one was with the SD smaller than 2.0 m, the other was with the SD larger than 2.0 m. Finally, we compared the in situ canopy height with the Simard-map for these two groups, as shown in Figure 5. These two groups did not show significantly different performances in the evaluation of the Simard-map, both with R², slope, and intercepts around 0.69, 0.45, and 10.0, respectively. It indicated that the evaluation results of this study had not been biased by the spatial heterogeneity of the field sites. This might be attributed to the issue that the validation sites were relatively homogeneous. It might also be partially owing to the issue that the Simard-map estimated the maximum canopy height within a pixel, and we extracted the maximum value of the in situ canopy height if several measurements were available.

5. Conclusions

In this study, we evaluated a global wall-to-wall canopy height product (the Simard-map, which has been widely applied to various applications) in boreal forests with latitudes higher than 40° N. We collected the field measurements of canopy height in the boreal forests by reviewing 448 literature resources, and compared the derived field data with the Simard-map at 88 selected observation sites. Evaluation results demonstrated that the Simard-map yielded a significant correlation with field canopy height with R² = 0.68 and p < 0.001. However, remarkable biases were also observed with the RMSE, regression slope, and intercepts of 6.88 m, 0.448, and 10.429, respectively; that is, the Simard-map yielded overestimations and underestimations in the lower and taller boreal forests, respectively.

Interestingly, we found that the evaluation of the Simard-map in boreal forests showed an identical trend with a validation of MODIS tree-cover product (MOD44B) in the same areas, which was used as a crucial input data set for generating the Simard-map. That is, both the Simard-map and MOD44B yielded overestimation (underestimation) in the lower (upper) tails of the scatterplots between the field and satellite data sets. This indicates that the input of MOD44B data would be a crucial error source for the biases of the Simard-map in boreal forests.

Finally, we carried out a simple calibration on the Simard-map for boreal forests using the relationship between field and satellite measurement of canopy height. Correspondingly, the lower (higher) canopy heights were decreased (increased) through the calibration. Moreover, some non-forest areas were removed from the Simard-map in boreal forests. This calibration has the potential to provide a more reliable wall-to-wall canopy height product for boreal forests for future application studies. All the data sets generated in this study are available by request.