4.1. ITC Deliniation
The delineation of individual tree crowns was performed in the CHM using the superpixel method. Since the main focus of the study is to investigate which combination of hyperspectral and LiDAR metrics better classify tree species and the superpixel method leads to over-segmentation, the over-segmentation was corrected in a semi-automated way. At first, small segments created by the superpixel algorithm were merged according to the predefined criteria (
Table 4). After this automatic step, the merged superpixels were checked, and the quality of the segmentation was improved using vector editing tools. This method ensured that all trees were correctly delineated, and no samples were left out. However, this worked for this study with a small sample of trees, but more robust methods of delineating tree crowns in tropical forests are needed.
It was shown that the superpixel approach was superior to the watershed algorithm for delineating tree crowns from the CHM at
Ponte Branca Forest remnant [
47]. A segmentation accuracy of 62% was achieved. However, at [
47], the presence of a SyRo palm tree whose crown is not circular was reported, making the automatic process challenging. It required smaller superpixels to distinguish palm tree crowns (star-shaped,
Figure 9), but it conferred over-segmentation to the other species within the tropical forest that have wider and more circular-like shape crowns. The same problem was found in our study, and the segments referring to the SyRo palm tree species had to be manually corrected in most cases.
In Brazilian Amazon forest, ref. [
103] tested some ITC delineation algorithms using the CHM from LiDAR available in
lidR package for R [
65]. The best result was obtained by the method developed by [
104], which is based on seeds and Voronoi tessellation, with an accuracy of 65%. The authors mentioned that raster CHM-based methods are ineffective to detect trees present in lower strata.
In another inland Atlantic Forest remnant in Brazil, ref. [
105] tested a new automatic method for delineating ITC using high-resolution multispectral satellite images. The method encompasses several steps, namely pre-processing, selection of forest pixels, enhancement and detection of pixels in the crown borders, correction of shade in large trees, and segmentation of the tree crowns. The accuracy of the method was 79%, showing it to be an effective method for large tree crowns; however, the method is ineffective in detecting trees in the understory and trees located in shadowed areas due to other trees or terrain shade.
All the authors cited above mention the difficulty of delineating ITCs in tropical forests due to the complexity and heterogeneity of forest formations and the difficulty of performing the segmentation of species in the lower strata, mainly because smaller trees are below the crowns of larger trees. According to [
75], a perfect ITC delineation in tropical forests is unrealistic. However, partial information that allows the delineation of dominant, rare, or invasive tree species that could be important ecological indicators is of great value for better understanding these complex ecosystems.
4.2. Tree Species Classification
In this study, we classified eight tree species in a Brazilian Atlantic Forest remnant using multisource remote sensing data. Three different datasets were used: hyperspectral images, PR LiDAR, and FWF LiDAR data. These data were used independently or combined to train and evaluate an RF tree species classifier. Many studies have addressed the classification of tree species in temperate and subtropical forests using spectral and/or geometric data (i.e., LiDAR) [
4,
6,
106,
107,
108,
109,
110,
111,
112,
113,
114], but few studies have been realized to classify tree species in Brazilian tropical forests, mainly due to the difficulty in access to these areas, difficulty in obtaining a sufficient number of samples of each species, great heterogeneity, and diversity of tree species in these forests. Thus, our discussion will be based on studies with similar applications for at least tropical and subtropical forests, whenever possible.
To classify eight emerging tree species in the
Ponte Branca Forest remnant (same study area), ref. [
35] used hyperspectral data from Rikola camera onboard of UAV, collected on three different dates, to understand whether multitemporal data can improve the classification. The variables used were normalized and non-normalized tree species spectra. The use of temporal spectral information improved classification performance for three of the eight analysed species. However, for the other species, a difference in environmental conditions between years influenced flowering and defoliation of the species even in the same season, thus altering the spectral response, as well as the time of image acquisition. The best result was obtained with the normalized spectra (OA of 50%). In our study, the OA was similar (55%) using the raw spectra (
Figure 11). The aforementioned authors reported a difference in spatial position between ITCs over the years, and some neighbouring trees interfere the spectral response of the tree species to be classified. In our case, there was no misalignment between the same ITC sample, but for the same species, there were samples on two different dates (2016 and 2017). Even though the data in different years were collected in the same season, there was a lag of one year and one month (
Table 3). Thus, the same species can present different physiological and phenological behaviour in different years, which may explain why the raw spectra and VIs were not effective in differentiating tree species in the
Ponte Branca Forest remnant. The most important features found by [
35] were the wavelengths, mainly from 628.73 nm to 780.49 nm (Band 10 to Band 23), which coincide with the bands that most contribute to the PC (
Figure 14) for the best classification scenario in our study. This makes sense as band configuration of the camera used in both studies were the same, and the species chosen for classification were similar, as well.
In another inland Atlantic Forest remnant in Brazil, [
29] classified eight forest species using airborne hyperspectral images obtained with the AisaEAGLE sensor in the VNIR spectrum (visible and near-infrared) and the AisaHAWK sensor in the SWIR spectrum (shortwave-infrared). Various combinations of wavelengths and VIs were tested. Species discrimination performed best using visible bands (mainly wavelengths located at 550 nm and 650 nm) and SWIR bands. Vegetation indices contributed positively to the classification when integrated with VNIR features and should be used if the sensor does not acquire data in the SWIR wavelengths. The PSRI vegetation index (see
Table 5) was one of the most important for tree species differentiation and, in our case, had the fourth highest relative importance (see
Figure 13) for the second-best classification scenario (S11). The best classification accuracy obtained by [
29] was 90.1%. This result was better than the result obtained by us, even for the best classification scenario (S13—OA of 76%). One of the reasons is the sample sufficiency. The forest remnant in [
29] study has a larger area and a better conservation state; consequently, more trees are present in the upper canopy. A total of 273 samples of eight species were selected while in our study, only 81 samples of eight species were selected. In addition, the Rikola camera does not collect data in the SWIR spectrum.
It was possible to verify that data from narrowband sensors (i.e., hyperspectral) are an important tool in the discrimination of tropical species since it is possible to obtain specific wavelengths and vegetation indices in parts of the spectrum where it is possible to differentiate species according to the spectral response. Even the results achieved by us using only spectral information were not satisfactory; wavelengths in the VIS, Red-edge, and NIR spectrum proved to be suitable to calculate most of the VIs and for classifying different tree species until certain level in our study.
It is possible with PR LiDAR data extract information that describe the structure and geometry of forests, and this information also has the potential to discriminate tree species, but their isolated use was not effective for classification in our study. The PR LiDAR metrics and their transformation by PCA showed the lowest classification accuracy and OA (33% and 36%, respectively). Michalowska and Rapiński (2021) [
6] commented that using only vertical structural features from PR LiDAR (i.e., height distribution) can decrease classification accuracy. Considering only the structure of the vegetation, it is not species-specific but more conditioned to the position on ecological succession (e.g., if the species is pioneer, secondary, or climax) or layer in the forest (e.g., understory, lower, medium, or upper stratum). Furthermore, tropical forests have multiple layers with smaller trees below the canopy, and therefore, PR LiDAR height distributions and pulse returns are ineffective for species separation in lower strata [
6,
47,
113] when used independently as input for tree species classification. In more complex forests, the spectral differences are usually more pronounced than structural differences when used independently [
110], which was confirmed in our study when analysing each of these features (hyperspectral images, PR LiDAR, and FWF LiDAR) separately.
While PR LiDAR metrics can decrease classification accuracy, the isolated use of FWF LiDAR metrics have great potential for classification of tree species, as the analysis of the complete waveform allows a better interpretation of the physical structure and geometric backscatter properties of the intercepted objects [
13,
17,
115,
116]. Some authors, such as [
19,
117], used metrics related to the number of waveform peaks, waveform distance, height of median energy, roughness, and return of waveform energy for tree species classification. Hollaus et al. (2009) [
118] used FWF LiDAR metrics related to echo height distributions, mean and standard deviation of echo widths, mean intensities, and backscatter cross-sections. In China’s subtropical forests, the OA was 68.6% for classification of six tree species [
117]. Reitberger et al. (2008) [
19] found an OA of 79% for the classification of leaf-on tree species and OA of 95% for the classification of leaf-off tree species in Bavarian Forest National Park in Germany, and in pre-Alps region of Austria, three species (beech, spruce and larch) were classified with an OA of 85% by [
118]. All these authors used metrics extracted from FWF LiDAR data. However, in our study, using only FWF LiDAR metrics to classify tree species was unsuccessful (OA of 36%). It is noteworthy that none of the cited studies were performed in complex tropical forests; in addition, the
DASOS software provided a different set of FWF metrics that related to the spatial distribution of voxels that contain or do not contain a waveform sample. It is worth noting that these metrics relating to the voxel distribution could also be extracted using point clouds, as each waveform sample is actually a point associated with an intensity.
When used isolated, spectral data from the Rikola camera and the geometric/structural data from LiDAR were not effective for classification (S1 to S5), LiDAR geometric data, especially when combined with radiometric data, intensity, and spectral data, provided valuable information for the classification of tree species in complex forests [
4,
6].
It is possible to notice that the metrics related to the voxels extracted from the FWF LiDAR data using
DASOS, and the VIs from the Rikola camera were one of the best combinations for the classification of the eight species of the
Ponte Branca Forest remnant, with an OA of 73%., an improvement of 18% comparing the classification performed with the spectra and VIs extracted from the Rikola camera, and 36% using the FWF LiDAR metrics alone. Buddenbaum et al. (2013) [
10], to classify two different forest species (Spruce and Douglas Fir) that were in different age classes, used 122 spectral bands of the HyMap hyperspectral sensor and normalized intensities of the waveforms that intercepted voxel columns with dimensions of 0.5 m. Combining these two data sources, the OA was 72.2%, improving classification accuracy by 16% when compared to using spectral bands only and 5.5% when using spectral bands and percentile heights of PR LiDAR data that was isolated. The FWF LiDAR metric related to the intensity of the voxels intercepted by the waveform samples proved to be an important metric in the differentiation of forest species, including in our study, in which it was important both for S11 and for S13 (
Figure 13). Liao et al. (2018) [
22] also confirmed an improvement in the accuracy of the classification of seven forest species in the western part of Belgium using different height percentiles of FWF LiDAR with hyperspectral images. The improvement was 7.7% compared to the classification using only hyperspectral bands and 24.8% compared to using only raster LiDAR FWF. It is worth mentioning that the metrics related to height percentiles are dependent on the point cloud density, which makes it difficult to compare with other study areas or different acquisition parameters of the point cloud. Thus, a benchmarking data to test algorithms across different acquisitions and study areas is necessary for understanding how the tree species classification performs using different types of LiDAR metrics [
119]. The
DASOS software normalizes the data during voxelization and deals with the irregular scan pattern, and the extracted metrics are not dependent on the point density [
72].
Although the FWF LiDAR metrics performed better when combined with the VIs in classifying tree species, the difference was small when we looked at the confidence intervals (
Figure 11) for the scenarios that combine both PR LiDAR and FWF LiDAR metrics with the information spectra and VIs (S8–S9 and S11 to S13). S13 includes all the metrics included in S11, and a small increase in classification accuracy was observed from including the extra metrics extracted from the PR data and the spectral data except for the scenarios, whose combinations were made with the PR LiDAR metrics transformed by PCA (S7 and S10).
As FWF LiDAR data require more computer memory than PR LiDAR data, the data processing is more time consuming and requires more computational effort, and few tools are available for processing and extracting information from FWF LiDAR point clouds [
72,
120]. In addition, due to the advancement of LiDAR systems, it is possible to obtain several returns for each emitted pulse. Thus, PR LiDAR metrics can be effective for tree species classification when FWF LiDAR data are not available and/or it is not possible to process them.
There are many tools and workflows for processing PR LiDAR point clouds, and several authors have already proven that the combined use of LiDAR metrics and spectral information from hyperspectral sensors (e.g., raw spectrum and VIs) improve the classification of tree species in different types of forest. Asner et al. (2008) [
121], for the classification of invasive tree species in Hawaiian tropical forests, found an improvement in accuracy from 63% to 91% using tree species spectra and LiDAR heights. Shen and Cao et al. (2017) [
122] found an improvement of approximately 6% in the classification of five species, and ref. [
110] found an increase in classification accuracy of 18 tree species of approximately 7% using both VIs and LiDAR metrics in Chinese subtropical forests. In our study area, improvements were 15% when combining raw spectra with LiDAR PR metrics and 9% when combining VIs with LiDAR PR metrics in classifying eight tree species. This corroborates the potential that the combination of hyperspectral and geometric data from LiDAR, mainly related to height percentiles and percentage of returns, have to improve the accuracy of tree species classifications, including in tropical forests.
Using all features extracted from different data sources for classification (i.e. spectra and VIs from images of the camera Rikola, PR, and FWF LiDAR metrics), we achieved a classification OA of 70% (Scenario 12), and transforming all features by PCA, the OA was 76% (Scenario 13), the best result obtained in this study. Using a large set of features as input data for classification, ref. [
110] combined VIs and PCA from hyperspectral images, textural information from RGB images, and structural metrics from LiDAR, totalling 64 features for the classification of tree species in Chinese subtropical forests. The OA was 91.8%, a better result than just using the isolated features or combining them two by two (e.g., LiDAR and VIs). For the classification of 12 tree species in a Brazilian subtropical forest, ref. [
79] tested several scenarios with different inputs, and one of the scenarios contained 68 features (e.g., raw spectra and VIs obtained from hyperspectral images, photogrammetric point cloud metrics, CHM, and textural information). The OA was 70.7% with a difference of less than 2% obtained for the best scenario that used raw spectra, VIs, and structural information from the photogrammetric point cloud as input.
When using many features as input data for classification with the RF algorithm, the features are randomly selected at each node of the tree [
123]. If any feature that does not contribute to species differentiation is selected, the classification performance may decline [
123]. The more features that do not contribute to the separation of tree species are added in the RF algorithm, the greater the probability of these features being selected at each node, increasing the algorithm generalization error in addition to generating very large trees [
123,
124]. Thus, a pre-selection of features that have the greatest potential to differentiate species can optimize and improve classification accuracy. However, as trees are complex structures, different features from different data sources (i.e., different remote sensors) can be complementary to improve the separability and classification of tree species. This can be seen in the results obtained in this study, as well as in other studies cited above, in which the use of many features did not decrease accuracy, but rather, it was similar to or even improved the classification accuracy when compared with a pre-selection or use of fewer features as input data for classifying tree species.
The transformation of the PR LiDAR metrics by the PCA and the use of these isolated features with the spectra or with the VIs were not very effective in the classification. However, when using all the features extracted from the hyperspectral images and the PR and FWF LiDAR point clouds transformed by the PCA, it resulted in the best classification accuracy (OA of 76%). According to [
22], the high dimensionality and redundancy when using hyperspectral data mainly makes it difficult to extract information from moving windows in raster data. Thus, the transformation by PCA facilitates the extraction of information using moving windows. In addition to decreasing the correlation between features that may be redundant, the PCA transformation can improve the classification accuracy when using small training set sizes [
125].