Identifying and Quantifying the Abundance of Economically Important Palms in Tropical Moist Forest Using UAV Imagery

Abstract

Sustainable management of non-timber forest products such as palm fruits is crucial for the long-term conservation of intact forest. A major limitation to expanding sustainable management of palms has been the need for precise information about the resources at scales of tens to hundreds of hectares, while typical ground-based surveys only sample small areas. In recent years, small unmanned aerial vehicles (UAVs) have become an important tool for mapping forest areas as they are cheap and easy to transport, and they provide high spatial resolution imagery of remote areas. We developed an object-based classification workflow for RGB UAV imagery which aims to identify and delineate palm tree crowns in the tropical rainforest by combining image processing and GIS functionalities using color and textural information in an integrative way to show one of the potential uses of UAVs in tropical forests. Ten permanent forest plots with 1170 reference palm trees were assessed from October to December 2017. The results indicate that palm tree crowns could be clearly identified and, in some cases, quantified following the workflow. The best results were obtained using the random forest classifier with an 85% overall accuracy and 0.82 kappa index.

1. Introduction

Palm trees are one of the most socially and economically important resources for local communities in Amazonia because they provide non-timber forest products like fruits, fabrics, fuel, and construction materials [1,2,3]. As a result, there is a high interest in improving the methods to evaluate the extent and health of the populations of palm species. This is particularly important in western Amazonia where the highest regional palm species richness is found [4], and particularly in the northern Peruvian Amazon, where palm fruit harvesting makes an important contribution to the local and regional economy [5].

The extensive palm swamps located in the northern Peruvian Amazon are ecologically and culturally important. These swamps are part of the largest known intact tropical peatland complex in the Amazon [6] and store a large amount of belowground carbon [7]. The forests are dominated by the Mauritia flexuosa palm tree (aguaje), but also host other species of arborescent palm species such as Oenocarpus bataua (ungurahui) and Euterpe precatoria (huasai) [3,4,8]. Fruit production of all these species and in particular Mauritia flexuosa sustain the fauna and local communities [6,8]. However, this ecosystem is threatened by the growing population and developing economies due to the high demand for palm fruits and the expansion of commercial agriculture, mining, and oil and timber extraction [6,9,10].

One way to keep the forest standing is through the promotion of the sustainable use of non-timber forest products [9]. In this region, this could be achieved by replacing the traditional methods of fruit harvesting of cutting the palms with climbing [11]. Several initiatives have promoted the sustainable management of palms in the region [6,8,11] but uptake of these projects is limited by the lack of information about the abundance of palms at large scales. Developing tools to assess the distribution and density of these palm trees, would help to estimate the total potential economic value of the forests and promote sustainable management [4,12].

Current studies employing remote sensing techniques have been useful to measure the extent of swamps and peatlands, but do not have the resolution to measure the abundance of palms at scales that are relevant to management. For example, the Peruvian Amazon Research Institute (IIAP, in Spanish acronyms) generated a classified LandSat TM mosaic showing different ecosystem types in the Loreto region, including a category for palm swamps, with 30 m resolution [13]. Later, Lähteenoja et al. [14] used this information as a base map to study the different wetlands in the region, finding five different peatland vegetation types. In 2014, Draper et al. [7] mapped the extent of the palm swamps in the northern Peruvian Amazon by combining optic (Landsat TM) and radar (ALOS Palsar and SRTM) satellite imagery. However, mapping the abundance and distribution of palm trees requires high spatial resolution imagery, which is difficult to obtain from satellites or airplanes due to cloud coverage, high costs [15,16], and the complexity of the tropical environments [17].

Small unmanned aerial vehicles (UAVs) offer the possibility to solve this issue using high spatial resolution imagery obtained in a relatively simple operation, in a cost-effective and safe manner [12,18,19,20]. UAVs have been exploited in precision agriculture, surveying different crop types in well-known and limited areas, estimating physiological status or yield production using object-based techniques [18,21,22,23]. Likewise, studies focused on mapping biodiversity are increasing over time [21], they are using object-based techniques to identify shrubs or trees in rangelands [20,24] and boreal forests [12,25]. However, the vegetation in these ecosystems is sparse and less diverse than in tropical forests [17,21].

Some emerging analytical techniques may be particularly useful for mapping tree biodiversity in moist tropical forests. Some options for object detection in high-resolution imagery are possible by combining machine learning and computer vision techniques such as object-based image analysis methodologies (OBIA) [26], bags of visual words [27], and deep learning techniques [28]. Among these approaches, OBIA has already been applied successfully in ecosystem types such as dry tropical forests and temperate forests using UAV imagery [26]. This approach provides a comprehensive understanding of the parameters used to detect the objects, it allows to work with more than the RGB bands of an image, and it does not require a lot of knowledge or training data to be implemented compared to other techniques.

Object-based techniques or object based image analysis methodologies (OBIA, or GEOBIA for geospatial object based image analysis) were developed due to the growing generation of high-resolution imagery that needed a faster and accurate classification [29]. Traditional classification methods (all pixel-based) are not suitable to process high-resolution imagery because they contain high spatial heterogeneity, requiring more computation power and more time in the post-classification processing due to the “salt and pepper effect” [17,29]. In this case, OBIA speeds up and improves the classification accuracies of vegetation mapping [21,30] by considering the spatial patterns for delineating features and extracting information from them before running a classification technique [29]. One way of performing the feature delineation is by grouping similar pixels into unique segments by meeting the criteria of dividing the image into relatively homogeneous and semantically significant groups of pixels [21,31]. In the case of vegetation, the segments can consist of crowns [23,32] or leaves [33]. Spatial patterns of the forest canopy can be captured by the texture of RGB images. Texture refers to particular frequencies of change in tones and their resulting spatial arrangements [34,35]. This allows identifying different objects where the features have the same color [35]. The texture extraction from the segments usually contributes to the improvement of the classification results [21,34,36].

The classification techniques that are used for tree species discrimination are constantly evolving in parallel with the statistical domain [37]. Currently, the most used classifiers are non-parametric decision trees such as support vector machines, random forest, nearest neighbors, and recursive partitioning [12,37,38]. These classifiers are generally chosen because they provide good classification results, do not need normally distributed input data, can work with categorical and numerical data, and are easy to apply in open source environments [36,37,38,39].

In this study, we test the use of small UAVs with RGB cameras to identify and quantify palm tree species in the tropical rainforest using the OBIA approach. This approach may be particularly useful when trying to identify palm trees due to their distinctive crowns in the rainforest. Since palms are monocots, their leaves are usually arranged at the top of an unbranched stem, with characteristic crowns that vary among genera [40]: M. flexuosa has a single large crown with rounded costapalmate leaves, E. precatoria has a star-shaped crown with orange-green pendulous long leaves, and O. bataua has a star-shaped crown with erect leaves [41]. We delineate palm tree crowns and identify them by using object-based classification in order to use spectral and textural information in an integrative way. Considering that these palm species are hyperdominant in the Amazon basin [42] and they are commonly used by locals [43], their identification on scales of tens to hundreds of hectares could help to support sustainable management of these resources.

2. Materials and Methods

2.1. Study Area

We surveyed ten 0.5 ha permanent forest plots (50 m × 100 m) that were already established in palm swamps dominated by Mauritia flexuosa, locally known as “aguajales”. Plots were established inside protected natural areas and in forests managed by local communities in the region of Loreto, in northeastern Peru (Figure 1). Each plot contains different densities of palm trees. The plots belong to the Amazon Forest Inventory Network (RAINFOR) and were established using a standard protocol [44]. Plot data are managed using the ForestPlots.net online database [45,46].

2.2. Data Collection

Within each permanent plot, for each tree, the relative location within the plot, species and diameter at breast height (DBH) were recorded. The height of all the palm trees and the height of 10 random trees per diameter class were measured in the plot following the RAINFOR protocol. Information of the plots is summarized in

Table 1. Characteristics of the surveyed plots.

Plot	Max. Canopy Height (m)	Mean Canopy Height (m)	No. Stems	No. Palm Tree Stems	No. Palm Tree Species	Palm Tree Species	Dominant species	% M. flexuosa Abundance	Palm Visibility * (%)
JEN-14	34.8	18.7	234	128	3	E. precatoria, S. exorrhiza, M. flexuosa	M. flexuosa	53.0	75.0
PIU-02	37.5	20.1	404	77	3	E. precatoria, Elaeis sp., M. flexuosa	M. flexuosa	17.3	77.9
PRN-01	37.9	19.6	310	199	6	E. precatoria, M. armata, A. murumuru, O. balickii, S. exorrhiza, M. flexuosa	M. flexuosa	35.2	67.3
QUI-01	29.1	15.75	398	204	3	E. precatoria, M. armata, M. flexuosa	Tabebuia insignis	22.1	58.3
SAM-01	34.7	19	251	123	4	E. precatoria, A. butyracea, S. exorrhiza, M. flexuosa	M. flexuosa	41.0	86.2
VEN-01	30.1	20.1	253	132	4	E. precatoria, M. flexuosa, O. mapora, S. exorrhiza	M. flexuosa	28.1	58.3
VEN-02	30.1	16.7	326	268	5	E. precatoria, M. flexuosa, M. armata, O. mapora, S. exorrhiza	M. flexuosa	56.4	57.5
VEN-03	30.1	15.38	254	196	3	E. precatoria, M. armata, M. flexuosa	M. flexuosa	70.9	75.5
VEN-04	32.3	12.9	270	221	2	M. flexuosa, M. armata	M. armata	47.8	77.4
VEN-05	28.1	16.6	248	124	2	M. flexuosa, M. armata	Ilex andarensis	32.7	84.7

* The palm visibility is a proxy to explain the number of tree species that receive overhead light in the plot. It is related to the palm density/openings in the plot that make the visibility of crowns possible in the RGB image. It is expressed as the percentage of the visible palm trees in the RGB UAV mosaic: PV(%) = Visible palm trees in the UAV mosaic of the plot*100/total palm trees recorded in the plot.

. The geographic location of the plots and the location of the palm trees were recorded with the handheld GPS Trimble Geo7X and the dual-frequency GNSS Trimble Tornado antenna, with an average error of approximately 5 m.

The whole area of each plot was surveyed using a DJI Phantom 4 Pro (PH4 Pro) UAV from October to December 2017. The PH4 Pro is a rotary-wing aircraft with four propellers (quadcopter). It has 30 min of endurance when the payload is lower than 200 g and 12 min endurance at 400 g. The ground control station and the UAV are radio-linked at 2.4 GHz frequency. Users can control the UAV in either manual or automatic mode via a remote control connected to a tablet. The PH4 Pro consists of a GNSS (GPS + GLONASS), compass, vision system and a 20 megapixel (MP) 1” CMOS RGB camera attached to the bottom with a gimbal. The camera can record high-resolution images or high-definition videos (4K) and provides a real-time connection between the camera and the tablet [47].

The missions were performed at 60 and 90 m above ground level (AGL). The forward overlap was 88% and the side overlap was 83%. Mission details are listed in Table A1.

2.3. Data Processing

The UAV image processing comprised the following 5 steps: (1) mosaicking, (2) segmentation of the mosaic, (3) training samples preparation, (4) classification, and (5) palm tree quantification. Figure 2 shows the workflow. The mosaicking (Step 1) was carried out using the commercial software program Pix4D mapper [48] in a PC of 3.5 GHz 14-core with 128 Gb of RAM and an NVIDIA Quadro M4000 graphics processing unit (GPU). The rest of the steps were performed using the open-source software R v. 3.4.4 [49] in a laptop of 2.6 GHz quad-core with 16 GB of RAM and an NVIDIA Quadro M1000M graphic card. All the scripts generated in R, are available on the Github repository (https://github.com/xime377/UAV-classification). To speed up the testing process, the HPC Manati cluster was used. It has a coordinator node (Intel Xeon E5-2680 v4 2.40Ghz of 56 cores with 64 GB of RAM) and 9 processing nodes (Intel Xeon E5-2680 v4 2.40Ghz of 28 cores with 64 GB of RAM per node), 6 of these nodes have an NVIDIA Tesla K80 graphic card.

Pix4D mapper uses the Structure from Motion (SfM) algorithm in combination with dense image matching to generate a digital surface model (DSM) and the orthomosaic [50,51]. Since vegetation is difficult to reconstruct due to its complex 3D structure (leaves, branches), data from different UAV missions were combined and different parameters were tested to obtain orthomosaics with as few artifacts as possible. Forty-nine orthomosaics were obtained in total, having on average five mosaics per permanent plot. For further analyses, only one mosaic per plot was selected, visually selecting the one that showed the least artifacts. In some cases, the orthomosaic was acquired in one mission, in other cases, the pictures from different flights over the same plot were combined. The characteristics of the ten selected orthomosaics are presented in

Table 2. Characteristics of the selected mosaics per RAINFOR permanent plot.

Plot	Flying Height AGL (m)	GSD (cm)	Area Covered (ha)	No. Images Used	2D Keypoints (median per image)	Reproj. Error (pix)	Point Density (points/m2)	Point Density	Interpolation Method
JEN-14	90	1.41	1.84	71	75,496	0.231	5,421,087	Optimal	IDW
PIU-02	90-65	1.9	5.36	191	71,505	0.180	84,420,906	Optimal	IDW
PRN-01	90	1.87	3.58	76	77,853	0.265	34,899,971	high/slow	IDW
QUI-01	90	2.09	5.09	94	75,794	0.239	2,729,608	high/slow	IDW
SAM-01	90-60	1.84	1.73	40	74,923	0.218	13,797,799	Optimal	IDW
VEN-01	90-65	1.28	1.96	73	74,312	0.216	7,362,904	Optimal	Triangulation
VEN-02	90-60	1.22	2.48	188	75,201	0.245	34,667,002	Optimal	IDW
VEN-03	90	2.06	9.27	168	74,250	0.218	5,292,017	Optimal	IDW
VEN-04	65	1.62	1.84	69	78,824	0.207	120,548,930	high/slow	IDW
VEN-05	90	2.06	3.49	60	76,969	0.205	31,389,942	Optimal	IDW

and some mosaics are shown in Figure 3.

In Pix4D mapper, the process is divided into 3 tasks. First, the initial processing aligns the images and selects key points from visual similarities between the overlapping images acquired with the RGB camera. The full keypoint image scale was selected and the automatic rematch was enabled. Second, a dense point cloud was generated using the multiscale option, selecting the 1/4 and 1/8 of the full image, and the optimal point density. Third, the DSM, the digital terrain model (DTM) and the mosaic were produced using the inverse distance weight (IDW) method and a sharp smoothing filter.

The segmentation (step 2 of the workflow) was performed in the software program R and GRASS GIS 7.2.2. The R package rgrass7 [52] allows the implementation of GRASS GIS functions in R and facilitates the data exchange between the two software packages. The function i.segment [31] with the region growing and merging algorithm was used to group similar pixels into unique segments. This segmentation algorithm sequentially examines all the current segments in the orthomosaic, merging neighboring segments if they are similar or close according to the distance formula.

Two parameters are required to set the size of the segments: a threshold value and a minimum segment size. The threshold value ranges from 0 to 1, lower threshold values would allow only identical valued pixels to merge, while higher values would allow the merging of close pixels even if they are not similar. The minsize parameter sets the minimum number of pixels in a segment, merging small segments with a similar neighbor. At higher spatial resolutions, it is recommended to have a higher minimum segment size, in order to have larger and fewer segments per orthomosaic, that captures at least one palm leaf per segment. Since the segmentation is based on the number of pixels, and the orthomosaics generated had a very high resolution (1.22–2.09 cm), the orthomosaics were re-scaled to a pixel size of 5 cm, keeping enough details of the palm trees and reducing the computation time. In general, with smaller pixel sizes, there will be more segments because the orthomosaic contains more details.

Different combinations of these parameters were tested visually in order to obtain segments that captured the shape of palm tree leaves (Figure 4). Overall, we found that mosaics with 5 cm pixel size, a minimum size of grouping pixels was 50, and a similarity threshold of 0.05 best delineated the palm leaves.

The training sample preparation (Step 3) consisted of combining the ground data with the UAV data. The ground data comprised the palm tree locations recorded with the GPS and the Forestplots database. These data were linked, obtaining a shapefile with the palm tree tag, species name, and the RAINFOR measurements.

The UAV data consisted of the RGB mosaic, the canopy height model (CH), the segmentation output from Step 2 (a vector and a raster layer per mosaic) and the textural layers.

The shapefile layer with the ground data was overlaid on the RGB mosaic using the open-source software Quantum GIS (QGIS) to check if the location points corresponded to the palm tree crowns in the mosaic. Misaligned reference palm trees were manually aligned with the crowns in the mosaic based on the relative coordinates provided in the Forestplots database or not considered in the classification if the corresponding palm tree was not clearly identified in the mosaic. The details of the palm tree species considered for the classification are shown in Figure A1. In order to constrain the classification for the identification of palm tree species, some ground data of soil, water, and other trees were included as extra classes to avoid that the classifiers would identify them as palm trees.

The canopy height (CH) model was obtained using the package raster [53] in R. The UAV DTM was subtracted to the DSM to generate the CH layer. The textural characteristics of each segment of the orthomosaic were calculated in R, integrated with GRASS GIS 7.2.2.

Table 3. Features used as predictors for the classification.

Predictor	Description
Canopy height model	Height above the ground (meters)
Area	Area size of each segment
Compactness	Compactness of each segment, calculated as: $Compactnes{s}_{segment}=\frac{perimeter lenght}{2 \times \sqrt{\pi \times area}}$
Fractal dimension	Fractal dimension of the boundary of each segment (Mandelbrot, 1982)
Mean RGB	Average of all the pixel values per segment per band
SD RGB	Standard deviation of all the pixel values per segment per band
Median RGB	Median of all the pixel values per segment per band
Max RGB	Maximum pixel value per segment per band
Min RGB	Minimum pixel value per segment per band
Mean of entropy RGB	Mean entropy values per segment (Haralick, 1979)
SD of entropy RGB	Standard deviation of the entropy values per segment (Haralick, 1979)
Mean of the Sum of Variance RGB	Mean of the sum of variance values per segment (Haralick, 1979)
SD of the Sum of Variance RGB	Standard deviation of the sum of variance values per segment (Haralick, 1979)

gives a description of the features obtained during this step.

The textural layers and the CH were stacked and overlaid with the shapefile containing the ground data, assigning the values to the corresponding training point. The ground truth points were split into training (2/3) and validation (1/3) dataset.

The classification (Step 4) was performed in R, using the CARET package [54], short for Classification and REgression Training. Four non-parametric decision tree algorithms were tested: (1) Kernel Nearest Neighbor (k-NN), (2) Recursive Partitioning (RP), (3) Random Forest (RF) and (4) Support Vector Machine Radial (SVMR). The default parameters of the software were used in the classifiers.

The k-NN is commonly used for single tree detection in temperate forests [12,39]. It uses the Euclidean distance from the observation to be classified to the nearest training sample observation and assigns it to that class [39,55]. The RP creates a decision tree that splits the observations several times into homogeneous groups based on several dichotomous independent variables. The process is recursive because each observation set is split an indefinite number of times until the splitting process terminates after a particular stopping criterion is reached [55]. The RF is one of the most frequently used classifiers in forest remote sensing of forests because it provides accurate results and it is less sensitive to overfitting compared to other decision trees [12,37]. It is an ensemble decision tree classifier that uses bootstrap aggregated sampling (bagging) to construct many individual decision trees, from which a final class assignment is determined [36]. The SVMR is widely used when working with tree species classification because it is robust to noise and high-dimensional data, comparably few training data is needed, and it splits the data using a non-linear approach [12,37], suitable when working with complex canopy vegetation.

To evaluate the algorithms, an ANOVA test and the post-hoc Tukey honest significant difference test were conducted.

The general performance of the different models in the classification was evaluated with the overall accuracy and the Cohen’s kappa coefficient.

The overall accuracy (OA) is the total number of correctly classified segments (true positives, tp), divided by the total number of samples (N_c):

$$OA=\frac{{{\displaystyle \sum }}_{class}^{i}tp}{N_c},$$

(1)

where i is the number of classes.

The Cohen’s Kappa coefficient (κ) is a statistic that measures the agreement of prediction with the true class. It takes into account the possibility of the agreement occurring by chance (expected accuracy, EA).

$$k=\frac{OA-EA}{1-EA},$$

(2)

$$EA={{\displaystyle \sum }}_{class}^i\frac{tp}{N_c}\times \frac{fp}{N_c},$$

(3)

where fp is the number of segments predicted as positive that were actually negative (false positives).

The species identification assessment consisted of the producer and user accuracy, and the corresponding confusion matrices. The producer’s accuracy (PA) is derived by dividing the number of correctly classified segments per class (tp) by the total number of segments corresponding to the ground truth of that class:

$$P{A}_{class}=\frac{t{p}_{class}}{N_{ground truth class}},$$

(4)

The user’s accuracy (UA) is the number of correctly classified segments (tp) in a class divided by the total number of segments that were classified in that class:

$$U{A}_{class}=\frac{t{p}_{class}}{N_{classified}},$$

(5)

After the classification step was conducted using all the predictors, the variable importance of the random forest models was analyzed to identify the layers that have the most predictive power in the classification per plot (feature selection). The variable importance was obtained from each plot model using the CARET package [54] in R. The layers that had repeatedly higher importance among all the plots were selected as the ones that contributed the most to palm identification.

The palm tree species quantification (Step 5) was performed in R, using the packages raster [53] and simple features [56]. This Step was mainly needed since the study area is a tropical forest dominated by Mauritia flexuosa palms and, in most cases, they were too close to each other, having many crowns in only one object. For this reason, this Step was conducted to split the crowns mask and obtain single crowns to be counted. This Step is not needed in areas with a lower density of palm species. The splitting process consisted of intersecting the five height classes from the K-means unsupervised classification of the canopy height model, with the classification crown mask. Finally, a table with the count of individuals per palm tree species was generated.

The accuracy of the palm tree quantification for each plot was determined by comparing the palm trees that were recorded during the ground data survey, and also the visible palms on the orthomosaics, with the palm trees detected by the classification algorithm. The precision, recall and F1 score were calculated per plot.

The precision is the number of polygons correctly classified (tp) as the corresponding palm species divided by the total number of polygons (N_q) generated in the quantification step:

$$Precision=\frac{tp}{N_q},$$

(6)

The recall is the number of polygons correctly classified (tp) as the corresponding palm species divided by the total number of ground truth data. In this case, we calculated the recall with the number of visible palms in the mosaic (Recall_visible) and the recall with the number of all the palm trees measured on the ground (Recall_ground):

$$Recal{l}_{visible}=\frac{tp}{No. of visible palms in the mosaic},$$

(7)

$$Recal{l}_{ground}=\frac{tp}{No. of palm trees\ge 10cm dbh on the ground},$$

(8)

The F1 score is the harmonic mean of recall and precision and thus it expresses the balance between recall and precision:

$$F1 score=2\times \frac{precision \times recall}{precision+recall},$$

(9)

The F1 score was calculated for both cases: using the Recall_visible and the Recall_ground. This evaluation index was used in this case instead of the overall accuracy because, unlike in the classification step where the number of segments per class was similar, the abundance of Mauritia flexuosa in most of the plots was higher than the abundance of the other palm species.

3. Results

3.1. Palm Species Identification

The palm tree species identification was performed using different classifiers. The Random Forest (RF) algorithm was the one with the highest overall accuracy (85% in average) and kappa coefficient (0.82 in average), closely followed by the Support Vector Machine Radial (SVMR) with 85% overall accuracy and Cohen’s Kappa coefficient of 0.81 on average. There was no statistically significant difference between the results obtained from RF and SVMR, but the results of those algorithms were significantly higher than those obtained from k-NN and Recursive Partitioning (p < 0.001, 95%). Table A2 presents the evaluation of the different methods used.

When examining the detection success per palm tree species, the identification of M. flexuosa was the most accurate since it was the dominant species in all plots. There was a general overestimation of the non-canopy dominant species (Astrocaryum murumuru and Oenocarpus spp.) and species with a small amount of training data (Socratea exorrhiza, Euterpe precatoria, and Attalea butyracea). The species that were not canopy dominant appeared more often with shadows, lacking enough good quality training data, and were misclassified as dark parts of trees or artifacts. In most of the cases, the classification was able to predict correctly Mauritia flexuosa, Mauritiella armata, and Euterpe precatoria even outside of the plots. Since there was no training/validation data for those areas, this was only verified visually. One of these cases is VEN-05 plot. Figure 5 shows the classification results of VEN-05 and PRN-01 RAINFOR plots.

On the other hand, plots with palm tree species with a very different crown shape, tend to obtain higher accuracy values. This can be seen when comparing QUI-01 and PIU-02, two plots with the same number of palm species and similar M. flexuosa abundance (Table 1). QUI-01 had a lower classification accuracy due to the presence of Mauritiella armata and less palm visibility (κ = 0.72), while PIU-02 composed just of M. flexuosa and E. precatoria, had a Cohen’s Kappa coefficient of 0.86.

More complexity is added when the plots have more palm tree species, as some of the crowns of the species look similar and there is only a small amount of training data for the non-dominant species. This is the case of PRN-01, where S. exorrhiza was classified as M. flexuosa or A. murumuru and viceversa; and Mauritia flexuosa and Mauritella armata were also misclassified (

Table 4. General confusion matrix of the Random Forest classification for all plots combined.

									Reference
Prediction	Trees	A. butyracea	E. precatoria	M. flexuosa	M. armata	A. murumuru	Oenocarpus spp.	S. exorrhiza	Water	Soil	Total
Trees	497	5	2	72	25	0	0	5	4	4	614
A. butyracea	7	50	0	12	0	0	0	0	0	1	70
E. precatoria	13	0	369	7	2	1	3	10	0	1	406
M. flexuosa	69	8	13	489	20	1	1	26	1	3	631
M. armata	39	0	3	36	348	0	3	9	0	1	439
A. murumuru	2	0	2	2	0	62	0	1	0	0	69
Oenocarpus spp.	3	0	11	4	0	0	190	1	0	0	209
S. exorrhiza	19	2	6	32	1	2	1	216	0	0	279
Water	3	1	0	1	0	0	0	0	253	4	262
Soil	2	1	2	5	0	0	0	0	6	667	683
Total	654	67	408	660	396	66	198	268	264	681	2702

).

3.2. Feature Selection

Five layers contributed the most to palm species identification: (1) the canopy height model, (2) compactness of the segments, (3) median of the green band values per segment, (4) mean of the sum of variance of the green band values per segment, and (5) the mean entropy of the red band values per segment. The impact on the species discrimination is shown in Figure 6; the accuracy assessment using these 5 predictors is shown in Table A3.

3.3. Palm Tree Species Quantification

The palm species quantification performed better in plots where the palm density was lower or the palms that were close to each other had different heights. The results of the quantification step using the output from the random forest classification were compared with the number of visible palms in the UAV orthomosaic obtaining on average a Recall of 71.6% and an F1 score of 0.65, and with all the geolocated palms in the RAINFOR plot (average Recall = 51.4%, F1 score = 0.65). The accuracy assessment per RAINFOR plot can be seen in Table A4. Figure 7 shows the comparison of the results obtained for the Mauritia flexuosa and Mauritiella armata species with the number of visible palms in the UAV orthomosaic (Figure 7a,b) and with all the geolocated palms in the RAINFOR plot (Figure 7c,d).

The number of palm tree stems captured by the UAV is lower than the ground data, capturing on average 69.8% of the palms present in the forest plots. This value strongly depends on the canopy density of the plot, varying from 58% in dense palm swamps, to 86% in less dense areas (Table A5). This occurs because the RGB camera only registers the top of the canopy. In Figure 7c,d, this “underestimation” of the understory palms can be seen. The points with similar values in the comparison with visible UAV palms (Figure 7a,b) and in the comparison with all the field measurements (Figure 7c,d), correspond to the plots with higher palm visibility. The quantification of these palms is more successful if there is higher palm visibility and if the palm species composition is more diverse or they are not too close to each other.

4. Discussion

This study is one of the first attempts to detect and quantify native palm species in a complex forest such as Amazonia. Most of the previous studies that allow detection and quantification using a segmentation step before conducting the classification have been performed either in plantations [28,57] or in open forests [32], areas where the studied features are easily discriminated due to the high contrast of them with the background. In all these studies, the presence of bare soil or very little presence of small plants in the background makes easier the detection and quantification process. In plantations, the systematic arrangement of the plants facilitates the detection more notably. The workflow here managed to identify and quantify the palm tree species from RGB UAV imagery in a tropical rainforest, where almost the whole mosaic consists of green vegetation. The approach was successful in detecting palm trees due to their distinctive crowns in the rainforest [40]. The assessment of the method using ten permanent plots with 1170 reference palm trees demonstrated that small UAVs with RGB cameras are useful for mapping this aspect of tree biodiversity in tropical forests.

4.1. Palm Tree Identification and Classification Results

The workflow developed here allows a semi-automatic classification using open-source software to detect different palm species in a complex environment. The segmentation process is one of the key steps for the correct identification of the palm species and it speeds up the processing times as mentioned in previous research [29]. It is recommended to try to get larger segments with higher similarity values to delineate the object of study properly. Since we are working with high-resolution images, it is important to consider that the smaller the pixel size, the more pixels per mosaic and thus more segments obtained [31]. This will influence the computational requirements and processing times.

In general, the accuracy of the classification was determined by plot species composition and vegetation density (expressed here as the palm visibility in the UAV mosaic). This is the case of QUI-01, VEN-04 and VEN-05, where the palm tree composition was characterized by Mauritia flexuosa and Mauritiella. armata. The main difference between these species is the size of leaves when looking from above (Figure A1), and thus, the misclassification occurred in both directions (Table 4). VEN-05 had a higher classification accuracy (κ = 0.84) due to the lower stem density and higher palm tree visibility in the plot, with more space between palm trees (Figure 5a).

Considering that the workflow uses a relatively small amount of training data since one palm tree could have more than one segment, it is important to have good-quality training data. This is related to data acquisition: the mosaics should have enough quality to be able to extract features from them, and the mosaic should contain enough features to train and validate the model. This can be seen in the lower classification accuracy in plots where there was a lack of good-quality training data, either due to illumination conditions as the case of E. precatoria, or where there were not enough palm trees per species as the case of A. butyracea (Table 4). This methodology would also not be suitable to study understory palm tree species since the RGB UAV imagery only captures the top of the canopy.

Since there were no radiometric corrections performed in the UAV images, illumination conditions and the solar elevation angle influenced the results of the classification. Missions conducted when there was a clear sky tended to have saturated images (like in the mosaics of PRN-01 and SAM-01), resulting in M. flexuosa and E. precatoria palms with white leaves. This caused some trees to be classified as M. flexuosa or vice versa when there were white branches similar to the border of the M. flexuosa palm leaves. In the plots with the presence of E. precatoria, the overestimation of the species occurred due to the similarity of the long leaves with some bright tree branches. Missions conducted when the solar elevation angle was low (<30°, such as in the mosaic of VEN-02 plot, mission details on Table A1), provided images with more shadows, having similar misclassification issues as mentioned before (trees classified as palms and viceversa). One way of reducing the illumination effect could be the use of ratios between the different color bands as a preprocessing step, as the normalized ratio between the red and green bands of the RGB mosaic [21].

As mentioned in Section 3.1, the accuracy of the classification is lower when the shape of the palm tree leaves are similar. During this study, some plots contained two species that look similar from above: M. armata and M. flexuosa. M. armata has smaller costapalmate leaves with a circular outline. The stems are clustered with short, stout, conical spines (usually not seen from the top of the canopy). M. flexuosa has costapalmate leaves with a circular outline deeply split in half, and each half is split into numerous leaflets and the tips of the leaflets tend to spread in different planes. Looking from the ground, it can be seen that this palm has a solitary stem [41]. In this classification method, the Canopy height model helped to discriminate them because M. flexuosa reaches greater heights. In addition, the different size of the leaves was reflected in the compactness of the segments, and the fact that the leaves have a different arrangement leads to different values of entropy for the two species. However, differentiating other groups of similar species may be challenging [58]. For example, Cecropia latiloba (a tree with somewhat similar leaf characteristics as M. armata in a UAV image) in some plots tends to be predicted as M. armata. A similar case could occur if the palms Socratea exorrhiza and Iriartea deltoidea are found in the same image, as from above, the crown shapes look similar [40]. To overcome this challenge, other types of technologies like hyperspectral imagery or Lidar techniques could be used in the future [58,59].

In terms of processing times, the most time-consuming operations were the texture extraction and value extraction from the raster. These tasks strongly depend on the size of the mosaic. They can be sped up by working only with the five suggested layers as mentioned in Section 3.2 or splitting the mosaic in tiles that are subsequently merged. Table A6 shows the computational time consumed by each step of the workflow.

4.2. Feature Selection

The canopy height model (CH) was the layer that contributed the most to the classification. It helped to discriminate lower areas like the soil and water, but it also contributed to discriminate canopy dominant species such as M. flexuosa and lower canopy species like Oenocarpus spp. The compactness layer, since it is related to the size and shape of the segments, helped to identify larger segments as tree crowns, elongated segments as E. precatoria and compact star-shaped segments as Oenocarpus spp. This layer is quite important for the robustness of the model because it helps to discriminate species by its shape when color layers do not provide much information due to changes in illumination conditions and a lack of radiometric corrections. A clear example can be seen in Figure 6, in the compactness of VEN-02 (blue frame), where some of the training samples were shaded but the segmentation step managed to capture the shape of these palms.

The median values of the green band contributed to discriminate Mauritiella armata and Mauritia flexuosa from the rest of palm species due to the intense green leaves, and in some cases, helped to distinguish Mauritiella armata (paler green) from M. flexuosa.

The sum of variance of the green band and the mean entropy values of the red band provided information about the complexity within the mosaic. Palms with leaves spread in different planes tend to have higher entropy values. These layers contributed to discriminate A. murumuru from E. precatoria (it has leaves in one plane) and M. flexuosa from S. exorrhiza (plumose appearance).

Even though different object parameters were calculated, only compactness was chosen as the layer contributing the most to the model (Section 3.2). This is because the different parameters are often correlated: for instance, both compactness and the fractal dimension (fd) are derived from the length of a segment. Compactness and fd have a negative correlation because while compactness indicates homogeneity, fractal dimension indicates heterogeneity [35]. Regarding the textural variables, the entropy and sum of variance work as an indication of the complexity within the mosaic [35], contributing to the discrimination of different palm species due to their different leaf arrangements. In this sense, the fact that only a few layers contribute to the classification accuracy makes the process easy to understand and to be replicated.

However, in most of the cases M. flexuosa, Mauritiella armata, A. butyracea, and S. exorrhiza were hard to differentiate: the first two, due to the similarity in the leave shape, and the latter due to the lack of enough good quality training data (images with strong shadows are prone to misclassification).

4.3. Palm Tree Quantification and Validation Data

The palm tree quantification works well for areas with low palm tree density or areas with a high diversity of species since the chances of finding palms close to each other in one area are lower. When the palm species were too close to each other and they had similar heights, obtaining individual crowns was not possible and there was an underestimation of the quantification. In other cases, the splitting step caused an overestimation of the palms due to the variation in height within a single crown (higher heights in the center of the crownshaft and lower heights around it).

The number of palms captured by the UAV imagery strongly depends on the canopy density, with more visible palms in less dense areas. This method is suitable for counting canopy dominant species since understory species could appear in the UAV mosaics, but they would be partially covered by the leaves of other species and separate parts of a visible crown may be counted as different individuals. This part of the workflow was designed as a quick alternative to split crowns that are grouped as one object after the classification step.

It is important to mention that a small part of the overestimation of palms in Figure 7a,b is also due to the detection of palms with a dbh lower than 10 cm, which are not geolocated, nor considered in the RAINFOR protocol. In plots with 57%–58% of palm visibility, there were no differences in the counting; however, in plots with 67% or higher palm visibility, palms with dbh lower than 10cm start appearing in the mosaics. Considering that the number of palms measured on the ground was 1676, the number of stems that were not georeferenced was less than 1%.

4.4. Considerations for Image Acquisition

Image capture needs to deal with variation in the illumination conditions due to cloudiness, blurry images or moving objects due to winds. These issues need to be considered for the mosaicking Step, especially if data from different missions are combined. Regarding the illumination conditions, since the fieldwork was conducted in the tropics, the suitable flying hours do not vary much throughout the year; however, variation during the day is crucial. The solar elevation angle influenced the results of the mosaicking and classification due to its influence on the presence of shadows (Figure 8a). As a result, it is recommended to fly when the solar elevation angle is higher than 30° (usually from 8:00–15:00 hours). Clouds also contribute to the presence of shadows, especially when there are the “popcorn” clouds. Overcast conditions provided more homogeneous mosaics, whereas clear sky conditions lead to saturated images. The result of combining missions with different illumination conditions (either different flying hours or presence of clouds) without a prior image normalization is a patchy mosaic (Figure 8b). For the aim of this research, radiometric calibration is not crucial since the classification is mainly based on geometrical patterns, compactness of the segments and textural information. However, the results could be improved if some brightness/shadow corrections could be conducted when the images are too dark to recognize similar pixels. Otherwise, those dark pixels will be recognized as ground by the SfM software and the segmentation step.

When combining missions when there was wind, the resulting mosaics are blurred or, in some cases, stitching was not even possible. Figure 8c shows part of a mosaic generated from one mission with strong wind, in which the palm trees are only partially visible. Even though the PH4 Pro is water-resistant [47], the pictures captured during rain events can also contain droplets of water. Even if the SfM software is able to generate a mosaic, some blurry areas may occur, or the whole mosaic may be blurry (Figure 8d), which prohibits proper segmentation and leads to poor classification results. For this reason, it is not recommended to acquire the images under poor weather conditions, like the ones described in this section.

Another important consideration is the flying height: most of the mission planning software takes into consideration the flying height above ground level (AGL), but the level of detail in a forest depends on the above canopy level (ACL). For this reason, it is necessary to know in advance the average canopy height and the maximum height of the trees. For this study, the best mosaics were obtained from the missions flying 60 m above the canopy, and, since the level of detail was extremely high, it may be possible to fly even higher (for instance 120 m AGL) in order to reduce some variation in leaves and to cover more area. With a spatial resolution of 5 cm, the level of detail was sufficient to distinguish the palm species.

4.5. Further Implications

The workflow developed here could also be applied to a different study object if it has a distinctive texture or shape and if the input parameters are tuned to do so. Moreover, this methodology could also be applied to high-resolution satellite imagery if the palm crowns are clearly visible. For example, the WorldView-3 satellite imagery (spatial resolution of 0.3 meters) can easily display the crowns of Mauritia flexuosa since the crown diameter of this palm species is on average 8 meters.

The segmentation step of this workflow could be helpful when working with deep learning and semantic classification. It could help to prepare the training data instead of delineating the features manually or using commercial software as was the case in [28] who used segments from UAV imagery for training a CNN to delineate citrus trees.

From an ecological point of view, small UAVs with only RGB cameras provide useful information in larger areas than the usually covered just by doing conventional ground-based surveys. Flying over a 0.5 ha plot took around 20 min including the assembling of the equipment and the UAV products can be used to derive different ecological indicators. The products obtained from the SfM software were the RGB mosaic, the digital surface model (DSM), and the digital terrain model (DTM). From the RGB mosaic, it is possible to visualize the emergent species and determine some illumination characteristics. If the mosaic is further processed as in this study, the combination with ground data makes possible the quantification of the canopy dominance per palm species by calculating the area of each of the species mask obtained from the classification. The gap area can also be quantified by calculating the area of the mask generated for the classes “soil” and “water”. The canopy height model can be derived from the DSM as was described in Section 2 and it could provide some information about the structure of the canopy.

5. Conclusions

A complete method for identification and quantification of the abundance of economically important palms using RGB UAV imagery in the rainforest was generated using open-source software. The process comprises mosaicking, segmentation of the mosaic, training samples preparation, classification, and palm tree quantification. Five descriptors were selected as the most useful for species discrimination, integrating canopy height with shape and textural characteristics. The workflow was tested in ten 0.5 ha permanent plots, using 1170 reference palm trees, providing good classification results. The method is suitable for areas where the palm density is medium to low, and the palms are not too close to each other. The use of small UAVs with RGB cameras, in combination with field data, has the potential to provide estimates of resource availability at relevant scales for tropical forest management, especially where cloud cover limits the use of satellite imagery, and the large areas and accessibility restrict ground-based surveys.