Point Cloud Inversion: A Novel Approach for the Localization of Trees in Forests from TLS Data

Abstract

Tree localization in point clouds of forest scenes is critical in the forest inventory. Most of the existing methods proposed for TLS forest data are based on model fitting or point-wise features which are time-consuming, sensitive to data incompleteness and complex tree structures. Furthermore, these methods often require lots of preprocessing such as ground filtering and noise removal. The fast and easy-to-use top-based methods that are widely applied in processing ALS point clouds are not applicable in localizing trees in TLS point clouds due to the data incompleteness and complex canopy structures. The objective of this study is to make the top-based methods applicable to TLS forest point clouds. To this end, a novel point cloud transformation is presented, which enhances the visual salience of tree instances and makes the top-based methods adapting to TLS forest scenes. The input for the proposed method is the raw point clouds and no other pre-processing steps are needed. The new method is tested on an international benchmark and the experimental results demonstrate its necessity and effectiveness. Finally, the proposed method has the potential to benefit other object localization tasks in different scenes based on detailed analysis and tests.

1. Introduction

Terrestrial Laser Scanning (TLS) has become popular in plot-level tree mapping due to its ability in penetrating canopies and acquiring 3D fine-grained structures of vegetation [1,2]. As tree instances form the basis of many applications such as stem mapping [3,4], tree volume and diameter at breast height (DBH) measurement [5,6], reconstruction [7,8], data registration [9], and biophysical parameter retrieval (e.g., leaf area index) [10], it is necessary to extract tree instances from point clouds in the preprocessing. In this letter, we focus on the tree localization which is a sub-step in the tree instance mapping, aiming at localizing trees in point clouds.

Most of the previous tree localization methods for TLS point clouds can be categorized into two classes: model-fitting methods and feature-based methods. The model-fitting methods treat the tree localization as the detection of stems from point clouds [11,12,13]. Maas et al. [14] divide tree point clouds into slices along the vertical direction and the circle fitting is applied to the sliced points, then model inliers of fitted circles will be recognized as stems. Similarly, Liang et al. [6] use a cylinder fitting strategy to find stems in point clouds. The feature-based methods utilize spatial and local geometric features of point clouds to identify stems [15,16]. In [17], trunk seeds are identified based on point distribution patterns in each slice. For example, the sizes of canopy slices are always much larger than stem slices. Tao et al. [18] use DBSCAN (Density-Based Spatial Clustering and Application with Noise) [19] to find stem clusters in sliced point clouds at DBH. Heinzel et al. [20] extract stem points from input TLS data using morphological operations that remove sparse structures like leaves and low vegetation. Stem points can also be recognized based on geometric features such as eigenvalue-based measurements. In [21], the eigenvalue-based geometric features are combined to classify photosynthetic and non-photosynthetic components; then, a group of predefined filters is introduced to remove noise and classification errors. Finally, stem points are grouped into instances based on the distance between non-photosynthetic points. As local features highly depend on the neighborhood sizes, Chen et al. [22] present an adaptive radius selection method based on the scanner-to-point distance, and the detected stem points are merged into tree objects by clustering methods.

A group of top-based tree localization methods is widely used in the processing of airborne and mobile LiDAR point clouds. In general, the top-based method assumes that objects in the real world are always higher than their surroundings, thus the potential objects can be localized by finding local height maxima in point clouds [23]. Li et al. [3] present a new tree segmentation method for airborne LiDAR point clouds where the highest points are treated as treetops. Lin et al. [24] assume that tree crowns can be modeled by circular cones, and propose an individual tree localization method based on multi-scale local maximums in mobile LiDAR point clouds. In [25], treetops in mobile LiDAR data are found by the maximums in the histograms of point numbers. Wang et al. [26] propose a voxel-based tree localization method for mobile LiDAR point clouds, which divides point clouds into 3D voxels, and voxels in the highest layer will be selected as potential treetops. If the distance between potential treetops is smaller than a given threshold, adjacent tree candidates will be grouped into one instance.

In summary, there exist several limitations in the aforementioned tree localization methods for LiDAR point clouds. First, the model-fitting methods may fail if stems are irregular or bifurcated, or point clouds are noisy and suffering from large data missing. Second, many existing methods such as slice-based ones always need the ground as the reference, while the ground modeling is still a challenging task in the complex forest scene. Third, methods that rely on geometric features often involve lots of specific rules or supervised learning processes that may need to be tuned from plots to plots. Last but not least, the top-based methods that are widely used in airborne and mobile LiDAR data processing can not be applied to TLS forest point clouds directly [27] because of the serious data missing in TLS forest point clouds and the complex canopy structures captured by the close scanning. Moreover, the previous top-based object detection methods can not handle overlapped objects.

In this letter, our main contribution is adapting the top-based tree localization method for TLS point clouds of forest scenes. The remainder of this paper is organized as follows: Section 2 elaborates the details of our proposed tree localization method. Section 3 presents the experimental results of the original top-based method and our new method on a TLS forest benchmark [28]. In Section 4, the new method together with compared ones are analyzed. Section 5 summarizes this work.

2. Methods

In the image processing domain, the image enhancement refers to a group of nonlinear refinement techniques which aims at making the images much easier to be interpreted by humans or processed by computers [29,30]. Analogously, to make the tree localization much easier, a new point cloud enhancement method named point cloud inversion (PCI) is proposed. The basic idea of PCI is applying a nonlinear transform to the original point clouds, aiming at enhancing the salience of tree stands in point clouds. The proposed nonlinear transform is inspired by the point cloud inversion operation [31] where point clouds are inverted before approximating the ground surface via the cloth simulation [32]. The detailed steps of the point cloud enhancement method is illustrated as follows.

The original point clouds are first organized into 3D voxels, each of which maintains one index list of points that fall into this voxel. Supposing a voxel can be indexed by its horizontal row, horizontal column and layer (vertical column) number (i,j,k) in the data management and there are m layers for the input scene, an empty index $E{I}_{i,j}$ for each vertical column indexed by (i,j) is calculated as

$$E{I}_{i,j}=\sum _{k=0}^{k=m}B(i,j,k),$$

(1)

where $B_{i,j,k}$ is a piecewise function which returns 0.0 if the voxel (i,j,k) contains points; otherwise, it returns 1.0. Then, point clouds are transformed according to the following equation:

$$p_{z_{new}}^t\leftarrow max\{Z_{max}-p_z^t-dV·E{I}_{i,j},0.0\},$$

(2)

where $p_t$ represents a point indexed by t, z is the z coordinate of a point, $Z_{max}$ is the maximal z of the input point clouds, and $dV$ is the voxel resolution. The $Z_{max}-p_z^t$ in Equation (2) stands for the inversion of point clouds, i.e., point clouds will be flipped after the inversion. The $dV·E{I}_{i,j}$ calculates the length of empty voxels in one vertical column. The updated z value will be large if the original point is close to the ground and its vertical column contains few empty voxels. Otherwise, the updated z value on the left in Equation (2) will be small. The objective of this transformation is enhancing the elevation of vertically connected objects (e.g., stems) while suppressing the elevation of other objects including branches, foliage, low vegetation, and the ground. The idea behinds Equation (2) can be further explained by the demo shown in Figure 1.

In this example, point clouds of one tree instance together with ground are colored by elevation. After dividing the input scene into voxels, i.e., grids in this demo, EI of each vertical column can be calculated. The EI values around tree tops will be small as $Z_{max}-p_z^t$ is close to zero. For root points, their EI values will be large as $Z_{max}-p_z^t$ is large and $dV·E{I}_{i,j}$ is small. Compared with stem points, the updated z values of branches (e.g., indicated by the arrow i) will be much smaller as there always be many empty voxels in their vertical columns. For noisy points, the updated z values are often small such as points highlighted by the arrow ii. As shown in Figure 1b, it is clear that non-stem points (e.g., branches, noise, and ground) are clustered at a much lower elevation than stem points which are visually enhanced after the transformation. Finally, the top-based tree localization strategy can be directly applied to localize trees in the enhanced point clouds. The flowchart of top-based tree localization method with the proposed PCI is shown in Figure 2.

3. Results

The proposed PCI is tested on the international benchmark for TLS-based forestry applications [28]. The TLS point clouds were collected in 2014 in the southern boreal forest, Finland. In this dataset, the main tree species are Scots pine, Norway spruce and silver, and downy birches. There are six released plots that are 32-by-32 m in size. They are classified into three groups based on the scene complexity (easy, medium, and difficult), and each group consists of two plots. Specifically, plots 1 and 2 are easy plots, plots 3 and 4 belong to medium plots, and plots 5 and 6 are difficult plots. In each plot, there are two kinds of datasets: one single-scan (SS) dataset and one multi-scan (MS) dataset merged from five single scans.

The newly proposed PCI and the original top-based tree localization methods are implemented in C++. The source codes will be released at https://github.com/GeoVectorMatrix/PointCloudInversion. The experiments are all conducted on a machine with one i7-6700HQ CPU and 16 GB RAM. The input for our method is original point clouds, and no pre-processing steps such as ground or noise removal are needed. The voxel size is set to 0.25 m based on the experimental tests and a 3 × 3 window is applied to retrieve local maximums from point clouds in the top-based tree localization step. The experiments are fully automatic with parameters fixed in these plots. The original and transformed point clouds of three represented plots from easy, medium, and difficult categories are shown in Figure 3. The localized trees are visualized by black vertical lines as shown in the enlarged view in Figure 3a. Figure 3b is dominated by red as trees in this plot are about the same height, and their canopies are closely adjacent. After applying PCI, the dominating color turns blue as canopy points are transformed to low points and tree-tops become spatially isolated. In Figure 3c, trees are at different heights, and it is challenging to distinguish canopy instances via visual inspection. With the help of PCI, the distance between tree instances is also increased as shown on the right in Figure 3c. Generally, by visually comparing the point clouds before and after the PCI process, it is clear that point clouds of stems are enhanced while other points are suppressed, enlarging the gaps between adjacent tree instances. Therefore, the top-based methods become much more applicable in localizing tree instances in point clouds. As shown in Figure 3, most tree instances can be localized by the top-based strategy which simply treats the detected local maximums as instance tops.

The completeness and correctness used in [28] are introduced for quantitative evaluations and comparisons:

$$Completeness=\frac{n_{match}}{n_{ref}},$$

(3)

$$Correctness=\frac{n_{match}}{n_{extr}},$$

(4)

where $n_{match}$ is the number of found trees, $n_{ref}$ is the number of reference trees, and $n_{extr}$ is the number of extracted trees. The coordinates of reference trees in all plots are also provided by [28]. The tree localization results with- and without PCI are shown in Figure 4. The objective of this comparative experiment is to demonstrate the necessity and effectiveness of the proposed PCI. The improvement by introducing PCI before the top-based tree localization is significant. In SS tests (Figure 4a,c), both completeness and correctness are almost doubled in nearly all the plots. The results of MS (Figure 4b,d) also show a considerable increase in both completeness and correctness. For instance, the completeness and correctness in easy plots are both lower than 0.4, which rise dramatically to 0.71 and 0.88 after applying PCI, respectively. In the MS datasets of complex scenes, the completeness is nearly trebled from 0.09 to 0.27 as shown in Figure 4b. Overall, this ablation study demonstrates that the newly proposed PCI is helpful in improving the salience of tree instances, making the top-based tree localization framework available in complex forest scenes.

The overall performance of the top-based tree localization method with PCI is also calculated and compared with the existing methods. Following the definition in [28], the balanced accuracy measure is defined as

$$Accuracy=\frac{2\ast n_{match}}{n_{ref}+n_{extr}}.$$

(5)

The mean accuracies of our results in all plots without- and with PCI are shown in Figure 5. A quick conclusion is that there is a dramatic increase in the accuracy by adapting PCI for the top-based method. Results of four baseline methods named TUDeflt, inraIGN, TreeMetrics, and NJU, which are reported in [28] are also drawn in Figure 5. The goal of this comparative experiment is to demonstrate the ability and potential of the top-based methods in localizing trees from TLS point clouds.

The TUDeflt uses a voxel-wise clustering method which merges point clouds into different instances. The inraIGN method builds ground reference based on the identified ground points; then, a horizontal slice of point clouds at DBH is extracted to find tree candidates. The TreeMetrics method also removes ground points first and tree candidates are detected by clustering of sliced points and model fitting. The NJU presents a supervised learning method that classifies original point clouds into scattering, linear, and surface types. After applying filters to remove branches and noise, stems can be identified from linear points. In most SS and MS datasets, the accuracy of the top-based method with PCI ranks the second among all six methods while the ranked-first method changes with the plot complexity and scanning modes. Specifically, the top-based method with PCI ranks first in the easy plot in SS datasets. As the original top-based strategy is not applicable in TLS forest point clouds, this improved performance is attributed to the proposed PCI.

4. Discussion

The experiments and results in Section 3 demonstrate the necessity, effectiveness, and stability of the proposed PCI. Although the top-based method relying on PCI can achieve comparable results with several baseline methods, it should be noticed that its performance is not as good as the top-ranked methods reported in [28]. However, the goal of our research is not achieving the highest accuracy in some specific benchmarks. The initial objective is introducing a new preprocessing framework which facilitates the data processing steps such as pole-like object detection. By introducing PCI, the top-based method which is rarely used in TLS forest data processing becomes applicable and can achieve competitive results.

A real example is given in Figure 6 to show the advantages of the PCI-aided top-based localization method for TLS forest scenes. There are two tree instances in Figure 6, the left one is tall and the right one is under its canopy. By using the original top-based method, multiple local tops can be identified due to their complex structures and noise. In addition, it is impossible to localize overlapped objects by the original top-based methods. As shown in Figure 6a, two vertical lines representing two tree top candidates are drawn, and the right dashed line indicates a wrong candidate. By applying PCI, the top-based method can find two correct tree instances as shown in Figure 6c. Another advantage of the PCI-aided top-based method lies in its low dependence on priors such as the accurate ground surface and geometric shapes of the stem. For example, the model fitting method can only find one correct circle in the upper-left corner in the sliced point clouds as shown in Figure 6b, making it impossible to localize the small tree. Moreover, both the PCI and the top-based method are fast and easy to be implemented, while the time-consuming data processing steps such as ground modeling and feature calculation are not involved.

The usage of the proposed PCI is not limited to the tree localization in forest. We apply PCI to TLS point clouds of a campus scene to find pole-like objects. In the upper row of Figure 7a, the open-access point clouds provided by [33] are colored by elevation, and the pole light among roadside trees is circled by a black rectangle. In the second row of Figure 7a, pole-like objects are localized by the top-based method aided by PCI. This test shows that the PCI is flexible and potential for localizing other objects in various point clouds. As shown in Figure 7a, many trees are surrounded by tree supports, increasing the difficulty of using model fitting or feature-based methods. In addition, due to the data incompleteness at DBH, the slice-based method may fail. As shown in Figure 7b, tree instances that are surrounded by tree supports or suffer from data missing at DBH are all visually enhanced by the PCI. Thus, the proposed method is robust against complex structures and missing data. Furthermore, the top-based method used in this letter is a simple implementation that can be improved by adaptive window sizes or combing other advanced algorithms [3,24,34]. Therefore, there is much room for improvement in the PCI-aided top-based methods.

5. Conclusions

In this letter, we propose a new point cloud enhancement method named PCI that makes the existing top-based object localization framework applicable in TLS forest scenes for the first time. The PCI is easy to implement and requires no ground reference or noise removal. Furthermore, PCI eliminates the need of the time-consuming feature calculation in other methods. In addition, the PCI can localize overlapped tree instances and other man-made pole-like objects in different scenes like the campus. The authors believe that the PCI also provides a new perspective for point cloud processing. For example, the existing object detection methods such as slice-based and model-fitting based ones may also benefit from the PCI. Our future work will focus on exploring potential applications based on the nonlinear transformations of point clouds.