Next Article in Journal
Elevational Shifts in the Topographic Position of Polylepis Forest Stands in the Andes of Southern Peru
Previous Article in Journal
Growing Season Stem Water Status Assessment of Qinghai Spruce through the Sap Flow and Stem Radial Variations in the Qilian Mountains of China
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Feasibility of Google Tango and Kinect for Crowdsourcing Forestry Information

1
Finnish Geospatial Research Institute, Geodeetinrinne 2, FI-02430 Masala, Finland
2
School of Engineering, Aalto University, P.O. Box 14100, FI-00076 Aalto, Finland
*
Author to whom correspondence should be addressed.
Forests 2018, 9(1), 6; https://doi.org/10.3390/f9010006
Submission received: 20 November 2017 / Revised: 15 December 2017 / Accepted: 16 December 2017 / Published: 21 December 2017
(This article belongs to the Section Forest Inventory, Modeling and Remote Sensing)

Abstract

:
In this paper, we demonstrate the feasibility of using the Microsoft Kinect and Google Tango frame-based depth sensors for individual tree stem measurements and reconstruction for the purpose of forest inventory. Conventionally field reference data in forest inventory are collected at tree and sample plot level by means of manual measurements (e.g., a caliper), which are both labor-intensive and time-consuming. In this study, color (i.e., red, green and blue channels, RGB) and range images acquired by a Kinect and Tango systems were processed and used to extract tree diameter measurements for the individual tree stems. For this, 121 reference stem diameter measurements were made with tape and caliper. Kinect-derived tree diameters agreed with tape measurements to a 1.90 cm root-mean-square error (RMSE). The stem curve from the ground to the diameter at breast height agreed with a bias of 0.7 cm and random error of 0.8 cm with respect to the reference trunk. For Tango measurements, the obtained stem diameters matched those from tape measurement with an RMSE of 0.73 cm, having an average bias of 0.3 cm. As highly portable and inexpensive systems, both Kinect and Tango provide an easy way to collect tree stem diameter and stem curve information vital to forest inventory. These inexpensive instruments may in future compete with both terrestrial and mobile laser scanning or conventional fieldwork using calipers or tape. Accuracy is adequate for practical applications in forestry. Measurements made using Kinect and Tango type systems could also be applied in crowdsourcing context.

Graphical Abstract

1. Introduction

Mapping forest resources and their condition has great economic and ecological importance. Especially so, as trees contribute significantly to the carbon balance of the Earth. International interest in biomass detection is strongly linked to forest health, photosynthetic activity and other processes related to the carbon cycle and the variability of the climate [1]. There is a growing need and constant shortage of data for improved forest monitoring (e.g., [2]). Furthermore, forest inventory provides an input for forest operations, forest management, and related decision making. Retrieval of forest canopy and stand information for large areas has been mainly carried out using remote sensing, especially space-borne techniques [3,4], but has increasingly shifted towards airborne laser scanning (ALS) [5,6,7,8,9]. Forest inventories based on remote sensing data depend on the quality and quantity of field data collected on sample plots, which are used to calibrate remote sensing measurements, and to reduce the systematic (i.e., bias) and random errors of estimates. Reference data for sample plots is today mainly collected by manual measurements, although there is active research towards applying e.g., Terrestrial Laser Scanning (TLS), Mobile Laser Scanning (MLS) or Personal Laser Scanning (PLS) for plot level inventory. The attributes measured in operational forest field reference inventories are mainly the number of trees, tree species and Diameter at Breast Height (DBH). The latter is measured using simple tools, such as calliper and measuring tape. As these conventional field measurement techniques are expensive and labor-intensive [10,11], more cost-effective, automated techniques are needed. Even though the DBH is currently the most important attribute measured in ground surveys, it would be beneficial to measure the stem curve, or at least a part of it, as the stem curve actually defines the 3D geometry of the tree trunk. At this moment, the extraction of stem curve is one of the main research focus from TLS and MLS data, see [12] for an example.
Geographic crowdsourcing is collection of geospatial data by voluntary citizens, untrained in the disciplines of geography, cartography or related fields [13,14]. In forestry, crowdsourcing has been applied in assessing condition of city trees [15,16], even though some questions concerning the reliability of such has been raised [17]. By utilizing applications potentially useable in crowdsourcing scenarios, a basal area accuracy of 5 m2/ha [18], RMSE for basal area of 19.7–29.3% [19] and DBH root mean squared error of less than 7 cm [20] have been achieved.
Large-area forest inventory estimates can be calculated based on either area based approach (ABA) or individual tree detection (ITD) approaches using ALS data. In Finland, ALS itself is available as open data, but there is a lack of open plot- or tree-level data for calibrating the ALS estimates. While the ALS data can reliably produce e.g., tree height estimates for dominant trees, it suffers from not as good depiction of suppressed trees, and issues in identification of adjacent trees when located very close to each other. These errors can be compensated with plot level reference data. Today, this lack of local reference data causes the largest misestimation under boreal conditions. ITD techniques operate by detecting individual trees, with tree-level variables, such as height and volume, measured or predicted from the ALS data. In these analyses, the basic unit is an individual tree. International comparisons of the use of ALS for ITD have been reported in [21,22,23]. Advantage of the ITD approach over area-based approach (ABA) in crowdsourcing context is that it requires a smaller amount of reference trees, compared to that in ABA, for reasonable accuracy. It is also easier for the land-owners to measure physically well-established parameters, such as the diameter of the tree (cm) instead of the basal area (m2/ha). In addition, the common disadvantage of ITD approaches of not being able to detect all suppressed trees, that are poorly observable through the above canopy, can be partially overcome by the use of crowdsourcing, by including the missing major trees of ALS interpretation. Thus, there is synergy between the ITD and crowdsourcing.
Originally commercialized as an accessory for the Xbox (Microsoft Corporation, Redmond, WA, USA) 360 game console, Kinect (Version 1, 2010, Microsoft Corporation, Redmond, WA, USA) is a structured light system capable of capturing several millions of data points per second, with an affordable and easily applicable consumer-grade device. It incorporates color and near-infrared cameras, as well as a near-infrared laser projector. It projects a pseudo-random pattern on the target and uses the parallax information to calculate distance from the target. It is capable of producing 640 by 480 pixels depth images at 30 frames per second, with a maximum measurement distance of about 4 m.
Kinect has been applied in various applications, such as indoor mapping (e.g., [24,25,26,27,28]). A benefit of Kinect is that it efficiently brings 3D sensing capability to any conventional computer, thus allowing application development. The use of Kinect for tree measurements has been proposed for forestry robots [29,30], and plot level data collection [31]. With Kinect, individual tree diameters with 4.4 cm and 9.2 cm accuracies for deciduous and coniferous trees have been attained.
Depth sensors fully integrated to mobile devices have also emerged, with the Google Tango (2014, Google Inc., Menlo Park, CA, USA) technology [32] being available on two consumer devices. Tango is a development platform aiming to bring depth sensing to mobile devices. It consists of standardized application programming interfaces (APIs) to utilize depth sensor in Android software development. Tango framework allows the depth sensing to be accomplished by stereo vision, structured light, or time-of-flight principle [33]. As integrated systems, Tango enabled devices allow 3D sensing on a highly mobile platforms, utilizing sensor integration and Simultaneous Localization and Mapping (SLAM) for 3D measuring [34]. Devices with Tango sensors have in research been applied to indoor mapping [35], interior planning [36] and forestry [37].
While Tango and Kinect rely on similar operating principles, they offer different possibilities for application. Kinect is operated on a regular computer, and can thus be integrated to other tasks & software. Field tasks are possible when using a laptop or a tablet PC. Tango represents a fully integrated mobile system akin to a mobile phone camera, making it more suited for mobile tasks and crowdsourcing. Both of these consumer-level technologies could be applied for collecting objective, non-biased reference information, later applicable for remote sensing inventories. To give an example, there are more than 600,000 forest owners alone in Finland (12% of the population). Many of them visit their forests regularly or live in the nearby area. If 0.1% of these forest owners would measure 20 trees each, from their own forests, the resulting data set would consist 12,000 individually measured trees, producing an accurate, non-biased countrywide biomass/volume map of unprecedented accuracy.
Our objective is to firstly show the value of utilizing a local field reference for ITD forest inventory from ALS data by evaluating its impact to forest inventory estimate errors, and secondly, demonstrate the feasibility of Microsoft Kinect and Google Tango sensors for collecting stem geometry data on an individual tree level. Compared to working with a measuring tape, Microsoft Kinect or Google Tango represent efficient and potentially user-friendly ways to obtain diameter measurements and directly produce digital information. Finally, we test if these sensors could also provide stem curve information, potentially allowing an even higher accuracy of field data.

2. Materials and Methods

2.1. Test Area and Field Reference

Three test sites are utilized in the study, one (Masala) being used for Kinect and Tango studies, and two (Evo & Kalkkinen) for evaluating the impact of local field reference data.
The test area for the Kinect and Tango studies is located in Masala, Kirkkonummi, in southern Finland near the Finnish Geospatial Research Institute. As ground truth, reference data were measured with tape and caliper. The accuracy of diameter measurements using the calliper has been earlier analyzed to be about 0.7 cm by repeating measurements for a large number of trees and assuming the measurements are independent and of equal accuracy [38]. Since the tree stem cross-sections are often ellipses rather than perfect circular, it is conventional to measure the diameters in both north-south and east-west directions with calliper and then average the two measurements, calculating the square root of the average of squared values. We also used tape to record the circumference of each tree. Measurements were taken at different heights along the trunk; diameters were taken at lower part of the trunk (heights between 10 and 200 cm) to be able to estimate the lowest part of the stem curve. A polymer/fibre measuring tape was used to obtain the circumference measurements, after which the resulting circumferences were divided by Pi to calculate the diameter. In theory, the diameter for the rest of the trunk can be estimated using this stem curve information, which is a common practice among harvesters. The test data set constituted 41 and 80 diameter measurements from Kinect and Tango, respectively. For Kinect, both birches and scots pines were scanned. For Tango, the dataset consisted of pines. Kinect and Tango measurements were performed by authors having prior experience in 3D scanning methods.
In the local field reference test, two test sites (Evo and Kalkkinen) were used to show the value of utilizing a local field reference for ITD forest inventory from ALS data. Both of these areas lie in the Southern Finland, separated by a distance of approximate 34.5 km. Kalkkinen site consisted of mature forests, where no silvicultural operations had been carried out for a considerable amount of time. In Kalkkinen, the dominant tree species are Norway spruce and Scots pine. Evo site, on the other hand, is a commercially exploited forest area, with dominant tree species being Scots pine and Norway spruce. Data from field measurements by traditional methods (of accuracy equivalent to Kinect and Tango measurements) were used for demonstrating the potential of new techniques in improvement of forest attribute estimates. Table 1 shows the difference in forest attributes at plot level: Mean volume of Kalkkinen is almost double to that in Evo.

2.2. Kinect Measurements

When measuring trees with Kinect, the heights were marked with pins (see Figure 1) for every selected tree in order to guarantee that measurements were taken from the same height. The pins are not needed in real-life application, as they were used to ensure the correspondence between Kinect data and manual measuring methods. Images were acquired from marked heights. The stem curve was measured by moving the Kinect along the tree trunk and registering consecutive images together using tie points (Figure 1). The whole trunk can be reconstructed by merging the obtained diameter estimates at each height. In a real-life application, the Kinect scanning would be completed from a single position, holding the device manually and scanning along the tree trunk. For higher accuracy, the scanning could be performed from two positions.

2.3. Kinect Data Processing

The processing of Kinect RGB and range data was based on range data, by fitting a circle to the scan points around the buffer area. The marked positions were detected semi-automatically using standard image processing techniques. The steps in detail are given as follows:
  • Extraction of trunk skeleton from range image by thresholding along the direction of the camera’s depth axis, i.e., removal of the background to produce a trunk mask;
  • Automatic detection of the markers on the trunk from RGB images (first converted to grey scale) by comparing intensities, as markers are brighter than surroundings;
  • Extraction of the point cloud from range images in a buffer area of detected marker positions;
  • Fitting an optimum circle to the extracted point cloud and computing the diameter of the reconstructed circle at the corresponding height.
If the tree stem was imaged from the same height by more than one scan, the mean value of the diameter, obtained from the reconstructed circles was used. Figure 2 demonstrates one example of diameter estimation and the major steps in the determination of diameters using Kinect data.

2.4. Measurements with Tango Sensor

A mobile phone with Google Tango sensor (Phab2ProAR (Lenovo, Morrisville, NC, USA) [39]) was used to 3D scan a set of selected large pines from Masala, Kirkkonummi, Southern Finland. The device contains conventional smart phone hardware with the addition of a depth camera, allowing active 3D data acquisition. The primary purpose of integrating a 3D sensing system in a smartphone is augmented reality applications, but several 3D scanning software have also been developed for Tango devices.
The Matterport Scenes application [40] was applied in scanning the tree stems. It utilizes the Tango depth sensor in conjunction with device’s orientation sensors and RGB camera, producing a 3D RGB point cloud. Scanning was performed from motion, maintaining an approximate orientation of depth camera sensor facing the tree stem while walking around the stem in a circular motion. Each tree was scanned three times, producing a total of 240 diameter estimates of 80 manually measured stem diameters. In addition, tree stem scanning was also tested by scanning two adjacent stems in one go, as shown in Figure 3.
Red markers were placed on the tree stem in 10 cm height intervals, from ground surface to 1.5 m. For ground truth, circumference of the stem was measured from these heights with a measuring tape.

2.5. Tango Data Processing

Figure 4 shows a point cloud obtained with the mobile device. The terrain and other vegetation surrounding the tree stem were manually segmented using CloudCompare (Version 2.8.1, 2016). After this 20 mm high ‘slices’ of point cloud were separated from the marked heights, identifying the markers from the point clouds RGB data (Figure 4). 2D circular fitting was then performed to extract diameter information for stem from marked heights.

2.6. Accuracy Assessment

Determination of the accuracy of the Kinect- and Tango estimates was performed by comparing them against calliper and tape measurements (dtape,calliper). The coefficient of determination (R2) was calculated between the data sets to describe the goodness of fit between the obtained estimates and reference measurements. Bias (Equation (1)) and the root-mean-square error RMSE (Equation (2)) were calculated as the measures of the accuracy of estimates. RMSE-% was obtained from RMSE by dividing it with the sample mean value.
B i a s = Σ ( d k i n e c t , t a n g o d t a p e , c a l l i p e r ) n
R M S E = Σ ( d k i n e c t , t a n g o d t a p e , c a l l i p e r ) 2 n

2.7. Local Field Reference Test

In the local field reference test, the influence of plot level information (diameter data with accuracy comparable to Kinect or Tango) to forest inventory estimates was explored utilizing traditionally obtained single tree data to calibrate estimations from ALS. For both of the sites, ALS data was used as the basis of inventory: Evo test area has been surveyed with Leica ALS50-II (Leica Geosystems, Sankt Gallen, Aarau, Switzerland), 400 m AGL, 2009, and point density of 16 pts/m2 whereas Kalkkinen data was collected with Toposys Falcon, 400 m AGL, 2003, and the point density was 10 pts/m2. For calibration, field data (292 plots in Evo and 33 plots in Kalkkinen) were collected with conventional methods at the corresponding time of laser acquisition. More details of these test sites can be found from [41,42,43]. To simulate the impact of local field reference, Evo plots served as conventional plots (with every tree measured by conventional methods) while Kalkkinen plots were used as test plots. In order to test the effect of locally collected reference data on forest parameter estimation accuracy we used three different strategies:
(1)
All 292 plots of Evo data were applied as references to develop prediction models using an area-based method. Developed models were then used to make the estimation of plot attributes for 33 plots in Kalkkinen. This corresponds to a situation where the calibration data comes from a different area than the area being the subject of inventory.
(2)
Kalkkinen data were used for both developing models and making predictions by area-based method. In this case, two third of data were used as training and one third as testing. The procedure was repeated 100 times. This corresponds to an ideal situation where calibration data can be collected from the area being studied.
(3)
A hybrid approach, consisting of the currently applied solution added with local reference data. All Evo data and 10% of Kalkkinen data (randomly selected 96 trees) were used to develop models. Predictions were conducted firstly for individual trees and then individual predictions were aggregated to plot level followed by area-based estimations. This corresponds to a situation where a certain amount of local reference data can be collected from the area being studied.
We used Random Forests (RF) to construct the forest prediction models, since previous studies have reported as good performance as other parametric or non-parametric methods if not better (e.g., [42]). In strategy 1 and 2, 23 plot features derived from ALS data were used as predictors. In strategy 3, 26 tree features derived from individual tree detection were used as predictors in individual tree prediction and aggregated values as predictors in area-based predictions. More details of the methods can be found in [43].

3. Results

3.1. Measuring Individual Trees

Table 2 gives the bias (cm) and RMSE (cm) and RMSE-% values when Kinect diameters are compared with calliper and tape, calliper with tape and Tango diameters with tape. The use of Kinect or Tango resulted in a smaller biases than using callipers, perhaps due to slack of the calliper jaws. The RMSE of 1.9 cm of Kinect (7.3%) compared to 1.2 cm of calliper (5.9%) is an impressive level of accuracy for a low-cost (100€) remote sensing device. An even more impressive RMSE of 0.73 cm (1.89%) was attained with a consumer smart phone having a Tango sensor. Scatter plots of diameters measured by Kinect and Tango compared to those obtained with measuring tape are shown in Figure 5a,b.
A positive mean bias of 3.34 mm was observed between Tango data and tape measures (ranging from minimum error of 0.3 mm to maximum error of 26.5 mm), with the RMSE being 0.73 cm. The performance is impressive for a handheld consumer device with on device point cloud reconstruction. When observing the repeated Tango measurements, it was discovered that the performance of the system varied significantly from scan to another. Figure 6 shows two different scans for the same tree stem. One of the scans seems to contain a misaligned segment, whereas the other is well composed. Looking at the results tree by tree, RMSE of 0.32 cm was attained for the best individual tree scan, the worst being 1.47 cm, calculated from all profiles obtained from the same tree in one scan.

3.2. Measuring Multiple Trees

If two trees were reconstructed in the same Tango scan (as in Figure 3), more performance variations were encountered (Figure 7). As the size of the scanned area increased, more incorrectly registered segments are produced by the system, negatively effecting the accuracy of diameter estimation (Figure 8).
In best cases, the performance remained high, with smallest RMSE of a single tree being 0.44 cm, calculated from 16 diameter estimates of the stem. However, in the worst case, an RMSE of 6.06 cm was observed, calculated from 16 diameter estimates of the stem. For individual diameter estimates, the best fit was near perfect (error of 0.01%), with the worst ones having coarse errors (20.48%). In scans covering several trees, the best ones did not reach same accuracy as single tree scans, and the worst ones were significantly more erroneous.

3.3. Local Field Reference Test

The local field reference test (Table 3) shows that without local field data, there is a large bias on the results (5 cm with DBH and 57 m3/ha with volume). This is due to different forest conditions of the two sites. This is significant, especially as the sites are in the same geographic area, only 34.5 km apart. Use of local plot data (Strategy 2) lead to almost zero bias and also improved RMSE. Compared with Strategy 1, both bias and RMSE have been reduced by using small amount local reference data (Strategy 3). When compared with Strategy 2, RMSE of volume estimate is at the same level while mean DBH estimate needs to be improved.

4. Discussion

The accuracy of both Kinect and Tango for stem diameter measurements is promising, taking into account that professional good-quality calliper results in precision of 7 mm [38]. The bias of the diameter estimate using Kinect range data seems to be negligible and in this test better than with the calliper. Even better accuracy was obtained with handheld, mobile integrated Tango system onboard a Phab2ProAR (Lenovo, Morrisville, NC, USA) [39]: A small positive mean bias (0.3 cm) was observed for individually scanned trees, most likely caused by noise in point cloud. Previous studies using TLS for tree DBH measurements have resulted in accuracies between 1 and 2.5 cm [2]. Use of other electronic devices, such as integrated laser beam and digital imaging, has resulted in accuracy of 0.6 cm using semi-automatic processing and 1.3 cm when using fully-automatic processing [44]. Accuracies of Kinect and Tango are comparable with those obtained with other sensors. Coefficient of determination (R2) of 0.98 for Kinect and 0.96 for Tango were found when regressed against measuring tape results, when measuring individual trees. When scanning several stems simultaneously with Tango, the results were clearly inferior (R2 = 0.23). According to our results, the Tango system is more suitable for scanning individual trees.
The results presented were obtained with the first version of the Kinect, after which an improved version was released, potentially providing better performance. We see two practical ways of using Kinect which have been demonstrated in this study: (1) measurement of DBH and processing all data related to one frame of the fixed height; (2) use of the Kinect to measure the first part of the tree stem to estimate the whole stem curve. Both areas should be further studied.
For Tango, our results are in line with [34], where RMSE in range 1.61–2.10 cm was attained for scanning whole plots in one go. A significantly higher accuracy for diameter estimation (RMSE 0.73 cm) was attained in our tests when scanning individual tree stems. This would suggest that for best performance, Tango devices should be used to scan individual trees rather than plots/patterns. As multi-sensor system, the performance of a Tango device in 3D scanning is influenced by several factors: the scanning accuracy of the depth sensor, accuracy of orientation sensors, correctness in registration of consecutive depth images, and the quality of the motion the device experiences while scanning. This is clearly visible in the performance variations found in repeated scans.
A benefit of Tango is that it provides an integrated package offering processing capabilities (with smartphones’ own computational resources), connections (wireless local area network (WLAN), global system for mobile communications (GSM)) and other sensors (such as global navigation satellite systems (GNSS)), whereas Kinect is a separate device that requires a computer to operate. From an application development point of view, the Tango sensor holds significant potential for developing mobile applications for crowdsourcing 3D data, if the sensors become a common feature in smart phones. Currently they are rare, hindering development of applications. In addition, the data quality of all hand-hold devices for crowdsourcing applications still needs to improve. Similar misalignments can be clearly seen from the current Tango devices, as shown in Figure 7b, and the image-based point cloud using structure from motion technique as shown in [20], which cost under- or overestimates of tree DBHs.
For both of the tested systems, the biggest efficiency improvement when compared with the manual measuring methods, such as measuring tape, is the digitization of the entire lower part of the tree stem in a single scan. Ideally they should therefore be applied rather to stem curve estimation than only producing a DBH estimate. In the presented experiment, both systems were operated by experienced professionals. Determining whether inexperienced users would attain similar results would require further study.
In the local field reference test, we applied airborne laser scanning data [41,43], area-based approach based on Random Forest and individual tree technique [42]. We aimed to get reliable inventory results for natural forest in Kalkkinen using Evo field plots and data collected in Kalkkinen corresponding to Kinect/Tango diameter measurements. Without any field data from Kalkkinen, the bias of stem volume estimation was 57 m3/ha. Using a small amount of local field data resulted in significant reduction of this bias and RMSE, Table 3. Thus, there is a need for local data. In this study, the data applied was still obtained with traditional forest inventory practices having accuracy comparable to individual tree data provided by either Tango or Kinect systems. Clearly, such data for forest inventory calibration could well be provided by crowdsourcing with these emerging sensor technologies.
In the work presented, the accuracy of DBH extraction using mobile sensors was estimated. While this is an essential component in application for crowdsourcing forestry data, it is not the only aspect of their applicability. An additional question is, how would land-owners be able to measure the location of the trees which they would like to input to the reference. Since GNSS accuracy in forest is not enough for plot or individual tree level inventory, an alternative solution would have to be applied. One possibility would be the application of a GNSS assisted 3D Game Engine based approach. In this, an approximate location from GNSS could be further refined by the user, referencing to visualized ITD representations from ALS data. This would allow the users to visually locate themselves more accurately than solely by GNSS.
A potential user interface can be drafted from the same outset: The output of the ITD interpretation is visualized in the 3D Game Engine running in smart phone, and user can select the tree groups or individual trees from ITD interpretation to which he/she likes to give field reference. If the ITD process indicated one tree which corresponds to several trees in the field, the land-owner records all trees and their attributes, especially the diameter breast heights and tree species. For suitable plots, he/she preferably records corresponding information of all trees. Also, if there are omissions in the ITD process, i.e., trees missing from the original estimate, these can be added manually.
In addition to technical challenges, success of crowdsourcing campaigns is dependent on the user participation. In the presented case, the motivation of using 3D sensing sensors for forest owners in a crowdsourcing context would be apparent: to get improved estimates of one’s own forest resources. The motivation for the system provider is also obvious: improvement in accuracy for the forest inventory.

5. Conclusions

In this paper, we have demonstrated that the Kinect and Google Tango depth sensors are feasible for tree stem mapping (i.e., diameter estimation and part of the stem curve estimation) with relatively good accuracy. Kinect-derived tree diameters agreed with tape measurements to 1.9 cm (RMSE) and 7.3% (RMSE-%). The stem curve from the ground to the height of 1.7 m agreed with similar statistics. For Tango enabled smartphone, an RMSE of 0.73 cm was achieved in diameter estimation compared to measurement tape. The accuracies are adequate for operational work. In the local field reference test, we showed that locally collected field reference improves significantly forest inventory estimates at boreal forest conditions. Based on these results, we have introduced a crowdsourcing concept based on individual tree diameter and stem curve measurements, using Kinect and mobile-phone embedded 3D sensor (Tango) for ALS (Airborne Laser Scanning)-based large-area forest inventory at individual tree level.

Supplementary Materials

Supplementary File 1

Acknowledgments

This work was supported by the Academy of Finland projects “Centre of Excellence in Laser Scanning Research (projects 307362, 292735)”, “New laser and spectral field methods for in situ mining and raw material investigations (project 292648)”, “Multispectral mobile and UAV-based laser scanning” (project 307823) and “Competence-Based Growth Through Integrated Disruptive Technologies of 3D Digitalization, Robotics, Geospatial Information and Image Processing/Computing—Point Cloud Ecosystem” (No. 293389). The Finnish Funding Agency for Innovation is acknowledged for support in project “VARPU” (7031/31/2016). The European Union, the European Regional Development Fund, the “Leverage from the EU 2014-2020” projects “AKAI” (301130) and 3D Cultural Hub (A72980) are acknowledged.

Author Contributions

J.H. developed the concept. A.J. and J.-P.V. made Kinect and Tango measurements. X.Y. was responsible for major part of data analysis. All authors corresponded to drawing conclusions. J.H. and J.-P.V. wrote most parts of the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Houghton, R.A.; Hall, F.; Goetz, S.J. Importance of biomass in the global carbon cycle. J. Geophys. Res. 2009, 114, 2156–2202. [Google Scholar] [CrossRef]
  2. Liang, X.; Kankare, V.; Hyyppä, J.; Wang, Y.; Kukko, A.; Haggrén, H.; Yu, X. Terrestrial laser scanning in forest inventories. ISPRS-J. Photogramm. Remote Sens. 2015, 115, 63–77. [Google Scholar] [CrossRef]
  3. Tomppo, E.; Olsson, H.; Ståhl, G.; Nilsson, M.; Hagner, O.; Katila, M. Combining national forest inventory field plots and remote sensing data for forest databases. Remote Sens. Environ. 2008, 112, 1982–1999. [Google Scholar] [CrossRef]
  4. Hyyppä, J.; Hyyppä, H.; Inkinen, M.; Engdahl, M.; Linko, S.; Zhu, Y.-H. Accuracy comparison of various remote sensing data sources in the retrieval of forest stand attributes. For. Ecol. Manag. 2000, 128, 109–120. [Google Scholar] [CrossRef]
  5. Nilsson, M. Estimation of tree heights and stand volume using an airborne lidar system. Remote Sens. Environ. 1996, 56, 1–7. [Google Scholar] [CrossRef]
  6. Næsset, E. Predicting forest stand characteristics with airborne scanning laser using a practical two-stage procedure and field data. Remote Sens. Environ. 2002, 80, 88–99. [Google Scholar] [CrossRef]
  7. Maltamo, M.; Eerikäinen, K.; Packalen, P.; Hyyppä, J. Estimation of stem volume using laser scanning-based canopy height metrics. Forestry 2006, 79, 217–229. [Google Scholar] [CrossRef]
  8. Hyyppä, J.; Inkinen, M. Detecting and estimating attributes for single trees using laser scanner. Photogramm. J. Finl. 1999, 16, 27–42. [Google Scholar]
  9. Brandtberg, T. Automatic Individual Tree-Based Analysis of High Spatial Resolution Remotely Sensed Data; Acta Universitatis Agriculturae Sueciae Silvestria: Uppsala, Sweden, 1999. [Google Scholar]
  10. Pouliot, D.A.; King, D.J.; Bell, F.W.; Pitt, D.G. Automated tree crown detection and delineation in high-resolution digital camera imagery of coniferous forest regeneration. Remote Sens. Environ. 2002, 82, 322–334. [Google Scholar] [CrossRef]
  11. Holmstrom, H.; Kallur, H.; Stahl, G. Cost-plus-loss analyses of forest inventory strategies based on kNN-assigned reference sample plot data. Silva Fenn. 2003, 37, 381–398. [Google Scholar]
  12. Liang, X.; Kankare, V.; Yu, X.; Hyyppä, J.; Holopainen, M. Automated stem curve measurement using terrestrial laser scanning. IEEE Trans. Geosci. Remote Sens. 2014, 52, 1739–1748. [Google Scholar] [CrossRef]
  13. Heipke, C. Crowdsourcing geospatial data. ISPRS-J. Photogramm. Remote Sens. 2010, 65, 550–557. [Google Scholar] [CrossRef]
  14. Fritz, S.; McCallum, I.; Schill, C.; Perger, C.; Grillmayer, R.; Achard, F.; Kraxner, F.; Obersteiner, M. Geo-Wiki.Org: The Use of Crowdsourcing to Improve Global Land Cover. Remote Sens. 2009, 1, 345–354. [Google Scholar] [CrossRef]
  15. Philly Tree Map. Available online: https://www.opentreemap.org/phillytreemap/map/ (accessed on 4 December 2017).
  16. Urban Forest Maps. Available online: https://urbanforestmap.org/ (accessed on 4 December 2017).
  17. Foster, A.; Dunham, I.M.; Kaylor, C. Citizen Science for Urban Forest Management? Predicting the Data Density and Richness of Urban Forest Volunteered Geographic Information. Urban Sci. 2017, 3, 30. [Google Scholar] [CrossRef]
  18. Molinier, M.; López-Sánchez, C.A.; Toivanen, T.; Korpela, I.; Corral-Rivas, J.J.; Tergujeff, R.; Häme, T. Relasphone—Mobile and Participative In Situ Forest Biomass Measurements Supporting Satellite Image Mapping. Remote Sens. 2016, 8, 869. [Google Scholar] [CrossRef]
  19. Vastaranta, M.; Latorre, E.G.; Luoma, V.; Saarinen, N.; Holopainen, M.; Hyyppä, J. Evaluation of a smartphone app for forest sample plot measurements. Forests 2015, 6, 1179–1194. [Google Scholar] [CrossRef]
  20. Liang, X.; Wang, Y.; Jaakkola, A.; Kukko, A.; Kaartinen, H.; Hyyppä, J.; Honkavaara, E.; Liu, J. Forest data collection using terrestrial image-based point clouds from a handheld camera compared to terrestrial and personal laser scanning. IEEE Trans. Geosci. Remote Sens. 2015, 53, 5117–5132. [Google Scholar] [CrossRef]
  21. Kaartinen, H.; Hyyppä, J.; Yu, X.; Vastaranta, M.; Hyyppä, H.; Kukko, A.; Holopainen, M.; Heipke, C.; Hirschmugl, M.; Morsdorf, F.; et al. An international comparison of individual tree detection and extraction using airborne laser scanning. Remote Sens. 2012, 4, 950–974. [Google Scholar] [CrossRef] [Green Version]
  22. Wang, Y.; Hyyppä, J.; Liang, X.; Kaartinen, H.; Yu, X.; Lindberg, E.; Holmgren, J.; Qin, Y.; Mallet, C.; Ferraz, A.; et al. International benchmarking of the individual tree detection methods for modeling 3-D canopy structure for silviculture and forest ecology using airborne laser scanning. IEEE Trans. Geosci. Remote Sens. 2016, 54, 5011–5027. [Google Scholar] [CrossRef]
  23. Vauhkonen, J.; Ene, L.; Gupta, S.; Heinzel, J.; Holmgren, J.; Pitkänen, J.; Solberg, S.; Wang, Y.; Weinacker, H.; Hauglin, K.M.; et al. Comparative testing of single-tree detection algorithms under different types of forest. Forestry 2011, 85, 27–40. [Google Scholar] [CrossRef]
  24. Henry, P.; Krainin, M.; Herbst, E.; Ren, X.; Fox, D. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robot. Res. 2012, 31, 1–17. [Google Scholar] [CrossRef]
  25. Kroshelham, K.; Oude Elberink, S. Accuracy and Resolution of Kinect Data for Indoor Mapping Applications. Sensors 2012, 12, 1437–1454. [Google Scholar] [CrossRef] [PubMed]
  26. Kroshelham, K. Accuracy analysis of Kinect Depth data. In Proceedings of the ISPRS Calgary 2011 Workshop International Archives Photogrammetry Remote Sensing Spatial Information Sciences, Calgary, AB, Canada, 29–31 August 2011; Volume XXXVIII-5/W12. [Google Scholar]
  27. Chen, J.M.; Liu, X.; Zhao, Z.X.; Zhong, W.T. Research on landscape environment with 3D-reconstruction and volume measurement of fruit tree canopy based on Kinect. Adv. Mater. Res. 2013, 788, 480–485. [Google Scholar] [CrossRef]
  28. Qi, X.; Lichti, D.; El-Badry, M.; Chow, J.; Ang, K. Vertical Dynamic Deflection Measurement in Concrete Beams with the Microsoft Kinect. Sensors 2014, 14, 3293–3307. [Google Scholar] [CrossRef] [PubMed]
  29. Ostovar, A. Enhancing Forestry Object Detection Using Multiple Features. Master’s Thesis, Umeå University, Umeå, Sweden, 2011. [Google Scholar]
  30. Pordel, M.; Hellström, T.; Ostovar, A. Integrating Kinect Depth Data with a Stochastic Object Classification Framework for Forestry Robots. In Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics, Rome, Italy, 28–31 July 2012; pp. 314–320. [Google Scholar]
  31. Brouwer, T. Low Budget Ranging for Forest Measurements. Master’s Thesis, Wageningen University, Wageningen, Germany, 2013. [Google Scholar]
  32. Tango. Available online: https://get.google.com/tango/ (accessed on 17 November 2017).
  33. Depth Perception. Available online: https://developers.google.com/tango/overview/depth-perception (accessed on 17 November 2017).
  34. It’s Your Turn. Build the Future. Available online: https://developers.google.com/tango/ (accessed on 17 November 2017).
  35. Diakité, A.A.; Zlatanova, S. First experiments with the tango tablet for indoor scanning. In Proceedings of the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Prague, Czech Republic, 12–19 July 2016; Volume III-4. [Google Scholar] [CrossRef]
  36. Zhang, E.; Cohen, M.F.; Curless, B. Emptying, refurnishing, and relighting indoor spaces. ACM Trans. Graph. 2016, 35. [Google Scholar] [CrossRef]
  37. Tomaštík, J.; Saloň, Š.; Tunák, D.; Chudý, F.; Kardoš, M. Tango in forests—An initial experience of the use of the new Google technology in connection with forest inventory tasks. Comput. Electron. Agric. 2017, 141, 109–117. [Google Scholar] [CrossRef]
  38. Päivinen, R.; Nousiainen, M.; Korhonen, K. Puutunnusten mittaamisen luotettavuus. Folia Forestalia 1992, 787, 1–18. [Google Scholar]
  39. Lenovo Phab 2 Pro. Available online: http://www3.lenovo.com/us/en/virtual-reality-and-smart-devices/augmented-reality/-phab-2-pro/Lenovo-Phab-2-Pro/p/WMD00000220 (accessed on 17 November 2017).
  40. Matterport Scenes. Available online: https://play.google.com/store/apps/details?id=com.matterport.capture&hl=en (accessed on 17 November 2017).
  41. Yu, X.; Hyyppä, J.; Kukko, A.; Maltamo, M.; Kaartinen, H. Change detection techniques for canopy height growth measurements using airborne laser scanner data. Photogramm. Eng. Remote Sens. 2006, 72, 1339–1348. [Google Scholar] [CrossRef]
  42. Yu, X.; Hyyppä, J.; Vastaranta, M.; Holopainen, M.; Viitala, R. Predicting individual tree attributes from airborne laser point clouds based on random forest technique. ISPRS-J. Photogramm. Remote Sens. 2011, 66, 28–37. [Google Scholar] [CrossRef]
  43. Hyyppä, J.; Yu, X.; Hyyppä, H.; Vastaranta, M.; Holopainen, M.; Kukko, A.; Kaartinen, H.; Jaakkola, A.; Vaaja, M.; Koskinen, J.; et al. Advances in forest inventory using airborne laser scanning. Remote Sens. 2012, 4, 1190–1207. [Google Scholar] [CrossRef]
  44. Melkas, T.; Vastaranta, M.; Holopainen, M.; Kivilähde, J.; Merimaa, M. Puun läpimitan mittauksen tarkkuus ja tehokkuus laser—Digitaalikuvatekniikkaan perustuen. Metsätieteen Aikakauskirja 2009, 4, 325–340. [Google Scholar] [CrossRef]
Figure 1. RGB image (left) and range image (middle) taken by Kinect sensor. Markers pinned on the trunk can be visually identified from the RGB image. A composite image showing the stem from the ground to the height of 1.7 m, formed by merging consecutive images (right).
Figure 1. RGB image (left) and range image (middle) taken by Kinect sensor. Markers pinned on the trunk can be visually identified from the RGB image. A composite image showing the stem from the ground to the height of 1.7 m, formed by merging consecutive images (right).
Forests 09 00006 g001
Figure 2. One example of diameter determination. Left: original RGB image, extracted trunk skeleton and identified position of the three markers. Right: circle fitting to the points at the height of the markers.
Figure 2. One example of diameter determination. Left: original RGB image, extracted trunk skeleton and identified position of the three markers. Right: circle fitting to the points at the height of the markers.
Forests 09 00006 g002
Figure 3. Two stems reconstructed in single Tango scan session.
Figure 3. Two stems reconstructed in single Tango scan session.
Forests 09 00006 g003
Figure 4. (Left) A reconstructed point cloud of a tree stem and surroundings obtained with mobile device. (Right) Extracted cross-sections of stem applied in circle fitting.
Figure 4. (Left) A reconstructed point cloud of a tree stem and surroundings obtained with mobile device. (Right) Extracted cross-sections of stem applied in circle fitting.
Forests 09 00006 g004
Figure 5. Diameters obtained with tape measure and (a) Kinect (n = 41, R2 = 0.984) and (b) Tango (n = 240, R2 = 0.957), for Tango the repetitive measurements have been included (80 diameter estimates repeated three times).
Figure 5. Diameters obtained with tape measure and (a) Kinect (n = 41, R2 = 0.984) and (b) Tango (n = 240, R2 = 0.957), for Tango the repetitive measurements have been included (80 diameter estimates repeated three times).
Forests 09 00006 g005
Figure 6. Misalignments of a tree stem segments from Tango device. The same tree scanned at two different points of time is shown in (a,b), respectively. The cross sections of marked areas are shown from a top-down view on the right. The misalignments can be seen in both scans, but the repeated scans at different points of time show different misalignments, which indicates the performance of the Tango system may clearly vary from scan to another.
Figure 6. Misalignments of a tree stem segments from Tango device. The same tree scanned at two different points of time is shown in (a,b), respectively. The cross sections of marked areas are shown from a top-down view on the right. The misalignments can be seen in both scans, but the repeated scans at different points of time show different misalignments, which indicates the performance of the Tango system may clearly vary from scan to another.
Forests 09 00006 g006
Figure 7. (a,b) Two scans of the same two adjacent stems with marked cross-sections showed from a top-down view in insets. In (b) the stem is clearly effected by a coarse registeration error. If diameter estimation is performed via circular fitting to such cross-section, the results fail to describe the tree diameter properly.
Figure 7. (a,b) Two scans of the same two adjacent stems with marked cross-sections showed from a top-down view in insets. In (b) the stem is clearly effected by a coarse registeration error. If diameter estimation is performed via circular fitting to such cross-section, the results fail to describe the tree diameter properly.
Forests 09 00006 g007
Figure 8. Scatterplot showing the diameter estimations for the same stem from 16 different heights, from three different iterations. In iteration 3, the results are clearly inferior.
Figure 8. Scatterplot showing the diameter estimations for the same stem from 16 different heights, from three different iterations. In iteration 3, the results are clearly inferior.
Forests 09 00006 g008
Table 1. Descriptive statistics of Kalkkinen (33 plots, 963 trees) and Evo forest data (292 plots, 8763 trees). Height, diameter at breast height (DBH), and volume are given.
Table 1. Descriptive statistics of Kalkkinen (33 plots, 963 trees) and Evo forest data (292 plots, 8763 trees). Height, diameter at breast height (DBH), and volume are given.
MinimumMaximumMeanStandard Deviation
Kalkkinen
Mean tree height (m)7.1228.1421.945.43
Mean DBH (cm)6.7925.8318.404.89
Volume (m3/ha)21.20540.39292.32122.75
Evo
Mean tree height (m)3.931.718.06.1
Mean DBH (cm)7.650.818.36.9
Volume (m3/ha)0.4586.2148.2110.7
Table 2. Bias, root-mean-square error (RMSE) and RMSE-% of diameter between Kinect sensor data, tape measurements, measurements by caliper, or measurements with Tango device in Kirkkonummi, Finland. For Tango, the values are given for individual tree measurements.
Table 2. Bias, root-mean-square error (RMSE) and RMSE-% of diameter between Kinect sensor data, tape measurements, measurements by caliper, or measurements with Tango device in Kirkkonummi, Finland. For Tango, the values are given for individual tree measurements.
Kinect Data vs. CaliperCaliper (Mean of Two Directions) vs. TapeKinect Data (Mean) vs. TapeTango (Mean) vs. Tape
East-WestNorth-SouthMean
Bias (cm)−2.600.301.500.950.540.33
RMSE (cm)2.252.632.501.161.900.73
RMSE-%10.210.010.05.97.31.89
Table 3. The bias, RMSE, and correlation coefficient (R) for the local field reference test, comparing the results of forest inventory from ALS with three different sets of manually obtained reference data.
Table 3. The bias, RMSE, and correlation coefficient (R) for the local field reference test, comparing the results of forest inventory from ALS with three different sets of manually obtained reference data.
Strategy 1—Reference Plots Located at Different Forest Area
BiasRMSERMSE-%R
Mean DBH (cm)5.086.2732.850.74
Volume (m3/ha)56.98104.3234.320.80
Strategy 2—Reference Plots Located at Test Site
Mean DBH (cm)0.222.9015.180.83
Volume (m3/ha)−3.4196.0631.610.73
Strategy 3—Reference Plots Located at Different Forest Area, Selective Measurements in Test Site
Mean DBH (cm)3.514.6724.420.83
Volume (m3/ha)24.4896.9731.900.77

Share and Cite

MDPI and ACS Style

Hyyppä, J.; Virtanen, J.-P.; Jaakkola, A.; Yu, X.; Hyyppä, H.; Liang, X. Feasibility of Google Tango and Kinect for Crowdsourcing Forestry Information. Forests 2018, 9, 6. https://doi.org/10.3390/f9010006

AMA Style

Hyyppä J, Virtanen J-P, Jaakkola A, Yu X, Hyyppä H, Liang X. Feasibility of Google Tango and Kinect for Crowdsourcing Forestry Information. Forests. 2018; 9(1):6. https://doi.org/10.3390/f9010006

Chicago/Turabian Style

Hyyppä, Juha, Juho-Pekka Virtanen, Anttoni Jaakkola, Xiaowei Yu, Hannu Hyyppä, and Xinlian Liang. 2018. "Feasibility of Google Tango and Kinect for Crowdsourcing Forestry Information" Forests 9, no. 1: 6. https://doi.org/10.3390/f9010006

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop