A Ground Point Fitting Method for Winter Wheat Height Estimation Using UAV-Based SfM Point Cloud Data

Abstract

Height is a key factor in monitoring the growth status and rate of crops. Compared with large-scale satellite remote sensing images and high-cost LiDAR point cloud, the point cloud generated by the Structure from Motion (SfM) algorithm based on UAV images can quickly estimate crop height in the target area at a lower cost. However, crop leaves gradually start to cover the ground from the beginning of the stem elongation stage, making more and more ground points below the canopy disappear in the data. The terrain undulations and outliers will seriously affect the height estimation accuracy. This paper proposed a ground point fitting method to estimate the height of winter wheat based on the UAV SfM point cloud. A canopy slice filter was designed to reduce the interference of middle canopy points and outliers. Random Sample Consensus (RANSAC) was applied to obtain the ground points from the valid filtered point cloud. Then, the missing ground points were fitted according to the known ground points. Furthermore, we achieved crop height monitoring at the stem elongation stage with an R² of 0.90. The relative root mean squared error (RRMSE) of height estimation was 5.9%, and the relative mean absolute error (RMAE) was 4.6% at the stem elongation stage. This paper proposed the canopy slice filter and fitting missing ground points. It was concluded that the canopy slice filter successfully optimized the extraction of ground points and removed outliers. Fitting the missing ground points simulated the terrain undulations effectively and improved the accuracy.

1. Introduction

Crop height plays an important role in accurately monitoring the growth status, predicting the yield of crops, and optimizing agricultural production management [1,2]. The crop height in this paper refers to the distance from the top of the leaves to the ground. Traditional crop height estimation methods are mainly layer-by-layer statistics and large-area field manual measurement. There are significant issues, such as a waste of measurement time and human resources, as well as a lack of spatial distribution information. Remote sensing technology has the advantages of being fast, macro, and dynamic [3,4], providing an effective means to the field of crop height monitoring. There have been many large-scale vegetation studies based on satellite remote sensing images [5,6] with the rapid development of remote sensing technology in the past few decades. For example, crop information was extracted to evaluate crop growth status by calculating different vegetation indices [7,8]. However, the calculation of crop parameters through the interpretation of a single remote sensing image still has many limitations, such as the occlusion of clouds and fog, the low temporal resolution of image shooting, and the lack of vegetation canopy structure information. Therefore, studies using LiDAR to obtain information on crops have emerged in recent years. LiDAR can effectively reconstruct the canopy structure by processing the point cloud and operating day and night without time limitation, which is an active technical method for accurately extracting crop parameters [9,10,11].

However, LiDAR equipment is expensive for everyday use. The Structure from Motion algorithm, on the contrary, uses a moving camera to determine the spatial and geometric relationship of a stationary target, making it a much cheaper alternative. Correspondingly, only an ordinary digital camera is needed to perform the three-dimensional reconstruction accurately, which costs less and provides wide applications [12,13,14,15]. In addition, the unmanned aerial vehicle has the advantages of its simple structure, low flying cost, and high flexibility. UAV remote sensing is currently a popular research topic that is progressing from the experiment stage to the application stage [16,17,18].

Generating a 3D point cloud based on digital images and LiDAR cannot label the type of object. So it is necessary to separate ground points and the target points or remove outliers to obtain valid information [19,20,21,22,23] before processing the point cloud. Nowadays, there are two main methods for extracting ground points: morphological methods and iteration methods. Zhang et al. [24] attempted to invert the point cloud and cover it with a rigid cloth to extract ground points. This method was a simplification and innovation compared with the past methods that always required complicated parameters. Zhao et al. [25] also made improvements to the classical progressive triangulated irregular network (TIN) densification. They obtained the seed points with a morphological method, and they built a TIN-based model to densify TIN iteratively until all ground points were classified. The two methods above are available to most applications with high accuracy. However, for the crop point cloud generated by SfM, the narrow spacing between crops results in leaf overlap between different plants. Part of the ground under the crop is impossible to be captured by the UAV images and is described as points. Consequently, in this case, the point cloud cannot satisfy the sampling point requirements of the two previous methods. Rather than that, methods for extracting ground points based on elevation and slope are more straightforward [26].

To eliminate outliers, it is usually necessary to define a proportional threshold to the point cloud’s structure. Khanna et al. [27] estimated crop height by removing outliers from the top 1% of the point cloud. However, the canopy structure was easily destroyed if there were noticeable ground undulations in the crop area. Furthermore, threshold selection will be more and more complicated with the diversification of filtering methods and applications [28,29]. To calculate the accurate canopy height, Song et al. [30] proposed a moving cuboid filter to remove noise points based on UAV point cloud data. Additionally, they conducted a detailed analysis of the threshold selection process. The methods outlined above have generated numerous ideas about how to filter the point cloud and extract the ground points. Whereas the point cloud of the crop, the unknown ground height under the canopy also needs to be fitted to simulate the terrain undulations when the ground is seriously covered by leaves [31,32]. For example, the maximum difference in ground height is 3 m in the study area, and the crop height is always less than 1 m. The changes in ground height have a serious impact on crop height estimation when the ground is seriously covered by leaves. The method proposed in this paper was available to fit the missing ground height and calculate the crop height. This method sliced and filtered the crop canopy, which separated the points near the ground and the top canopy points. The filtered point cloud was used to extract the ground point by the Random Sample Consensus algorithm, and we estimated the elevation information of the unknown ground points based on the known ground points. Finally, the height difference between the crop vertex and the corresponding ground point was the crop height.

2. Materials and Methods

2.1. Materials

2.1.1. Study Site Description

The study site was located in a 150 m × 250 m winter wheat field near Melbourne in southwestern Ontario, Canada, as shown in Figure 1a. The growing period of winter wheat usually began in October of the previous year and lasted until June. Nonetheless, the spring of 2019 was relatively cold, and the growth period was extended to July. The row spacing was about 20 cm, and a total of 32 sampling points were evenly selected. The distribution of sampling points is shown in Figure 1b. Within a radius of 2 m of each sampling point, three plants were randomly selected to calculate the average crop height as the actual measured height. In addition, the black and white target boards were set up as ground control points at 12 places to ensure the matching accuracy of UAV images and point cloud generated. The sampling process lasted from the beginning of May to June. The crop height was sampled about every five days, covering the tillering, stem elongation, and heading stage.

2.1.2. Point Cloud Generation

The UAV images of the crop area were obtained during the sampling process based on a DJI Phantom 4 RTK UAV system equipped with a 5K high-resolution digital RGB camera and an RTK base station. We flew the UAV when the sky was clear to reduce the impact of shadows on image quality caused by clouds. The flying height was set to 30 m above the ground, and the image overlap was set to 90%. The UAV images captured had a spatial resolution of 9 mm, which met the accuracy requirements for the subsequent point cloud generation. The SfM algorithm generated the point cloud based on the UAV images. We achieved it with the Pix4Dmapper Pro (Pix4D) v2.4 (Pix4D SA, Lausanne, Switzerland). The generated point cloud data was similar to LiDAR data at a lower cost, containing each point’s 3D coordinates and RGB information, as shown in Figure 2. We performed data preprocessing based on Python for the original point cloud, including data cropping and format conversion. In the subsequent processing, the representative data from six days were selected to estimate the crop height, and the relevant information from these six days was listed in

Table 1. UAV flight date, points in data, point density, average measured crop height, and winter wheat growth phenology.

Flight Date	Number of Points	Density (pts/m2)	Averaged Height(cm)	Growth Stage (BBCH *)
11 May 2019	52,680,932	1479	21.2	Tillering (21)
16 May 2019	96,219,799	2702	29.5	Tillering (25)
21 May 2019	87,226,883	2449	29.4	Stem elongation (31)
27 May 2019	96,324,119	2705	42.6	Stem elongation (39)
3 June 2019	95,140,823	2671	53.7	Booting (49)
11 June 2019	85,375,485	2397	62.8	Heading (59)

* BBCH scale: Biologische Bundesanstalt, Bundessortenamt, and Chemical industry.

.

2.2. Methods

2.2.1. The Canopy Slice Filter

In the crop height estimation, the effective information was only the elevation of canopy vertices and the corresponding ground points. So, the middle canopy points that interfered with the extraction of ground points could be removed, as shown in Figure 3a. Similarly, in the same sub-area, the spatial distribution of the middle canopy points was sparser than that of other points due to the overlap of leaves. So, the density of the valid points was significantly higher than that of the invalid points, and the canopy slice filter separated dense valid points from sparse invalid points based on this density difference [30]. At the same time, the point cloud was divided into 1 m × 1 m sub-areas to filter to preserve the canopy structure and terrain undulations. Then, we built a bounding box based on the point cloud in each sub-area, and we divided it into many slices with a thickness of 5 cm from bottom to top in the z-axis direction, as shown in Figure 3b. The average number of points N_5cm in all slices was calculated and compared with the number of points N_down in each slice in the lower half of the bounding box. If N_idown < N_5cm, the points in the ith slice were considered invalid canopy points. If N_idown ≥ N_5cm, the points in the ith slice were considered valid points, which were the ground points and a small number of canopy points. The slice thickness was adjusted from 1 cm to 20 cm with the step of 1 cm. The slice thicknesses with the best filtering effect were selected at the tillering and stem elongation stage. Finally, we found that the most appropriate slice thickness for the ground points was 5 cm.

In the z-axis direction, the bounding box was redivided into many slices with a thickness of 10 cm from top to bottom (Figure 3c). Because the top of the crop had a certain thickness, and the points at the top 10 cm were denser. In the same way, the point number N_up of each slice in the upper half of the bounding box was compared with the average number of points N_10cm in all slices. Then, we obtained the valid points, which were canopy vertices in the upper part. There was no perfect slice thickness for the canopy, and 10 cm was set to be the thickness of the slices in the upper half of the sub-area to avoid damage to the structure of the top canopy during the filtering process. Additionally, 5 cm was set to be the thickness of the slices in the lower half of the sub-area to remove outliers near the ground. After the above processing, we achieved the removal of the sparse middle canopy points and outliers in the sub-area. However, not all sub-areas contained both the canopy and ground points, as shown in Figure 4. So, a cluster analysis was performed on the points in the sub-area according to elevations at first. Density-based clustering method can find clusters of different shapes and sizes in the data. The key idea is to find points with higher density first and then gradually connect similar points with higher density to produce different clusters. When the points were concentrated at two different heights, there was an upper cluster center and a lower cluster center. Therefore, we believed that there are two parts of valid points, and we performed the above operations. When the points were concentrated at one height, we considered only one part of the valid points. Moreover, we determined which cluster center of the adjacent sub-areas was closer to this cluster center. If it was closer to the lower cluster center, the points were considered near the ground. Then, the 5 cm slice filter would be applied. If it was closer to the upper cluster center, the points were considered at the top canopy, and the 10 cm slice filter would be applied.

2.2.2. Extraction and Fitting of Ground Points

For the robust estimation of the model based on data, the main difficulty often comes from the interference of outliers. In comparison, RANSAC can generate hypotheses from random samples and validate the data without complex optimization algorithms or large amounts of memory [33,34,35], and it is widely used to solve such problems as fitting models. So, in this paper, RANSAC was applied to extract ground points from the valid points obtained with the canopy slice filter. According to the following basic idea, the points near the ground were divided into two categories: ground points and crop points. We first merged the original sub-areas into new sub-areas of 10 m × 10 m to improve the continuity of ground points. Then, for the valid points in the lower half, 3 sample points were randomly selected to fit a plane, and the tolerance range was set to 5 cm according to the undulations of the ground points. Then, the points within the tolerance range from the fitted plane were found [36,37], and the number of internal points δ was counted. The above process was repeated until the δ no longer increased. Finally, the δ of each plane was compared, and the points in the plane with the maximum δ were considered as the actual ground points, as shown in Figure 5.

Due to the overlapping of crop leaves with each other, more and more ground will be covered from the beginning of stem elongation. This resulted in the inability of UAV images to capture these ground parts and thus generate ground points, as shown in Figure 6. Therefore, the missing ground points must be fitted according to the known ground points extracted above. Fitting ground points can prevent the subsequent crop height estimation from being influenced by the ground undulations. Firstly, a point j at the top of the canopy was randomly selected. There was a significant difference in ground height between the two places that are 20 m apart in the study area. So, the nearest 20 ground points within a radius of 20 m of point j in the XY plane were found, as shown in Figure 7a. The number of nearest points was determined by the point cloud density. The weights [38] of these selected ground points were assigned according to their distances from point j in the XY plane. The points closer to point j had a greater impact on point j.

$$W_x=\frac{\frac{1}{R_x^2}}{\sum _{x=1}^{20}\frac{1}{R_x^2}}$$

(1)

W_x is the weight of the xth selected ground point, and R_x is its distance from point j in the XY plane.

The weighted average height H_groundj of these points was calculated as the height of the ground point directly below the crop point j.

$$H_{groundj}=\sum _{x=1}^{20}{W}_x{H}_x$$

(2)

H_x is the height of the xth selected ground point.

So, the difference between the height of point j H_upj and the ground height H_groungj was the crop height at point j.

$$H_j={H_{upj}-H}_{groundj}$$

(3)

By traversing the upper valid points as described above, we fitted all missing ground points and obtained the crop height everywhere, as shown in Figure 7b. It was found that the ground was higher in the north-central part and lower in the surrounding area from the ground points fitted. This result was consistent with the actual ground conditions in the study area, and the effect of ground undulations on crop height estimation was reduced.

2.2.3. Three Different Crop Height Estimation Flows

Overall, the crop height estimation method proposed in this paper is shown in Figure 8. During the experiments, we compared the results of the following three different crop height estimation flows: a The complete method proposed in this paper is performed to estimate crop height. Firstly, the canopy slice filter was applied to remove invalid canopy points and outliers in the point cloud. Then RANSAC was used to extract known ground points after filtering. Finally, we fitted the missing ground points and calculated the crop height. b Without applying the canopy slice filter, ground points are extracted from the original point cloud, and missing ground points are fitted to estimate crop height. c Without processing the point cloud, the crop height is calculated directly with the bounding box. It is built based on the points within a 2 m × 2 m area of the sampling point in the XY plane, and its height is considered as the crop height. From comparing three different estimation results, we can analyze the contribution of the canopy slice filter and fitting ground points to the accuracy of crop height estimation and the necessity of applying them.

To observe and analyze the crop height estimation results of the above three different flows at different growth stages of winter wheat, we calculated (root mean square error (RMSE) and mean absolute error (MAE) of three different flows on each day. RMSE was used to describe the deviation between the estimated and the actual values, and MAE was used to describe the actual situation of the error. In addition, R² and the average estimated height were used to describe the linear fit of three different estimation results during the six days.

3. Results

3.1. Comparison of the Three Different Flows

For the Flow c without point cloud processing, there were outliers in the point cloud generated by SfM. These outliers will affect the crop height estimation when using the bounding box. They were mainly below the actual ground and above the actual crop, as shown in Figure 9a. Thus, these outliers will make the crop height estimation results appear overestimated. So, the height estimation results of most sampling points in Flow c were higher than those in Flow a and Flow b. In special circumstances, all actual ground points were missing in the bounding box, and the lowest point of the bounding box may be a crop canopy point, which reduces the estimated height of the crop.

For Flow b, without applying the canopy slicing filter, we found that the ground plane was always seriously tilted in the process of applying RANSAC. Since we chose the plane whose inner points were the most, the actual ground was covered by crop leaves, and there were few ground points generated based on UAV images. The canopy points near the ground were easily misclassified as ground points and expanded the number of inner points, as shown in Figure 9b. This problem led to an apparent overestimation of the ground elevation and an underestimation of the crop height. When the plants were short, and the leaves were sparse, this situation had a more negligible effect on the extraction of ground points. Where it was especially serious when the plants were tall, and the leaves were dense. However, the canopy slice filtering method we proposed can remove most canopy points near the ground. RANSAC can easily separate the ground points and the crop points. The tilting of the fitted ground plane was effectively avoided, and the ground point extraction accuracy was improved. So, the error of crop height estimation in the complete Flow a with the canopy slice filter is smaller than in Flow b.

3.2. Crop Height Estimation Results on Each Day

3.2.1. Tillering Stage

For the crop height estimation result of Flow c without point cloud processing on May 11 (BBCH = 21), RMSE was 7.9 cm, and MAE was 7.1 cm. The average estimated crop height was 14.1 cm, and its difference from the average measured height (Table 1) was 33.5%.

$$Difference=\frac{\left|{{Average Height}_{measured}-Average Height}_{estimated}\right|}{{Average Height}_{measured}}$$

(4)

For the result of Flow b without canopy slice filtering, RMSE was 11.8 cm, and MAE was 11.0 cm. The average estimated value was 10.1 cm, and the difference was 52.4%. For the result of the complete Flow a, RMSE was 10.9 cm, and MAE was 10.1 cm. The average estimated value was 11.0 cm, and the difference was 48.0%., as shown in Appendix

Table A1. Crop height estimation results of three different flows on each day.

Date	Flow	RMSE	MAE	Average Estimated Height	Average Measured Height	Deviation	Difference
11 May	a	10.9 cm	10.1 cm	11.0 cm	21.2 cm	−10.2 cm	48.0%
	b	11.8 cm	11.0 cm	10.1 cm		−11.1 cm	52.4%
	c	7.9 cm	7.1 cm	14.1 cm		−7.1 cm	33.5%
16 May	a	15.8 cm	15.4 cm	14.1 cm	29.5 cm	−15.4 cm	52.2%
	b	17.2 cm	16.8 cm	12.6 cm		−16.9 cm	57.3%
	c	11.9 cm	11.2 cm	18.3 cm		−11.2 cm	38.2%
21 May	a	3.7 cm	3.0 cm	26.9 cm	29.4 cm	−3.5 cm	8.5%
	b	5.0 cm	4.1 cm	25.4 cm		−4.0 cm	13.6%
	c	13.3 cm	10.5 cm	40.0 cm		+10.6 cm	36.1%
27 May	a	15.9 cm	14.1 cm	39.0 cm	42.6 cm	−3.6 cm	8.5%
	b	10.5 cm	9.6 cm	33.0 cm		−9.6 cm	22.5%
	c	11.6 cm	10.9 cm	53.5 cm		+10.9 cm	25.6%
3 June	a	3.6 cm	2.5 cm	51.9 cm	53.7 cm	−1.8 cm	3.4%
	b	15.7 cm	14.7 cm	39.0 cm		−14.7 cm	27.4%
	c	15.9 cm	14.1 cm	66.5 cm		+12.8 cm	23.8%
11 June	a	7.6 cm	6.9 cm	56.3 cm	62.8 cm	−6.5 cm	10.4%
	b	21.4 cm	19.7 cm	43.2 cm		−19.6 cm	31.2%
	c	18.7 cm	17.4 cm	79.2 cm		+17.6 cm	26.1%

and Figure 10. In Flow c on May 16 (BBCH = 25), RMSE was 11.9 cm, and MAE was 11.2 cm. The average estimated crop height was 18.3 cm, and the difference was 38.2%. In Flow b, RMSE was 17.2 cm, and MAE was 16.8 cm. The average height was 12.6 cm, and the difference was 57.3%. RMSE was 15.8 cm in flow a, and MAE was 15.4 cm. The average estimated height was 14.1 cm, and the difference was 52.2%. We can find that the crop height estimation results of the three different flows were all significantly underestimated when the BBCH of the crop was 21 to 25. In fact, at the tillering stage of the crop, as shown in Figure 11a, most of the ground was not covered by leaves. Therefore, the estimated crop height obtained by the bounding box in Flow c should be maximum and larger than the actual measured value because of the outliers in the point cloud. However, the estimated height in Flow c was significantly smaller than the actual value. We guessed that the canopy points, especially those at the top of the canopy, were missing when the point cloud was generated based on UAV images. Since the leaves of winter wheat at this stage were too slim, the overall crop canopy structure was incomplete, and the crop height estimation results got worse. Our suspicion was verified as the leaves gradually grew and the BBCH reached 31 or more.

3.2.2. Stem Elongation Stage

In Flow c on 21 May (BBCH = 31), RMSE was 13.3 cm, and MAE was 10.5 cm. The average estimated crop height was 40.0 cm, and the difference was 36.1%. In Flow b, RMSE was 5.0 cm, and MAE was 4.1 cm. The average value was 25.4 cm, and the difference was 13.6%. In Flow a, RMSE was 3.7 cm, and MAE was 3.0 cm. The average estimated value was 26.9 cm, and the difference was 8.5%. The average measured height on May 21 was close to that on 16 May because of the wind on 21 May. When we captured the UAV images and measured crop height, the leaves on the top of the crop appeared to fall, as shown in Figure 11b. The estimated height and the measured height were lower than the real crop height, but the lodging of the leaves at the top was equivalent to expanding the leaf area on 16 May. Lastly, since the top of the crop was more obvious in the UAV images, the canopy structure was more complete in the point cloud, avoiding the serious underestimation of crop height.

In Flow c on 27 May (BBCH = 39), RMSE was 11.6 cm, and MAE was 10.9 cm. The average estimated crop height was 53.5 cm, and the difference was 25.6%. In Flow b, RMSE was 10.5 cm, and MAE was 9.6 cm. The average estimated value was 33.0 cm, and the difference was 22.5%. In Flow a, RMSE was 4.3 cm, and MAE was 3.7 cm. The average estimated value was 39.0 cm, and the difference was 8.5%. In Flow c on 3 June (BBCH = 49), RMSE was 15.9 cm, and MAE was 14.1 cm. The average estimated crop height was 66.5 cm, and the difference was 23.8%. In Flow b, RMSE was 15.7 cm, and MAE was 14.7 cm. The average estimated value was 39.0 cm, and the difference was 27.4%. In Flow a, RMSE was 3.6 cm, and MAE was 2.5 cm. The average estimated value was 51.9 cm, and the difference was 3.4%. The estimation results of these two days were consistent with our speculation based on experimental principles. The estimated height in Flow c without ground points fitting was much higher because of the outliers and missing ground points. The extraction of known ground points was poor in Flow b without the canopy slice filtering, which led to underestimating the crop height. Though Flow a had complete processing steps, the estimated height was slightly lower than the actual measured value due to the accuracy of ground point extraction and the quality of point cloud data.

3.2.3. Heading Stage

In Flow c on 11 June (BBCH = 59), RMSE was 18.7 cm, and MAE was 17.4 cm. The average estimated crop height was 79.2 cm, and the difference was 26.1%. In Flow b, RMSE was 21.4 cm, and MAE was 19.7 cm. The average estimated value was 43.2 cm, and the difference was 31.2%. RMSE was 7.6 cm in flow a, and MAE was 6.9 cm. The average estimated value was 56.3 cm, and the difference was 10.4%. From this day on, crop leaves covered almost all of the ground, as shown in Figure 11c. So the number of known ground points extracted in Flow a and Flow b got much smaller, making it hard to fit missing ground points. As a result, the terrain undulations were unable to be simulated, and the accuracy of height estimation was reduced. This problem would become more severe as winter wheat continued to grow.

We analyzed the correlation between the estimated height and the measured height in three different flows based on the data from these six days, as shown in Figure 12. The R² in Flow a reached the maximum of 0.90, and the RMSE and MAE were also the lowest. The RRMSE of height estimation was 5.9%, and the RMAE was 4.6% at the stem elongation stage. The average estimated height in three different flows and the average measured height on each day is as shown in Figure 13. As mentioned above, when winter wheat was at the tillering stage, the estimated height was lower than the measured height. This is because the quality of the point cloud limited it. At the stem elongation stage, the estimated height of Flow a and Flow b was slightly lower than the measured heights due to the extraction accuracy of ground points. Moreover, the bounding box in Flow c contained outliers, so the estimated height is much higher than the measured height. When winter wheat reached the heading stage, there were so few ground points that the ground fitting of Flow a and Flow b became unreliable. Nevertheless, the estimated height of Flow a was still the closest to the measured height. So, fitting ground points and applying the canopy slice filter improved the estimation accuracy of winter wheat height when the BBCH was between 31 and 59. Consequently, the crop height estimation method proposed in this paper was effective.

4. Discussion

4.1. Advantages of the Height Estimation Method

For crop height estimation in a small area, UAV platforms have the advantage of being highly flexible and easy to control. Additionally, the point cloud generated by the SfM algorithm based on UAV images has a similar format to the point cloud generated by LiDAR scanning. But digital cameras cost less than LiDAR when we need the same size as the point cloud. So, there are many studies on calculating crop parameters based on the UAV point cloud. Most of them are based on outlier removal to improve calculation accuracy. However, the neglect of terrain undulations often brings more obvious errors to the estimation of crop height in many planting areas. As a consequence, the method proposed in this paper fits the elevation of missing ground points based on known ground points to estimate the crop height.

For the fitting process, we searched 20 known ground points, which were the closest to the sampling crop point within a radius of 50 m horizontally. Enough reference samples that strongly correlated with the missing ground point were collected. In addition, we assigned weights according to the horizontal distance between each known ground point and the missing ground point. Then, we calculated the weighted average height of these ground points as the ground height below the sampling point. A reliable simulation of ground undulations was achieved in this way.

To extract the real ground points faster in the data, we analyzed the effectiveness of each part of the point cloud. For the crop height, the points at the top of the canopy and near the ground were the valid points. The remaining points were invalid points and outliers. Therefore, we removed the invalid part of the point cloud to reduce the difficulty of extracting ground points and improve the efficiency of the following processing. The canopy slice filtering method proposed in this paper can identify the canopy vertices and the points near the ground based on the density variation of the point cloud. The outliers above the canopy and below the ground were removed to reduce the interference with the process of extracting ground points. In the specific implementation of the filter, we set two kinds of slices with different thicknesses of 5 cm and 10 cm according to the different geometric characteristics of two parts of the valid point cloud. The filtering process was prevented from destroying the structure of the valid point cloud.

For the extraction process of ground points, we only applied RANSAC to the lower part of the valid point cloud. Because some continuous and flat canopy vertices were easily mistaken for ground points occasionally, as shown in Figure 14. Moreover, we expanded the range of sub-areas from 1 m × 1 m to 10 m × 10 m, which avoided the impact of extraction errors in small areas on following fitting processing. As a result, the spatial distribution of ground points would be more continuous.

4.2. Limitations of the Height Estimation Method

When we applied the canopy slice filter, the slices were created with a thickness of 10 cm from the highest point to the lowest point. Though, if there were outliers above the crop, the slices would be created starting with the top outliers, and a small number of canopy vertices might be divided into the same slice with these outliers. The points in this slice were very few, and they were regarded as outliers and removed, as shown in Slice 1 in Figure 15. In other words, the filtering method proposed in this paper had the risk of destroying the canopy structure when outliers appeared above the crop. So, the crop height might be underestimated in some sub-areas. In this paper, the removal of outliers and invalid canopy points was performed simultaneously to save time, but it also made two filtering processes affect each other due to the difference in their size. If the filtering time is not limited, the outliers can be removed first and followed by the invalid canopy points. In this way, we can protect the canopy structure and improve the accuracy of crop height estimation.

So, a detection mechanism was added to the canopy slice filter. If the point number of the top slice N_top was less than N_10cm, the top slice would not be removed immediately. The point density D₂ below the second highest point in the top slice and the average point density D_10cm in the sub-area were calculated, as shown in Figure 16a. If D₂ < D_10cm, we continued to calculate the point density D₃ below the third highest point and compared D₃ and D_10cm, as shown in Figure 16b. The point density below each point in the top slice was calculated until the point density below a certain point D_n was more than or equal to D_10cm, as shown in Figure 16c. If there was a D_n that satisfied the above conditions, we believed that some canopy vertices might be misclassified as outliers. Then, we removed the highest point and performed the filtering process again until the top slice would not be removed, as shown in Figure 16d. On the contrary, if there was no such D_n, all the points in the top slice would be labeled as outliers and removed. Through the detection process, damage to the canopy structure could be effectively avoided.

The extraction of ground points was achieved by applying RANSAC to the lower valid part of the point cloud. To find the plane containing the ground undulations, we need to increase the thickness of the plane. In the specific experiment, we set the thickness to 5 cm because the ground undulation was about 5 cm within the range of 10 m × 10 m. However, a small number of crop points near the ground were mistaken for ground points due to the thickness of the plane. The ground height was fitted higher, and the crop height was estimated lower. For a sub-area, if its area was too large, the ground undulations would require the plane with a larger thickness to contain. There would be more crop points misclassified as ground points as well. If its area was too small, the plane would be more random, and the continuity of ground points would be affected. Therefore, avoiding misclassification and preserving the terrain undulations in the process of ground point extraction needs to be explored in the future, especially when the points near the ground are incomplete in the SfM point cloud.

4.3. Applications of the Height Estimation Method

The crop height estimation method proposed in this paper is mainly based on the fitting of missing ground points to reduce the impact of terrain undulations on crop height estimation. When the BBCH of winter wheat was below 30, the leaves covered a little ground surface in this experiment. The ground points in the point cloud were complete. So, fitting the missing ground points cannot significantly improve the accuracy of crop height estimation at the tillering stage. When the BBCH was above 59, the leaves covered the ground seriously, and UAV images could capture very little ground. Therefore, few ground points could be extracted from the point cloud, and the reliability of fitted ground based on these few ground points worsened. Additionally, the ground undulations were easily ignored, which affected the accuracy of crop height estimation. When the BBCH was between 30 and 59, the leaves covered most of the ground. However, there were still enough widely distributed ground points to simulate the ground undulations. The reliability of fitting the missing ground points was excellent, and the accuracy of crop height estimation could be significantly improved.

The height change of winter wheat in the stem elongation stage is evident. So monitoring the crop height at this stage is helpful to measure the growth rate, analyze the growth status, and predict the final yield of the crop. UAV platforms are more convenient than large-scale satellites and terrestrial lasers. The range and frequency of data collection are no longer limited by space and time, making crop growth monitoring more flexible. In addition to winter wheat, the height estimation method proposed in this article can also be applied to other crops, such as rice and corn. Only the canopy slice filter needs to be adjusted according to the geometric characteristics of the different crop canopy structures. In this method, when the ground fitted is close to the real ground, the terrain undulations have little effect on height estimation. So the process of fitting missing ground points determines the accuracy of height estimation.

5. Conclusions

The crop height estimation method proposed in this paper mainly weakened the effect of ground undulations on height estimation. The crop leaves started to cover the ground from the stem elongation stage, and the ground points in the original point cloud generated based on UAV images became less. For that reason, the crop height estimation would be easily affected by ground undulations and the uneven distribution of the point cloud. According to the difference in point density between the top canopy, the middle canopy, and the ground, the canopy slice filter was first applied to remove outliers and invalid canopy points. Then RANSAC was applied to extract the ground points from the lower part of valid points. The filtering process also optimized the extraction result, and the missing ground points were fitted based on the extracted ground points to calculate the crop height. In the planting areas with terrain undulations, the estimation accuracy in this method significantly improved.

At the heading stage, the leaves of winter wheat became much denser, and there were very few ground points to fit the ground undulations. The accuracy of crop height estimation decreased, and this method gradually became unavailable. On the contrary, the leaves were too slim at the tillering stage. Therefore, the top of the canopy structure was incomplete in the SfM point cloud generated based on UAV images, and the estimated height was lower than the measured height. However, for all growth stages of winter wheat, the height changed most significantly at the stem elongation stage. This method provides a more accurate solution to monitor the crop height at the stem elongation stage. We will also try to optimize the crop height estimation at other stages in future experiments.