Estimation of Vegetable Crop Parameter by Multi-temporal UAV-Borne Images

Abstract

3D point cloud analysis of imagery collected by unmanned aerial vehicles (UAV) has been shown to be a valuable tool for estimation of crop phenotypic traits, such as plant height, in several species. Spatial information about these phenotypic traits can be used to derive information about other important crop characteristics, like fresh biomass yield, which could not be derived directly from the point clouds. Previous approaches have often only considered single date measurements using a single point cloud derived metric for the respective trait. Furthermore, most of the studies focused on plant species with a homogenous canopy surface. The aim of this study was to assess the applicability of UAV imagery for capturing crop height information of three vegetables (crops eggplant, tomato, and cabbage) with a complex vegetation canopy surface during a complete crop growth cycle to infer biomass. Additionally, the effect of crop development stage on the relationship between estimated crop height and field measured crop height was examined. Our study was conducted in an experimental layout at the University of Agricultural Science in Bengaluru, India. For all the crops, the crop height and the biomass was measured at five dates during one crop growth cycle between February and May 2017 (average crop height was 42.5, 35.5, and 16.0 cm for eggplant, tomato, and cabbage). Using a structure from motion approach, a 3D point cloud was created for each crop and sampling date. In total, 14 crop height metrics were extracted from the point clouds. Machine learning methods were used to create prediction models for vegetable crop height. The study demonstrates that the monitoring of crop height using an UAV during an entire growing period results in detailed and precise estimates of crop height and biomass for all three crops (R² ranging from 0.87 to 0.97, bias ranging from −0.66 to 0.45 cm). The effect of crop development stage on the predicted crop height was found to be substantial (e.g., median deviation increased from 1% to 20% for eggplant) influencing the strength and consistency of the relationship between point cloud metrics and crop height estimates and, thus, should be further investigated. Altogether the results of the study demonstrate that point cloud generated from UAV-based RGB imagery can be used to effectively measure vegetable crop biomass in larger areas (relative error = 17.6%, 19.7%, and 15.2% for eggplant, tomato, and cabbage, respectively) with a similar accuracy as biomass prediction models based on measured crop height (relative error = 21.6, 18.8, and 15.2 for eggplant, tomato, and cabbage).

1. Introduction

The ability to non-destructively collect information about crops makes remote sensing a less time consuming and less labor-intensive tool in agricultural science than traditional methods of crop growth monitoring. Unmanned aerial vehicles (UAV) have emerged as a promising remote sensing platform to capture detailed imagery from agriculture crop fields [1]. The ability of UAVs to collect data over wide areas in flexible intervals makes them a superior tool compared to satellite or airborne remote sensing [2]. Especially UAV systems equipped with RGB (red, green, blue) cameras are widely distributed, but systems with other cameras installed (e.g., multi-spectral cameras) are also getting more and more available. Detailed information about crop health [3], crop biomass development [4], and crop water status [5] have been already successfully extracted from UAV remote sensing for various agriculture crops.

Collecting information about phenotypic traits, such as plant height or biomass, at high temporal resolution, is essential for many site-specific management practices or plant physiological studies [6]. Plant height is considered as a good indicator for plant growth and biomass [7,8]. However, the use of classical ground-based height measurements is time consuming, and can rarely be applied non-destructively across larger areas in a repetitive manner [8]. Alternative methods based on LiDAR (light detection and ranging) [9,10], ultrasonic sensors [11], or high resolution RGB imagery [12] have been developed recently. While LiDAR sensors provide highly accurate and dense 3D point measurements of crop surfaces, they are still very expensive, and require specific expertise for handling of sensors and the subsequent analysis of the data [13]. Although less cost intensive alternatives exist (Ehlert 2010), they cannot be used to cover large areas due to their limited mobility. Ultrasonic sensors are considered as low-cost and user-friendly approach, but are often limited by their spatial resolution and their susceptibility to wind [14]. RGB image-based detection of crop height is the most recently evolving approach for many different crops, including barley [12], maize [15], vineyards [16], wheat [17], sorghum [1], or alfalfa [18]. Especially, 3D point clouds generated from UAV-borne RGB images using SfM (structure from motion) techniques offer new options for deriving crop height information [1]. Despite the successful application of 3D point clouds based on UAV imagery and the SfM procedure, many studies have been limited to single acquisition dates or only short periods in the crop growth cycle. In addition, most of the studies concentrate on crops, which are characterized by homogenous crop surfaces. To our knowledge, no study has yet tested the application of UAV-borne RGB imagery for estimating vegetable crop height over an entire cropping season.

The aim of this study, therefore, was to assess the applicability of UAV-borne RGB imagery for capturing crop height information of three vegetable crops eggplant, tomato, and cabbage over an entire cropping season. These three crops are the major vegetable species in the larger area of Bengaluru, Karnataka, India. They represent three different crop growth forms, and canopy shape pattern ranging from a less complex ball shape for cabbage, to a complex multi-layer appearance for tomato and eggplant. The specific objectives of this study were (1) evaluating the ability of UAV imagery and SfM techniques to model the plant height of three vegetable crops over an entire crop growth cycle; (2) monitoring the effect of the crop development on the model prediction quality of crop height values using UAV imagery; (3) quantifying the effect of crop growth form on the crop height models; and (4) evaluating the applicability of the crop height models for estimating the biomass of vegetable crops.

2. Methods

2.1. Study Site

The study was conducted on the experimental farms of the University of Agricultural Sciences (UAS), Bengaluru (12°58′20.79′′N, 77°34′50.31′′E, 920 asl), India (Figure 1A), where Kandic Paleustalfs and Dystric Nitisols are the dominating soil types. The mean annual temperature is 29.2 °C, with an average precipitation of 923 mm, of which most occurs during the monsoon season from June to October [19].

2.2. Experimental Design

The field experiment was established in 2016 with a respective rainy and dry season crop rotation. Within the rainy season (July to October) maize (Zea maize L.) (cultivar: Nithyashree), millet (Eleusine coracana Gaertn.) (cultivar: MR 65), and lablab (Lablab purpureaus L. Sweet) (cultivar: HA 4) were grown, while in the dry season (January to May), tomato (Solanum lycopersicum L.) (cultivar: NS-501), eggplant (Solanum melongena L.) (cultivar: Ankur), and cabbage (Brassica oleracea L.) (cultivar: Unnati) were grown on the experimental site as per the design of Figure 1B. The aim was to include the most commonly grown crops in the greater region of Bengaluru in the field experiment. The selection of crops and cultivars was based on recommendations of scientists at UAS. During the dry season, the crops were irrigated with a drip irrigation system. The split-plot experiment comprised 12 main plots (12 × 18 m), whereby each treatment was replicated four times (Figure 1A). In each main plot, three subplots (6 × 12 m) with different levels of nitrogen (N) fertilizer were randomized. Besides a blanket application of phosphorus and potassium (40 kg P ha⁻¹ and 20, 24, and 50 kg K ha⁻¹ for eggplant, tomato, and cabbage), the mean N rate (50 kg N ha⁻¹ as urea for eggplant, 46 kg N ha⁻¹ for tomato, and 60 kg N ha⁻¹ for cabbage) reflected the recommended N dose in the region, whereas the high (+50%) and low (−50%) dose represented additional intensities typical to the study area. Nitrogen application was split into two dressings to reduce leaching losses by the typically strong rainfalls immediately after fertilizer application. As fertilizers were not distributed evenly on the total plot area like previously planned, but directly applied to individual plants, no nutrient deficiency was noticeable in either treatment. To allow for destructive measurements, each plot was divided equally in a sampling sub-subplot (S-plot; Figure 1B), which was used for all destructive biomass samplings and a harvest sub-subplot (H-plot) for undisturbed crop growth and non-destructive measurements, such as determination of plant height. The S- and H-plots had a size of 6 × 6 m each. The layout thus comprised a total of 36 subplots (3 crops × 3 fertilizer treatments × 4 replicates).

2.3. Plant Sampling and Measurements

To calculate average crop height, the height of 30 plants in each S- and H-plot was measured with a ruler to the nearest 0.1 cm (Figure 2) whereby the distance from the ground to the top most vegetative element was determined. The measurements were conducted at five dates throughout the growing season (

Table A1. Sampling dates for studied crops.

	Eggplant	Tomato	Cabbage
Sampling date 1	8 March 2017	9 March 2017	7 March 2017
Sampling date 2	29 March 2017	30 March 2017	28 March 2017
Sampling date 3	20 April 2017	18 April 2017	10 April 2017
Sampling date 4	16 May 2017	4 May 2017	11 May 2017
Sampling date 5	13 June 2017	5 June 2017	7 June 2017

and Figure A1). To measure total biomass (including fruits when present at later growth stages), three randomly selected plants were harvested 5 cm above ground in all S-plots (Figure 2). At each harvest, the biomass (t ha⁻¹) was weighted in the field and averaged for each plot.

2.4. RGB Imagery Sampling

Prior to destructive sampling, each main plot was measured using a RGB camera mounted on a UAV (DJI 3 Professional, DJI, Shenzhen, Guangdong, China). The camera model used was the standard DJI FC300X with a focal length of 3.61mm. At every sampling date, flights were conducted for each plot using the autopilot and the software Pix4D Capture (Pix4D SA, Lausanne, Switzerland). The flight path was kept constant throughout all sampling dates and for all dates. For capturing the images, a gridwise flight path was defined, for which the area of the main plot was overlaid with a grid of several perpendicular flight lines. The flight delivered nadir images, which were used to generate a 3D point cloud. On average ~200 images each with a resolution of 4000 × 3000 pixels were taken during each flight (Figure 2). The images were taken with an 80% forward and side overlap, whereby the flight altitude was set to 20 m (ranging from 17 m to 24 m, due to variations in the built-in GPS of the drone). The flying height represented a compromise of getting enough images (ranging from 189 to 219) for the point cloud processing and still covering the plot in as little time as possible. The variation in the flying height also leads to variation in the mean ground resolution ranging between 6 and 8 mm per pixel. In total, 60 flights (4 main plots × 3 crops × 5 sampling dates) were conducted. The flights were conducted only on days with no or only low wind, to reduce the effect of moving plants during recording. The spatial position of each image was measured by the internal GPS of the UAV (spatial accuracy ~2 m). However, its accuracy is not sufficient for direct georeferencing. Thus, for each main plot, four permanent ground control points (GCPs) were measured at the corners of the plot using a differential GPS (Trimble Inc., Sunnyvale, CA, USA) with a spatial accuracy of ~5 cm. These GCPs were subsequently used for georeferencing, geo-correction and co-registration of the images.

2.5. Point Cloud Processing

To receive point cloud data from the UAV images, a SfM algorithm was applied (Figure A1). The first step of the algorithm is to extract features in each image that can be matched to their corresponding features in other images [20,21]. Prior to this matching step, the SfM performs a bundle adjustment among the images, based on matching features between the overlapped images to estimate interior and exterior orientation of the onboard sensor. Applying a multi-view stereo matching algorithm to the images, a dense point cloud is generated [22,23]. All images were processed using the software Photoscan Pro (AgiSoft LLC, St. Petersburg, Russia). The bundle adjustment was conducted with the high accuracy settings and a key point limit of 100,000 and a tie point limit of 1000. The resulting sparse point cloud was georeferenced using the GCPs to ensure precise alignment and georeferencing of the point cloud. The RMS of the projections ranged from 0.2 to 0.4. All GCPs were manually identified in the raw images, and their coordinates were fed into the Photoscan software. The spatial error of the GCP for all datasets ranged from 5–20 cm. The accuracy decreased towards the end of growing season, due to difficulties in identifying the GCP due to overgrowing (Figure 2). Subsequently, the dense point cloud was processed using the “medium quality” setting. Although higher quality settings were possible, the medium quality was selected to reduce processing time. Depth filtering allows removing outliers within the point cloud and was set to “medium” following [24]. Afterwards, the point clouds were divided into corresponding S- and H-plot.

2.6. Ground Classification

Besides the x-, y-, z-coordinates, each point is associated with a red, green, and blue color information. This color information was used to calculate the Green-Red Vegetation Index (GRVI, Equation (1); [25]). The GRVI has been shown to be efficient in spectrally discriminating vegetation from soil [26].

$$\mathrm{GRVI}=\frac{\left(\mathrm{Green}-\mathrm{Red}\right)}{\left(\mathrm{Green}+\mathrm{Red}\right)}$$

(1)

with Green and Red indicating the green and red color information for all points of the point cloud. Low GRVI values are characterized by small differences between Green and Red colors, which is typical for soil. Vegetation, on the other hand, is characterized by larger differences between the Green and Red color values, which would lead to higher GRVI values. Using an unsupervised k-means algorithm, all points were classified into vegetation or ground, based on the GRVI values. For this, the k-means algorithm classified all points into k classes (here, k = 2) based on the smallest distance of the points to the mean of the class centroid. This classification was done for all plots and sampling dates individually. The classified ground points were subsequently used for the creation of a digital elevation model (DEM) using Delaunay triangulation (Figure 2). The DEM had a resolution of 1 m, which was a compromise between available ground points and necessary interpolation. To normalize the height information of the vegetation points, from each z-value, the corresponding DEM value was subtracted. This allowed for calculation of the height of each vegetation point above ground surface (Figure 2). The resulting average point density of the vegetation point cloud was 955 points m⁻² and 988 points m⁻² for the S- and H-plot, respectively. As plant individuals were continuously removed from the S-plot during the growing season, the total standing biomass in the S-plot steadily decreased. This, however, did not affect the presented approach, as the non-vegetated areas were removed using the GRVI values from the drone imagery, and so the average crop height was only based on actual existing plant individuals. For each plot and sampling date, 14 different point cloud metrics were calculated (

Table 1. Height metrics derived from point clouds and used as explanatory variables in modelling crop height.

Metric	Description
Hmin	Minimum crop height
Hmax	Maximum crop height
Hmean	Mean crop height
Hsd	Standard deviation of crop height
Hmedian	Median crop height
Hskew	Skewness of crop height
Hkurt	Kurtosis of crop height
Hcv	Coefficient of variation crop height
Hq70	70th percentile of crop height
Hq80	80th percentile of crop height
Hq90	90th percentile of crop height
Hq95	95th percentile of crop height
Hq99	99th percentile of crop height
Hrelief	Crop canopy relief height (Hmean-Hmin)/(Hmax-Hmin)

). For each height metric only, the points classified as vegetation were used for calculation (

Table A2. Values for mean and standard deviation (brackets) of parameter used in the present study. The values for the height metrics and the measured canopy height are based on the values from the S- and H-plot (n = 24). The biomass information is based on the values from the H-plot (n = 12).

Crop	Sampling Date	Measured Crop Height	Biomass [kg m−2]	Hmin	Hmax	Hmean	Hsd	Hmedian	Hskew	Hkurt	Hcv	Hq70	Hq80	Hq90	Hq95	Hq99	Hrelief
Cabbage	1	5.856 (0.379)	0.028 (0.028)	0.001 (0)	0.173 (0.417)	0.017 (0.050)	0.028 (0.107)	0.005 (0.001)	3.669 (3.537)	46.453 (97.946)	0.989 (0.437)	0.042 (0.145)	0.066 (0.248)	0.081 (0.306)	0.089 (0.316)	0.109 (0.329)	0.099 (0.056)
	2	11.936 (1.020)	0.139 (0.139)	0.001 (0)	0.168 (0.508)	0.012 (0.012)	0.02 (0.057)	0.007 (0.001)	1.742 (0.673)	3.857 (3.906)	1.005 (0.688)	0.026 (0.039)	0.034 (0.056)	0.051 (0.116)	0.080 (0.234)	0.108 (0.328)	0.132 (0.035)
	3	14.947 (1.061)	0.446 (0.446)	0.001 (0)	0.198 (0.446)	0.017 (0.007)	0.021 (0.042)	0.013 (0.001)	1.685 (1.356)	6.123 (12.136)	0.933 (0.737)	0.035 (0.022)	0.042 (0.034)	0.055 (0.063)	0.075 (0.130)	0.132 (0.322)	0.153 (0.052)
	4	22.296 (0.892)	2.746 (2.746)	0.001 (0)	0.235 (0.374)	0.035 (0.005)	0.032 (0.031)	0.029 (0.002)	1.132 (1.417)	2.738 (12.325)	0.854 (0.462)	0.070 (0.016)	0.080 (0.024)	0.095 (0.044)	0.114 (0.086)	0.177 (0.298)	0.207 (0.047)
	5	24.967 (1.029)	4.972 (4.972)	0.001 (0)	0.320 (0.643)	0.035 (0.004)	0.030 (0.022)	0.029 (0.002)	1.283 (1.885)	5.397 (22.835)	0.825 (0.401)	0.069 (0.011)	0.078 (0.015)	0.093 (0.027)	0.108 (0.052)	0.166 (0.233)	0.186 (0.054)
Eggplant	1	8.744 (0.883)	0.015 (0.015)	0.001 (0)	0.445 (1.299)	0.006 (0.002)	0.014 (0.033)	0.005 (0.001)	4.546 (6.417)	75.084 (171.526)	1.631 (2.860)	0.011 (0.005)	0.013 (0.007)	0.018 (0.016)	0.021 (0.017)	0.076 (0.174)	0.095 (0.056)
	2	20.020 (1.736)	0.099 (0.099)	0.001 (0)	0.100 (0.013)	0.018 (0.004)	0.016 (0.003)	0.012 (0.003)	1.321 (0.248)	1.451 (1.016)	0.928 (0.067)	0.038 (0.007)	0.044 (0.008)	0.053 (0.009)	0.060 (0.009)	0.074 (0.009)	0.168 (0.030)
	3	44.833 (3.433)	0.727 (0.727)	0.001 (0)	0.456 (0.260)	0.042 (0.004)	0.034 (0.003)	0.035 (0.004)	1.450 (0.493)	6.031 (9.016)	0.812 (0.038)	0.084 (0.008)	0.095 (0.009)	0.114 (0.010)	0.132 (0.010)	0.176 (0.016)	0.112 (0.047)
	4	68.798 (4.790)	2.239 (2.239)	0.001 (0)	0.326 (0.103)	0.074 (0.039)	0.049 (0.015)	0.066 (0.042)	0.877 (0.395)	0.894 (1.329)	0.719 (0.108)	0.134 (0.056)	0.149 (0.058)	0.171 (0.061)	0.191 (0.062)	0.229 (0.063)	0.226 (0.097)
	5	70.315 (4.778)	1.902 (1.902)	0.001 (0)	0.271 (0.042)	0.054 (0.010)	0.042 (0.007)	0.045 (0.009)	1.100 (0.167)	1.185 (0.719)	0.786 (0.036)	0.107 (0.019)	0.121 (0.021)	0.144 (0.024)	0.164 (0.026)	0.204 (0.030)	0.199 (0.033)
Crop	Sampling Date	Measured Crop Height	Biomass [kg m−2]	Hmin	Hmax	Hmean	Hsd	Hmedian	Hskew	Hkurt	Hcv	Hq70	Hq80	Hq90	Hq95	Hq99	Hrelief
Tomato	1	9.249 (0.592)	0.014 (0.014)	0.001 (0)	0.088 (0.187)	0.008 (0.011)	0.011 (0.033)	0.005 (0)	2.253 (1.805)	14.877 (22.386)	0.833 (0.408)	0.018 (0.035)	0.022 (0.046)	0.035 (0.102)	0.044 (0.132)	0.059 (0.162)	0.129 (0.056)
	2	24.544 (2.504)	0.050 (0.050)	0.001 (0)	0.131 (0.240)	0.009 (0.006)	0.013 (0.025)	0.006 (0.001)	2.852 (0.848)	13.540 (9.779)	1.079 (0.576)	0.020 (0.018)	0.024 (0.026)	0.037 (0.054)	0.054 (0.106)	0.084 (0.169)	0.091 (0.027)
	3	43.651 (4.829)	0.360 (0.360)	0.001 (0)	0.500 (0.534)	0.033 (0.006)	0.036 (0.007)	0.021 (0.006)	2.880 (4.579)	69.698 (311.362)	1.078 (0.128)	0.076 (0.012)	0.090 (0.014)	0.113 (0.020)	0.137 (0.031)	0.192 (0.064)	0.089 (0.030)
	4	53.727 (5.782)	1.140 (1.140)	0.001 (0)	0.611 (0.374)	0.060 (0.030)	0.061 (0.038)	0.038 (0.011)	1.718 (0.403)	4.884 (3.454)	0.974 (0.126)	0.136 (0.080)	0.160 (0.097)	0.198 (0.120)	0.232 (0.137)	0.306 (0.159)	0.108 (0.033)
	5	46.474 (3.091)	0.827 (0.827)	0.001 (0)	0.305 (0.134)	0.017 (0.005)	0.021 (0.010)	0.010 (0.002)	4.061 (1.885)	37.573 (41.772)	1.220 (0.213)	0.039 (0.013)	0.048 (0.018)	0.065 (0.029)	0.084 (0.045)	0.139 (0.075)	0.060 (0.023)

). The selection of the metrics was based on the results from studies focusing on forest biomass estimation [27,28]. The extraction and handling of the point cloud was done using the lidR package [29] of the program R [30].

2.7. Statistical Methods

In a first step, the effect of fertilizer treatment and sampling date on the measured crop height was tested using ANOVA. A significant effect of the fertilizer would imply that the crop height modelling should be done separately for each N level. To predict crop height, two machine learning methods were used, namely random forest regression (RFR) [31] and support vector regression (SVR) [32]. Both methods were chosen because they can handle the high intercorrelation of the point cloud metrics (Table 1). Partial least squares regression (PLSR), another common machine learning method, was also tested for their predictive abilities. However, PLSR did not outperform RFR and SVR, and was thus removed from the further presentations. For model calibration, only point clouds from the S-plot were used, while for validation of the model prediction quality, the point clouds from the H-plot were used. The RFR and SVR calculations were done using the packages randomForest [33] and e1071 [34] in R [30].

To evaluate model prediction quality, the explained variance using the pseudo-R² (Equation (2) was used, as well as the root mean square error (RMSE) of the predicted values, that is, values from the H-plot (Equation (3). Additionally, the bias was calculated (Equation (4). To evaluate the prediction performance for each sampling date, the deviation of the predicted values from the measured height values, scaled by the mean of the measured height values, were used. For the comparison of the models, also, the relative error (rel.err.) was calculated (Equation (5). To assess the applicability of the predicted crop height as an estimator of biomass, a linear regression was calculated between the predicted crop height and the biomass for the H-plot. The model prediction quality was evaluated using the R² and RMSE.

$$Pseudo-R^2=\left[1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(y_i-{\overline{y}}_i\right)}^2}\right]$$

(2)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}{n}}$$

(3)

$$bias=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left(y_i-{\widehat{y}}_i\right)$$

(4)

$$rel.err.=\frac{RMSE}{\max \left(y_i\right)-\min \left(y_i\right)}$$

(5)

with $y_i$ being the measured crop height values, and ${\widehat{y}}_i$ the predicted crop height values. ${\overline{y}}_i$ indicates the average measured crop height, and n is the number of samples.

3. Results

To test for the combined effects of sampling date and N-fertilizer on the average crop height, a two-factorial ANOVA was conducted (

Table 2. Effects of sampling date and N fertilizer on average crop height for eggplant, tomato, and cabbage grown at UAS Bangalore, India.

	p-Value
	Sampling Date (SD)	N Fertilizer (NF)	SD × NF
Eggplant	<0.001	0.141	0.453
Tomato	<0.001	0.978	0.720
Cabbage	<0.001	0.454	0.691

). Prior to the ANOVA, the values were tested for normality of residuals and homoscedasticity, and if necessary, were transformed (results not shown). For all three crops, the interaction term of sampling date and N fertilizer was not significant, while the effect of sampling date was highly significant. This is not surprising, as it was assumed that crop height increases similarly with increasing crop age at all N levels, whereby as stated earlier, the range of occurring N supply to crops was limited.

3.1. Crop Height Estimation

To predict mean crop height, 14 point cloud-derived crop height metrics were used (

Table 3. Correlation among the point cloud derived crop height metrics.

	Hmin	Hmax	Hmean	Hsd	Hmedian	Hskew	Hkurt	Hcv	Hq70	Hq80	Hq90	Hq95	Hq99
Hmax	0.01
Hmean	0.38	0.36
Hsd	0.1	0.67	0.75
Hmedian	0.53	0.15	0.83	0.33
Hskew	−0.06	0.64	−0.19	0.13	−0.25
Hkurt	−0.02	0.55	−0.08	0.07	-0.09	0.84
Hcv	−0.04	0.82	0.01	0.42	-0.14	0.7	0.45
Hq70	0.24	0.43	0.97	0.86	0.67	−0.14	−0.06	0.08
Hq80	0.18	0.43	0.91	0.91	0.53	−0.1	−0.05	0.11	0.98
Hq90	0.15	0.49	0.86	0.95	0.45	−0.05	−0.04	0.19	0.95	0.98
Hq95	0.12	0.55	0.79	0.97	0.39	0.01	−0.02	0.27	0.89	0.91	0.97
Hq99	0.07	0.74	0.62	0.92	0.29	0.23	0.1	0.51	0.71	0.73	0.81	0.9
Hrelief	0.34	−0.33	0.38	−0.04	0.57	−0.57	−0.3	−0.35	0.24	0.16	0.08	0.01	−0.14

). The correlation among the metrics ranged from 0.01 to 0.98, with an average intercorrelation of 0.45, indicating a moderate intercorrelation (Table 3). Thus, regression methods, which can handle intercorrelations among the explanatory variables, like RFR or SVR, should be preferred to classical ordinary least squares regression models.

In Figure 3, the measured average crop height values versus the predicted crop height values are shown for: (left) eggplant, (middle) tomato, and (right) cabbage. Two machine learning methods were tested: (top) random forest regression, and (bottom) support vector regression. For RFR, the best model was acquired for cabbage (pseudo-R² = 0.97, RMSE = 1.3, bias = 0.01). For eggplant and tomato the results were similarly good (eggplant: pseudo-R² = 0.93, RMSE = 6.86, bias = 0.14, tomato: pseudo-R² = 0.89, RMSE = 5.49, bias = 0.45) (Figure 3). For SVR, the results were always weaker than for RFR, with a pseudo-R² for eggplant, tomato, and cabbage of 0.91, 0.87, and 0.89, respectively. The RMSE and the bias, on the other hand, showed larger errors for the SVR models (eggplant: RMSE = 7.36, bias = −0.66, tomato: RMSE = 5.91, bias = 0.91, and cabbage: RMSE = 2.31, bias = −0.24).

3.2. Crop Height Deviation within the Growing Season

To check whether the predictability of the average crop height varied between RFR and SVR along the growing season, the deviation of the predicted from the measured crop height was calculated (Figure 4). For eggplant the median deviation was between 1–20% for all sampling dates for both RFR and SVR. An increase of the relative deviation was visible from sampling date 1 to 5, except for sampling date 3, for which both methods overestimated the measured average crop height. For tomato, the relative deviation shows no clear pattern along the sampling dates for both RFR and SVR. The median deviation for RFR was lowest for sampling date 1 (−0.009) and highest for sampling date 4 (0.16). For SVR, the fluctuation shows a higher amplitude than for RFR, ranging from 0.06 to −0.19. For cabbage, the deviation was lowest for RFR across all sampling dates, ranging from −0.008 in sampling date 2, to 0.10 for sampling date 5. For SVR, the highest deviation was found for sampling date 1 (−0.2). For the other sampling dates, the deviation is comparable to the results from the RFR.

3.3. Crop Biomass Estimation

To test whether the predicted crop height values are reliable estimators for the biomass of the three crops, a log-linear model was calibrated with the predicted crop height values as predictors and the biomass of the H-plot as dependent variable (Figure 5). The predicted crop height values were based on RFR. The models for all three crops showed significant relationships with R² values from 0.90 (RMSE = 6.01 t ha⁻¹) for eggplant, 0.88 (RMSE = 3.76 t ha⁻¹) for tomato, and 0.95 (RMSE = 9.97 t ha⁻¹) for cabbage. In order to compare the model based on the predicted crop height with a model based on the measured crop height, a log-linear model was also calibrated with the measured average crop height (Figure 5 and Figure 6). For eggplant, the model on the measured crop height showed a higher relative error (rel.err. = 21.6%) than the model based on the predicted crop height. For tomato the model based on the measured crop heights showed a slightly lower error of 18.3% and for cabbage both models the relative error is the same (rel.err. = 15.2%).

4. Discussion

The aim of the current study was to assess the applicability of UAV-borne RGB imagery for capturing crop height information for the three vegetable crops (eggplant, tomato, and cabbage), and to assess whether the developed methods can be used over an entire cropping season. Overall, results indicate that a successful estimation of crop height for vegetable crops with very different crop growth forms could be achieved (with R² value ranging from 0.89 for tomato to 0.97 for cabbage). These results are similar to multi-temporal crop height models for maize and sorghum (ranging from 0.68 to 0.78) [1], although, in contrast to this study, no systematic bias according to the vegetable growth stage could be found.

Our experiment shows that UAV-borne RGB imagery in combination with SfM techniques can be used to estimate vegetable crop height during the complete growing period. The suggested method using several extracted crop height metrics from 3D point data was used previously in forestry research [27], and in agroecological research [35]. For example, [35] used a group of point cloud derived height metrics to estimate variations of maize crop height at landscape scale. The advantage of this method in comparison to the more frequently used single best metric approach is that the information content about crop height differs for each single metric, as can be seen in the correlation among all metrics (Table 3). This indicates that multiple crop height metrics predict biomass information more comprehensively than single crop height metrics. Modern machine learning methods can handle highly intercorrelated variables [36] better than ordinary least squares regression models, and should therefore be the tool of choice for the prediction of crop height based on multiple crop height metrics from UAV-borne RGB imagery. The results showed no clear superiority of RFR over SVR, although RFR always yielded slightly better results (Figure 3). The slightly better performance might be explained by the lower sensitivity to data skewness and to model overfitting [37].

UAV-borne RGB imagery-based crop height values indicate a systematic deviation from the field measured crop height values (Figure 4). The apparent increase in the median bias, from early growing stages to the late stages for eggplant and cabbage, indicates an increasing overestimation of the vegetable crop height by the prediction model. A similar trend is evident in [1] for maize and sorghum. They state that the deviations are mainly due to time lags between the UAV-based measurements and the field-based measurements. For the current study, the sampling date was identical for both UAV and field measurements, indicating that other factors might be the reason for the deviation. The biased estimation is more likely a result from inaccuracies in the point cloud generation process and is, thus, likely to be related to a biomass increase during the crop development, which could complicate the SfM processing [38]. For tomato crop, this trend in bias could not be found, but in contrast to the other two crops tomato was tied up to horizontal threads from sampling date 3, to prevent molding of the fruits. This agricultural practice might also be the reason for the high bias values for tomato at sampling date 4 (Figure 4). In addition to the deviations in estimated crop height from the SfM algorithm, the field-based height measurements are not bias-free. For the present study, the maximum height of any vegetation part at each measurement point was used for the height measurements, which could lead to an overestimated average plot crop height.

The three vegetable crops examined in the present study represent different growth forms with different internal vegetation structures. While the cabbage is characterized by a relatively flat and round shape with compact structure, tomato and eggplants are growing taller, which is characteristic for more complex structures. In comparison to tomato, eggplant creates larger leaves and thicker stems. These vegetation characteristics lead to differences in the biomass and vegetation structure, particularly at the end of the growing season (Figure 5), and could have affected the vegetable crop height estimation [39]. Despite these variations, no difference in model prediction quality was found (Figure 3 and Figure 4). These results agree with the results from [39], who have shown that, with high spatial resolution UAV-borne RGB imagery, even small structural details can be successfully differentiated.

Crop height is considered as an important indicator for biomass of crops, such as maize [35], barley [12], sorghum [1], and poppy [40]. The predicted vegetable crop height values of the present study show strong and highly significant relationships to the biomass for all three crops (Figure 5). The model prediction quality (measured as R²) was better in the present study than in the study by [36], who estimated maize aboveground biomass using UAV-borne RGB imagery. However, in addition to being only a single date data [35], the flying height of the UAV used by [35] was much higher (130 m) than the flying height (20m) maintained in this study, leading to manifold differences in the spatial resolution of imagery. In another similar study, [41] found a comparably strong relationship between UAV-borne RGB derived from multi-temporal crop height and barley dry matter yield (R² = 0.85). The accuracies of the models based on the predicted crop height showed similar prediction qualities than the corresponding models based on field-based crop height values (Figure 5). This supports the idea that point cloud analysis could replace intensive field campaigns for measuring crop height with a similar prediction accuracy for the crop biomass. Both modelling approaches showed clear deviations for the late growing stages (growing stage 4 and 5; Figure 6). These deviations indicate that height is a less accurate estimator of biomass for crops in late growing stages. The results of the present study support the finding of other studies that high resolution multi-temporal crop height information is needed for successful estimation of crop biomass [10,41]. Alternative plant phenotypic traits, such as plant volume, may even yield more accurate predictors for biomass [40]. Thus, future studies should evaluate the further potential of 3D point cloud analysis for estimating key plant phenotypic traits, which can be used as reliable predictors for plant biomass. Furthermore, the results indicate that for a successful estimation of biomass, height information alone is not enough. Evolving sensor fusion approaches might improve the model prediction performance, as it has been shown for example for grassland biomass [42,43].

5. Uncertainties, Errors, and Accuracies

Disturbances of vegetation canopy through environmental conditions (e.g., wind, clouds) can affect point cloud creation and lead to inaccuracies during the point cloud processing. While wind can move the vegetation canopy and, thus, may lead to a distortion of height, clouds can lead to differences in the RGB values which were used for ground classification. However, as the average canopy height was used, we assume these effects are minor, and no UAV flight was conducted during high wind conditions.

For a successful height estimation, a highly precise georeferencing is mandatory. Four GCPs (as used in this study) represent the minimum number of control points for georeferencing. More GCPs could have led to a more accurate representation of the actual crop height. To keep the relative error between the measurements constant, permanent GCPs were used.

Accurate estimation of height for each vegetation point needs an accurate DEM model. While at the beginning of the growing season, plenty of space among individual plants allows for identifying enough ground points for an accurate calculation of the DEM, much of the ground is covered by vegetation at later crop growth stages, which reduces the amount of ground-classified points. This problem is not prominent for cabbage, as there was always enough space between the plant individuals, whereas tomato and eggplant developed a dense vegetation cover with progressing growing season.

Although the accuracies of the biomass prediction models based on the estimated crop height were similar to the predicted biomass based on the measured crop height, circa 10% of the variation in the biomass values remained unexplained. The relationship between height and biomass is exponential, indicating that large values of vegetation height are not a stable estimator of biomass anymore. A reason for this might be that the linear relationship between biomass and crop height is only valid at the beginning of the growing season, whereas with progressing growing season the crop biomass further increases, which is rather due to the growth of fruits than due to height increase.

6. Conclusions

High resolution subplot or level estimation of crop phenotypic traits from UAV-borne RGB imagery is fast emerging as one time and cost effective remote sensing tool for agricultural crops. The processing and analysis approach developed in this work, based on the extraction of several height parameter from the 3D point clouds and machine learning regression methods, exhibit stable model prediction quality for biomass during the entire growing period and for the three vegetable crops namely, eggplant, tomato, and cabbage. However, the results also show that crop height information, based on UAV imagery analysis, is affected by vegetable crop internal structure and agricultural management. Further research is needed to examine the effect of other crop phenotypic traits on model prediction quality. Overall, the study demonstrated that time-consuming manual height measurement can be replaced by remote sensing approaches for field vegetable crops, and recommend further multi-site and multi-crop studies to better understand the temporal variation between field measured crop height and crop height, based on UAV-borne RGB-based 3D point clouds. In the future, the suggested approach could be used for a real-time evaluation of vegetable biomass.