Editorial for the Special Issue “Estimation of Crop Phenotyping Traits using Unmanned Ground Vehicle and Unmanned Aerial Vehicle Imagery”

Abstract

High-throughput crop phenotyping is harnessing the potential of genomic resources for the genetic improvement of crop production under changing climate conditions. As global food security is not yet assured, crop phenotyping has received increased attention during the past decade. This spectral issue (SI) collects 30 papers reporting research on estimation of crop phenotyping traits using unmanned ground vehicle (UGV) and unmanned aerial vehicle (UAV) imagery. Such platforms were previously not widely available. The special issue includes papers presenting recent advances in the field, with 22 UAV-based papers and 12 UGV-based articles. The special issue covers 16 RGB sensor papers, 11 papers on multi-spectral imagery, and further 4 papers on hyperspectral and 3D data acquisition systems. A total of 13 plants’ phenotyping traits, including morphological, structural, and biochemical traits are covered. Twenty different data processing and machine learning methods are presented. In this way, the special issue provides a good overview regarding potential applications of the platforms and sensors, to timely provide crop phenotyping traits in a cost-efficient and objective manner. With the fast development of sensors technology and image processing algorithms, we expect that the estimation of crop phenotyping traits supporting crop breeding scientists will gain even more attention in the future.

1. Introduction

Under changing climatic conditions, global food security is challenged. Crop production needs to be increased urgently, while coping with limited resources and plant stresses. Currently, crop breeding as the central pillar for yield increases and resource efficiency, is limited by breeding efficiency, and phenotyping selection [1]. Phenotyping is defined as the application of methodologies and protocols to measure a specific trait, ranging from the cellular level to the whole plant or canopy level, related to plant structure, biochemicals, and function [2,3]. High-throughput crop phenotyping is receiving increased attention for its potential to harness genomic resources in the genetic improvement of crop production under changing climate conditions.

Traditional phenotyping traits measurements are done manually with a lot of professionals’ effort, time, and resources. Remote sensing techniques are complementary to this field work and offers estimation of high-throughput phenotyping traits, in particular, taking into account the maturity of unmanned ground vehicle (UGV), unmanned aerial vehicle (UAV) technology, as well as micro-sensors and advanced sensors such multi/hyperspectral, thermal infrared, LiDAR sensors etc. Many experts in remote sensing and plant physiology have recognized the value of UGV/UAV remote sensing phenotyping, given their high abundant spectral, spatial, and temporal information [4,5]. The potential advantages of UAV remote sensing are not only greatly enhancing the efficiency of data acquisition, at the same time, data standardization is becoming easier, thereby reducing personal subjective assessments. Moreover, image processing and machine learning algorithms are making good progress, including advancements in physically based radiative transfer models. Advances are also made in data preprocessing, system or platform testing, and modern machine learning algorithms. All these elements are critical to access the target traits by raw data [6].

This Special Issue aims to contribute the latest innovative research results in the field of remote sensing technology, senor technologies, and image processing algorithms and also provides a number of specific applications specifically addressing issues estimating specific crop phenotyping traits based on UGV and UAV images.

2. Overview of Contributions

The contributions published in this SI clearly demonstrate the added value that phenotyping provides for various plants (Figure 1) that were monitored by multi-sourced sensors deployed on UGV and UAV platforms. In particular, we wish to highlight the high number of target traits that were addressed by machine learning and image segmentation algorithms (

Table 1. Estimation of crop phenotyping traits in this Special Issue. The table distinguishes between crop type, phenotyping platforms, sensors, and methods.

No.	Crop	Traits	Platforms	Sensors	Methods	Reference
1	Wheat	Seed emergence uniformity	UAV	RGB	ALA	[7]
2	Blueberry	Height, extents, canopy area, volume; crown diameter and width	UAV	RGB	Motion algorithms	[8]
3	Wheat	LAI	UAV	MSI	OLS	[9]
4	Wheat	AGB	Ground	Hyperspectral	ANN, MLR, DT, BBRT, PLSR, RF, SVM, PCR	[10]
5	Wheat	Spikes	Field-based phenotype platform	RGB; MSI	MES	[11]
6	Barley	Fresh/dry Biomass	Elevated position	RGB	CSM	[12]
7	Maize	CC; Senescence	UAV	RGB	Senescence Index	[13]
8	Maize	Yield	Ground; UAV	RGB; MSI	MLR	[14]
9	Apple Tree	LCC; 3D reconstruction	Ground fixed	3D laser scanner	ANN	[15]
10	Sugar Beet	Beet cyst nematode; yield	Handheld and UAV	Hyperspectral; Thermal images; HSI	PCR; DT	[16]
11	Eggplant, Tomato, Cabbage	Height; biomass	UAV	RGB	RF; OLS	[17]
12	Wheat	Senescence Rate	UAV	MSI	Correlation	[18]
13	Tree	Crown perimeter; width; height; area; CC	UAV	MSI	Image segmentation	[19]
14	Wheat	Height; vigor	Ground; UAV	RGB	-	[20]
15	Wheat	Height; LAI; AGB	UAV	HSI; RGB	RF; PLSR;	[21]
16	Soybean	Height; greenness index	Ground	Photonic mixer detector; RGB	DBSCAN; PCR; ICP	[22]
17	Oilseed Rape	Flower number	UAV	MSI; RGB	RF; OSR	[23]
18	Potato	Late blight severity	UAV	MSI	MLP, SVR, RF, ANN	[24]
19	Mazie	Lodging	UAV	MSI	NC	[25]
20	Cotton	WUE, FVC	UAV	MSI	ET model	[26]
21	Maize	AGB	UAV	RGB + point cloud	CSM; PLSR	[27]
22	Cotton	Cotton bolls; yield	UAV	RGB	Automatic open cotton boll detection algorithm	[28]
23	Maize	Height	UAV	RGB; LiDAR	CSM	[29]
24	Maize	Leaf length; width; inclination angle; azimuth; area; height	Ground	3D laser scanning; 3D digitizing	--	[30]
25	Avocado tree	Crown height; extent; CC	UAV	MSI	CSM; OLS; RF	[31]
26	Blueberry	Stem water potential; Cab; fluorescence; leaf gas exchange	Ground	Hyperspectral	MLR; PLSR	[32]
27	Barely	Height; lodging percentage; severity	UAV	RGB	SfM	[33]
28	Rice	CC	UAV	MSI	OLS	[34]
29	Cotton	Height	Ground; UAV	Nadir/Multi-Angle View Sensor	--	[35]
30	Soybean	Height; breadth; color	Ground	RGB-D		[36]

Note: Traits: LAI, leaf area index; AGB, above ground biomass; CC, canopy cover; FVC, fractional vegetation cover; LCC, leaf chlorophyll content; WUE, water use efficiency; C_ab, leaf chlorophyll a. Platforms: UAV, unmanned aerial vehicles. Sensors: RGB, red-green-blue imagery; MSI, multi-spectral imagery; HSI, hyperspectral imagery; RGB-D, RGB and depth imagery. Methods: ALA, area localization algorithm; ANN, artificial neural network; MLR, multivariable linear regression; DT, decision-tree regression; BBRT, boosted binary regression tree; PLSR, partial least squares regression; RF, random forest regression; SVM, support vector machine regression; PCR, principal component regression; MES, Maximum entropy segmentation; CSM, crop surface models; OLS, ordinary least squares; DBSCAN, density-based spatial clustering of applications with noise; ICP, iterative closest point; OSR, optimal subset regression; MLP, multilayer perceptron; NC, nomogram computation; SfM, structure from motion.

).

2.1. Platforms and Sensors

High-throughput phenotyping is currently mostly based on remote sensing, or more specifically, near ground platforms: actual ground and aerial platforms. Twelve papers in this special issue reported results from ground platforms, and twenty-two papers used the UAV platform.

For ground platform, ground fixed scanning system [15,22,30,35], handheld-based field measuring [11,14,16,32], mobile ground platform (MGP) [14,20], and lifting hoist-based elevated platform [12,36] were reported for different crops types (Figure 2). These platforms were easy-to-use with low cost, but data acquisition was semi-automatic.

The recent development and increasing acceptance of UAV or drones in terms of cost and reliability has made data collection much more efficient with unprecedented spatial, spectral, and temporal detail, thereby supporting their application in phenotyping traits mapping [6]. The UAV remote sensing improves the efficiency of crop phenotyping traits data acquisition. It allows high-throughput measurements of canopy structure (e.g., crop height, leaf angle, leaf area, etc.,) in more than 1000 plots within more or less one day by RGB stereo imagery. This is clearly far more efficient compared to traditional manual measurement. Khan et al. [20] also compared the estimating accuracy based on mobile ground platform (MGP) and UAV platform, and showed that the estimated canopy height derived from MGP imagery were better than UAV imagery, while the opposite results were obtained regarding the estimating of canopy vigor.

Besides the platforms, sensors play a very important role in advance phenotyping. Lightweight sensors can be loaded in the UAV platform improving their data capture quality. In this special issue, applications based on RGB and multi-spectral imagery cameras, hyperspectral sensors, thermal cameras, and light detection and ranging (LiDAR) sensors are shown (Figure 3). Up to now, RGB imagery for classification or segmentation and multi-spectral imagery cameras for biochemical or physical traits are most prominently exploited. Hyperspectral sensors and LiDAR loaded on UAV platforms are recommended to be further exploited in the future.

2.2. Phenotyping Traits

For this Special Issue, phenotyping traits are broadly classified into two types: morphological/structural traits and biochemical traits (Figure 4). Morphological and structural traits include canopy height, leaf area, canopy coverage, as well as the volume, size, diameter, and width of crowns, etc. These structural traits were determined by three-dimensional point clouds from RGB imagery or LiDAR scans. Tu et al. [31] and Patrick et al. [8] established a structure from motion algorithm (SfM) for accurately estimating crown height, extent, plant projective cover in avocado tree and Blueberry bush, respectively. Wang et al. [30] compared three representative 3D data acquisition approaches, including 3D Laser scanning, multi-view stereo reconstruction, and 3D digitizing estimates, and these approaches with respect to leaf length, width, inclination angle, azimuth, area, and height in Maize. The SI also reports work on seed emergence uniformity [7], number of spikes [11] in wheat, number of flowers in oilseed rape [23] and cotton bolls [28] under various ecological conditions. Concerning biochemical traits, senescence, beet cyst nematode, late blight, water use efficiency, stem water potential; chlorophyll a, fluorescence, were measured by RGB camera, thermal images and multi/hyperspectral images from handheld and UAV platforms. Measurements of fluxes were also reported, e.g., leaf gas exchange.

2.3. Data Processing Methods

In the age of big data, data processing methods are at the core for improving the quality and efficiency of information extracted from data capture systems. The application of various data processing methods is shown in this special issue (Figure 5). The methods are broadly classified into two types, and each was applied to the two above mentioned types of phenotyping traits. For biochemical traits, machine learning and artificial intelligence methods, including principal component analysis (PCA), partial least squares regression (PLSR), random forest regression (RF), artificial neural network (ANN), etc., were usually preferred and recommended [10,14,15,16,17,18,21,23,24,32]. Those methods proved to be very efficient for the various target phenotyping traits. Physical model integrating optimizing algorithm was also considered in one remote sensing application, and there was one study introduced by Thorp et al. [26], which used the fractional vegetation cover (FVC) to drive a daily ET-based soil water balance model for seasonal crop water use quantification.

In comparison to biochemical traits, morphological and structural traits were widely extracted using image segmentation methods. In Zhou et al. [11] study, the maximum-entropy method was used to do the coarse-segmentation for recognizing wheat spikes. Junho et al. [28] proposed an automatic open cotton boll detection algorithm from UAV imagery for yield estimation. Above referred methods and other image segmentation methods in this special issue were mainly used for extracting the target morphological traits, while reducing the interference from the background information. Other morphological traits such as crown height, extent, volume, and diameter were mainly derived and processed from three-dimensional point clouds by structure from motion algorithms [8,19,30,31,33].

3. Conclusions

The contributions of 30 papers reported in this special issue highlight the hot topic on estimation of crop phenotyping traits using by UGV and UAV imageries. The Special Issue—and our short Editorial—show the importance of high-throughput crop phenotyping for crop production. Second, the SI discusses the application of the platforms and sensors for high-throughput phenotyping traits of crops. Finally, it provides important hints on how to use data processing methods to estimate crop phenotyping traits for different crop types.

Despite the tremendous progress in the field of phenotyping, there is still ample opportunity for follow-up investigations into some key points. In particular, we recommend more research on the development and application of UGV platforms and data fusion algorithms combining multi-source data. More research is also warranted regarding deep learning methods, linking radiative transfer models and functional structure models, which are not sufficiently covered in this spectral issue. With fast development of sensors technology and image processing algorithms, the above key points will certainly receive more attention by the respective remote sensing, image processing, and crop breeding communities.