Detection of Italian Ryegrass in Wheat and Prediction of Competitive Interactions Using Remote-Sensing and Machine-Learning Techniques

Abstract

Italian ryegrass (Lolium perenne ssp. multiflorum (Lam) Husnot) is a troublesome weed species in wheat (Triticum aestivum) production in the United States, severely affecting grain yields. Spatial mapping of ryegrass infestation in wheat fields and early prediction of its impact on yield can assist management decision making. In this study, unmanned aerial systems (UAS)-based red, green and blue (RGB) imageries acquired at an early wheat growth stage in two different experimental sites were used for developing predictive models. Deep neural networks (DNNs) coupled with an extensive feature selection method were used to detect ryegrass in wheat and estimate ryegrass canopy coverage. Predictive models were developed by regressing early-season ryegrass canopy coverage (%) with end-of-season (at wheat maturity) biomass and seed yield of ryegrass, as well as biomass and grain yield reduction (%) of wheat. Italian ryegrass was detected with high accuracy (precision = 95.44 ± 4.27%, recall = 95.48 ± 5.05%, F-score = 95.56 ± 4.11%) using the best model which included four features: hue, saturation, excess green index, and visible atmospheric resistant index. End-of-season ryegrass biomass was predicted with high accuracy (R² = 0.87), whereas the other variables had moderate to high accuracy levels (R² values of 0.74 for ryegrass seed yield, 0.73 for wheat biomass reduction, and 0.69 for wheat grain yield reduction). The methodology demonstrated in the current study shows great potential for mapping and quantifying ryegrass infestation and predicting its competitive response in wheat, allowing for timely management decisions.

1. Introduction

Italian ryegrass (Lolium perenne ssp. multiflorum (Lam) Husnot) is one of the most problematic weeds in wheat (Triticum aestivum L.) production in the United States (U.S.) [1]. Italian ryegrass is a cool-season winter annual weed that thrives best under a temperature range of 20 to 25 °C. It has faster leaf expansion rate than wheat and its competition can negatively impact tiller production, uptake of soil nutrients, photosynthesis and overall growth of wheat, resulting in significant crop yield loss [2,3]. Italian ryegrass densities as low as 1 plant m⁻² can reduce wheat grain yield by 0.4% [4]. Early management of this species is vital to prevent yield loss, given its high competitive ability with wheat [4].

Advancements in precision agriculture can facilitate site-specific weed management (SSWM) [5], which involves variable application rates for effective weed management based on weed distribution, location, and density in crops [6]. This approach can assist with effective management of herbicide resistance in weeds such as Italian ryegrass [7,8]. Given the vital need for early-season weed control to prevent crop yield loss, information on weed distribution through effective detection and mapping in crop fields is of paramount importance [9]. Furthermore, an ability to predict the outcomes of weed-crop competitive interactions, particularly crop yield reduction and weed seed production, using early-season weed infestation levels can facilitate informed management decisions for timely action [10].

Precise detection and mapping of Italian ryegrass in wheat fields, especially during early growth stages, is a challenge, generally due to high morphological similarities and indistinct canopy boundaries. Although Italian ryegrass has a characteristic pale green color and could stand out visually from wheat plants, such differences may not be obvious or too intricate to recognize in spectral data. These challenges in the classification of grass weed species, such as wild oat (Avena sterilis L.) and rigid ryegrass (Lolium rigidum L.), in a grass crop such as wheat based on spectral signatures have already been reported [11,12]. Few other attempts have been made to classify ryegrass from wheat using digital imagery [13,14], but primarily using traditional classification approaches that may be less robust.

Addressing the challenge of grass weed detection in a cereal crop would require solutions on two major fronts: acquiring high spatial resolution imageries of production fields and developing effective image analysis models for precise species detection. Although varying spatial resolution of imageries and maps could have a differential impact on model uncertainties [15,16], high-resolution imageries have proven effective for detection of weeds at the individual plant level even at early growth stages [17]. Mapping ryegrass at individual plant canopy level allows for better estimation of weed infestations across the crop field and thus effective implementation of SSWM. Unmanned aerial systems (UAS), one of the popular remote-sensing platforms, have been successfully utilized in obtaining high-resolution aerial imageries for weed detection and mapping [18,19,20,21,22]. However, the benefits of high-resolution imagery can be fully exploited only if the image analysis/classification approach used for the problem is robust.

Several machine-learning classification approaches have been employed for various classification problems in the agricultural sector, including mapping of crops and weeds using aerial imageries. Yang et al. [23] applied the single feature probability technique to generate features, which were later trained with decision trees and maximum likelihood classifier to map rice fields and assess lodging. Gašparović et al. [24] utilized a fusion of random forest-based supervised and K-means algorithm-based unsupervised classification methods to map oat (Avena sativa L.) in fields using a low-cost unmanned aerial vehicle (UAV)-borne red, green and blue (RGB) imagery. Combining the object-based image analysis (OBIA) with random forest-based prediction, De Castro et al. [21] analyzed UAV imagery and its derivatives to map weeds in cotton and sunflower fields. Gao et al. [17] fused row-detection algorithm results with OBIA-derived features to map weeds in maize fields using a random forest classifier.

Every classification problem poses a unique level of intricacy and therefore demands a suitable classification mechanism. For discriminating among different grass species, a powerful classification approach accompanied by a machine-learning classifier may be desirable. One of the effective ways in this regard is to generate multiple features, select the best and most informative features, and make inferences using a powerful machine-learning algorithm. The feature selection process ensures the elimination of irrelevant features, which would otherwise compromise the ability of the machine-learning models [25]. Artificial neural networks (ANNs), one of the most powerful and advanced machine-learning classifiers, have been frequently used for weed detection and mapping [26,27,28]. ANNs, in general, are the computing systems that mimic the biological neural networks, comprising of three main systems namely the input layer to receive the data, hidden layer(s) to learn the pattern in the data, and output layer to provide the best parameters for classification [29].

The current study utilizes deep neural networks (DNNs) for detection and mapping of Italian ryegrass in wheat fields. DNNs are ANNs with more than one hidden layer, designed to improve the ability to learn complex patterns. With the increase in the number of hidden layers, the neural networks become denser with an enhanced ability for pattern recognition [30]. Here, we apply a rigorous hyperparameter tuning process and exhaustive feature selection to improve the DNN-based classification accuracy. The specific objectives of this study were to: 1) detect and map Italian ryegrass in wheat fields using UAS-derived imageries and DNNs; and 2) develop and test models to predict the impact of early-season Italian ryegrass infestations determined using UAS-derived imagery on end-of-season productivity of wheat and ryegrass.

2. Materials and Methods

2.1. Location and Experimental Setup

The study was conducted in 2018 at two distinct sites (0.2 ha each) at the Texas A&M AgriLife Research (Site A, 30°32′15″N, 96°25′35″W, elevation: 70 m) and Extension farms (Site B, 30°30′37″N, 96°25′13″W, elevation: 68 m) located in Burleson County, TX (Figure 1). The locations are characterized by a sub-tropical climate, with average monthly maximum and minimum temperatures during the study period/winter wheat growing season (November–May) of 20 °C and 8.5 °C, respectively. The total rainfall during the growing season in 2018 for this area was 889 mm. The sites mainly varied in soil composition; the soil type of Site A was Weswood silty clay loam, whereas that of Site B was Belk clay [31].

The winter wheat crop (TAM 304) was drill-seeded at a seeding rate of 120 kg ha⁻¹ and 19 cm row spacing on 15 November 2018 at Site A and 20 November 2018 at Site B. An Italian ryegrass biotype sourced locally was broadcast planted in the plots immediately after planting wheat. The experimental area was divided into model training (red polygons in Figure 1) and validation sections (blue polygons in Figure 1). The training area consisted of three Italian ryegrass density (low, moderate, and high) treatments and a weed-free check, replicated four times (16 total plots; plot size: 2 m × 3 m, with 2 m buffer in all sides) in a randomized complete block design. Within each training plot, a 1 m² quadrat was established at the center which served as the sampling unit for image analysis and ground-truth data collection. In the quadrats, the ryegrass seedlings were thinned to simulate a gradient of different densities across sites, with achieved final densities of 20, 50, and 80 plants m⁻² for Site A, and 50, 100, and 150 plants per m⁻² for Site B. The validation area (25 m × 9 m) had a random gradient of densities of Italian ryegrass inter-mixed in wheat, and a total of 5 quadrats were established within the validation area for each site as the sampling units for model validation. Wheat was raised as a rain-fed crop, and nitrogen fertilizer (150 kg ha⁻¹) was split-applied at 45 days after planting (DAP) (50 kg ha⁻¹) and at 90 DAP (100 kg ha⁻¹). No pest control treatments were required.

2.2. General Workflow

The experiment began with image collection during the early growth stage of weed, followed by an end-of-season collection of ground-truth data for both weed and crop. The next step was to process the imagery, which was conducted in three sub-steps, including image mosaicking and calibration, feature extraction and selection, and image classification and validation. Regression modelling was performed to develop predictive models using image- and ground-based information. Finally, the models were implemented on the validation plots to build a heatmap for different measured variables (Italian ryegrass biomass and seed production, and wheat biomass and grain yield reduction) and validate the accuracy of the models. Figure 2 shows the schematics of the general workflow followed in this research.

2.3. Data Collection

2.3.1. Image Collection

In order to collect early-season information required for the study, aerial flights were carried out on 6 March 2019 and 13 March 2019 at Site A and Site B, respectively. The timing coincided with the peak tillering stage of wheat and ryegrass, at about 90 DAP on both sites. A quadcopter UAV “DJI Phantom 4 Pro” (DJI, China) attached with an RGB sensor (12 megapixels) was flown at an altitude of 10 m to acquire aerial images at three different bands (Red, Green, and Blue) during ±2 h of solar noon (10 AM to 2 PM) in both sites. The average wind speed was 9.6 kmph for Site A and 8 kmph for Site B throughout the flight duration. Images were acquired at an overlapping mode (75% for both side and end overlap), the exposure was set to automatic mode, and the flight plan was executed in a grid structure at an operating speed of 5 m/s. The flight mission was executed using the mobile application “Pix4Dcapture” (Pix4D, Lausanne, Switzerland) and was completed in 20 min at each site. Reflectance panels/tarps were placed in the field at the time of flights to perform spectral calibration in the imagery at a later stage.

2.3.2. Ground-Truth Data Collection

Upon wheat maturity, ground-truth data pertaining to ryegrass biomass, ryegrass seed yield, wheat biomass, and wheat grain yield were obtained from each quadrat on 23 May 2019, to develop regression models between early-season ryegrass densities and end-of-season biophysical parameters of ryegrass and wheat. In order to account for potential ryegrass seed loss due to shattering prior to harvest, a visual estimate of seed shattering was documented at the time of harvest. The ryegrass and wheat plants were manually harvested from each quadrat at the ground level, separated by species, placed in individual paper bags, and dried in an oven at 63 °C for 36 h prior to the estimation of dry biomass. Wheat plants from each experimental unit were threshed to obtain grain yield. Ryegrass spikes were hand threshed and seed yield was determined after adjusting for shattering loss.

2.4. Image Processing

2.4.1. Image Mosaicking and Calibration

Images acquired for each site were stitched together using the Pix4D mapper software (Pix4D, Lausanne, Switzerland) to generate qualitative, high-resolution (3 mm/pixel) orthomosaic imageries. Generating qualitative orthomosaic imageries can sometimes be challenging as the process depends heavily upon several factors, including camera internal and external orientation parameters, flight parameters, and the robustness of the image-matching algorithm [32,33]. Failure to optimize the camera parameters can result in distortion of the imageries. The Pix4D mapper mitigates this issue by optimizing the camera parameters during the initial run and allowing users to re-run the process with the optimized parameters. In this study, camera model parameters were initially loaded from the exchangeable image file format metadata, generated automatically by the UAV during the image acquisition process, into the Pix4D mapper. To further improve the quality, the initial calibration phase was re-run using the optimized parameters. A detailed description of how Pix4D mapper generates an orthomosaic imagery from sets of UAV-borne imageries can be found in this link (https://support.pix4d.com/hc/en-us/articles/204272989-Offline-Getting-Started-and-Manual-pdf).

Following the orthomosaic generation, the digital number (DN) values of the imageries were calibrated to reflectance values using the three different custom spectral panels (black, grey, and white). Three different datasets, each with 300 DN or pixel values of a band as the X-variable and the reflectance values of corresponding pixels in the spectral panels derived using Analytical Spectral Devices FieldSpec Pro HandHeld spectroradiometer (Analytical Spectral Devices, Boulder, CO, USA) as the Y-variable were prepared. Furthermore, simple linear regression analyses were conducted for the X- and Y-variables to derive three separate regression models (Equations (1)–(3)) for predicting reflectance values using prepared datasets. The model was then applied to predict the values for all the pixels in red, blue, and green bands.

$${\left(\mathsf{\sigma }\mathrm{j}\right)\mathrm{r}=\mathsf{\mu }1\ast (\mathsf{\lambda }}_{\mathrm{j}})\mathrm{r}+\mathrm{c}1$$

(1)

$$\left(\mathsf{\sigma }\mathrm{j}\right)\mathrm{g}={\mathsf{\mu }2\ast (\mathsf{\lambda }}_{\mathrm{j}})\mathrm{g}+\mathrm{c}{2}_{}$$

(2)

$$\left(\mathsf{\sigma }\mathrm{j}\right)\mathrm{b}={\mathsf{\mu }3\ast (\mathsf{\lambda }}_{\mathrm{j}})\mathrm{b}+\mathrm{c}3$$

(3)

where, $\mathsf{\sigma }\mathrm{j}=$predicted reflectance value of a jth pixel for the red (r), green (g), and blue (b) band; $\mathsf{\lambda }$j = DN value of a jth pixel for the red (r), green (g), and blue (b) band; $\mathsf{\mu }$1, $\mu 2,\mathrm{and}\mu 3$are slope values derived from the linear equations for red, green, and blue band, respectively; and c1, c2, and c3 are intercepts for models for red, green, and blue band, respectively.

2.4.2. Feature Extraction and Selection

Following the spectral calibration, 12 feature layers were extracted and/or computed for further image-processing purposes (

Table 1. Details of various features extracted and/or computed for image classification, through several computational procedures on the pixel value of imageries. The value in parenthesis indicates number of features belonging to the feature category.

Category	Features	Description/Formula **	Reference
Original bands (3)	Blue	-	-
	Green
	Red
Vegetation indices (3)	Excess Green Index (ExG)	$2\times \frac{\mathrm{G}-\mathrm{R}-\mathrm{B}}{\mathrm{R}+\mathrm{G}+\mathrm{B}}$	[35]
	Triangular Greenness Index (TGI)	$-\frac{190(\mathrm{R}-\mathrm{G})-120(\mathrm{R}-\mathrm{B})}{2}$	[36]
	Visible Atmospheric Resistant Index (VARI)	$\frac{\mathrm{G}-\mathrm{R}}{\mathrm{G}+\mathrm{R}-\mathrm{B}}$	[37]
Color space transformed features (3)	Hue	A gradation or variety of a color	[38]
	Saturation	Depth, purity, or shades of the color
	Value	Brightness intensity of the color tone
Wavelet transformed coefficients (2)	Wavelet coefficient mean	Mean value calculated for a pixel using discrete wavelet transformation	[39]
	Wavelet coefficient standard deviation	Standard deviation calculated for a pixel using discrete wavelet transformation
Principal components (1)	Principal component 1	Principal component analysis-derived component accounting maximum amount of variance	[40]

** Abbreviations: R, G, and B represent Red, Green, and Blue bands, respectively.

). Optimizing the feature subset is required before feeding into machine-learning algorithms for improving the classification process and making it cost- and time-efficient [34]. For this purpose, first, 1000 training samples for each of the user-defined classes (in this case, 5-classes: Ryegrass-A, Ryegrass-B, Non-ryegrass vegetation, Bareground, and Shadow) were collected from the imageries of both sites. Ryegrass-A and B represent different categories of ryegrass pixels in the imagery, indicating normal green pixels (A) and illuminated pixels (B). A considerable number of illuminated ryegrass pixels were observed in the experiment area, and the two categories were treated separately since combining them might compromise the prediction ability of the classifier. Second, the distribution of features within/across the user-defined classes were explored to select the best feature combination qualitatively. However, such a selection approach was too complex since there was much variation in the distribution of these features (Figure 3). Therefore, a wrapper-based feature selection approach called “exhaustive selection” was employed to select the 10 best feature combinations (hereafter referred to as feature models) using one-fourth of the training samples for each class (i.e., 250).

2.4.3. Image Classification and Validation

After the selection of best feature models, each feature model was tested for classifying images sampled in each quadrat (1 m × 1 m) (hereafter referred to as quadrat images) into user-defined classes using the supervised machine-learning system. The back-propagation multilayer perceptron (MLP), a commonly used and widely available ANN structure [41], was used as the machine-learning system in this study. Keras, a high-level neural networks application programming interface (API) written in python computer language [42], was used to build the MLP-based custom DNN system. Various hyperparameters were tested for several values prior to final training process to derive the best set of values (

Table 2. Various hyperparameters and several potential values for corresponding hyperparameters tested for the best performance of the custom deep neural network system using the grid search cross validation method. A total of 150 trials were conducted to obtain the best set of values.

Hyperparameter	Values	Selected Value(s)
Number of hidden layers	1,2,3,4,5	5
Nodes in hidden layers *	(20) (40, 20) (60, 40, 20) (80, 60, 40, 20) (100, 80, 60, 40, 20)	(100, 80, 60, 40, 20)
Activation function	“Rectified linear unit” “Sigmoid function”	“Rectified linear unit”
Batch size	10, 20, 30, 40, 50	20
Number of epochs	100, 300, 500	100

* The values for hyperparameter “nodes in hidden layers” were assigned with respect to “number of hidden layers”.

), using the same subset of training samples used for exhaustive feature selection. Categorical crossentropy was fixed for the loss function and “adam”, a very popular stochastic gradient descent-based weight optimization technique, was fixed for the optimizer in the tuning and training process.

This best set of parameters was then used in the custom DNN system to train each feature model. One-half of the training samples for each class (i.e., 500) were used for training purposes. Each trained DNN model was finally implemented over each quadrat image and in total 10 different classification outputs were generated for 10 different feature models for each image. Once the quadrat image was classified into user-defined classes, post-classification operations such as filtering, smoothing, and generalization were carried out to remove any speckled appearance and improve the quality of the classified output. For accuracy purposes, different indicators such as precision, recall, and F-score were calculated for each feature model using an independent set of validation samples [i.e., remaining one-fourth of training samples (250) for each class] and the best feature model was determined. Precision was measured as the number of correctly classified samples of a class divided by the number of samples labeled as that class by the system (Equation (4)). Recall was calculated as the number of correctly classified positive samples of a class divided by the number of validation samples allocated for that class (Equation (5)). F-score is a combination of both precision and recall (Equation (6)).

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(4)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(5)

$$\mathrm{F}\text{-}\mathrm{score}=2\times \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

(6)

where, TP, FP, and FN represent true positive, false positive, and false negative instances, respectively.

2.5. Regression Modeling

With the classification of imageries and evaluation of the models, the best classified output, i.e., classified imagery for each quadrat (altogether 32, including both sites) were used in the regression modelling procedure. Each classified imagery comprised of pixels classified into either of the user-defined classes. First, both ryegrass classes (i.e., Ryegrass-A and B) were merged into a single class. Then, the ryegrass pixels pertaining to this class within each classified imagery were enumerated and the number of enumerated pixels was divided by total number of pixels in the imagery to calculate ryegrass canopy coverage area (%). In the next step, four separate models were developed by regressing the canopy coverage area of ryegrass (%) as the predictor variable, while considering ryegrass biomass (g), ryegrass seed yield (g), wheat biomass reduction (%), and wheat grain yield reduction (%) as predicted variables. Wheat biomass and grain yield reduction (%) were calculated as a relative measure with weed-free check plots. Altogether, 32 pairs of predicted and predictor variables (16 pairs corresponding to the quadrats for each site) were used in the regression analysis for ryegrass biomass and seed yield, whereas only 24 pairs were used for wheat biomass reduction (%) and grain yield reduction (%). Finally, the coefficient of determination (R²) and root mean square error (RMSE) were calculated as statistical measures of how well the regression predictions approximated the datapoints.

2.6. Predictive Model Implementation and Validation

The validation areas (Figure 1) were demarcated within the experimental field and the orthomosaic imageries were clipped to their extent for spatial implementation of the predictive models and independent model validation. The best feature model was also applied to the clipped imageries to obtain the classified outputs (i.e., early-season ryegrass canopy coverage maps), followed by all the post-classification operations described earlier to improve the results. The classified map was partitioned into several (1 m × 1 m) grids and the ryegrass canopy coverage area (%) was calculated for each of the grid. The predictive models developed earlier were then applied to the grids to obtain the values for all the predicted variables. Ground truth values for the 5 quadrats in each validation area pertaining to ryegrass biomass (g), ryegrass seed yield (g), wheat biomass reduction (%), and wheat grain yield reduction (%) were assessed against predicted heatmap values for those corresponding grids to determine the reliability of the whole classification and predictive model framework. RMSE and coefficient of determination (R²) were calculated as the measure of agreement between predicted and observed variables.

3. Results

3.1. Ryegrass Detection Using Feature Combinations

Among the approximately 4000 model runs of various features and their combinations tested in the study, the top 10 best performing models had a combination of four or more features, illustrating the robustness of multivariate analysis for species detection. Based on independent validation samples, the average F-score values ranged between 89% and 96% for different feature models (i.e., feature combinations) tested (

Table 3. Validation samples-based accuracy statistics for 10 best feature models used for detecting Italian ryegrass in wheat. The 10 best feature models were determined through the exhaustive feature selection process. The accuracy statistics for each feature model were based on 250 samples for each user-defined class.

Feature Model #	Features Used **	Precision (%) *	Recall (%) *	F-score (%) *
1	Green, Blue, Hue, Sat, Value, TGI, ExG, PC-1	94.28 ± 4.86	94.28 ± 5.24	94.27 ± 4.96
2	Hue, Value, VARI, ExG	94.28 ± 4.50	94.28 ± 7.94	94.27 ± 5.26
3	Red, Hue, Sat, VARI, ExG, Wavelet_Mean	94.95 ± 4.58	94.91 ± 4.98	94.91 ± 4.43
4	Hue, VARI, TGI, ExG	91.53 ± 6.75	90.63 + 8.93	90.54 + 2.29
5	Green, Blue, Hue, Sat, Value, ExG, Wavelet_Mean, PC-1	95.33 ± 4.23	95.38 ± 5.07	95.36 ± 4.14
6	Red, Green, Hue, Sat, ExG, Wavelet_Std	94.45 ± 4.67	94.44 ± 5.83	94.41 ± 4.98
7	Red, Green, Hue, Sat, TGI, Wavelet_Std	95.01 ± 4.31	94.94 ± 5.5	94.91 ± 4.25
8	Sat, Value, VARI, ExG	91.74 ± 8.14	91.59 ± 9.48	91.54 ± 8.21
9	Red, Blue, Sat, VARI, ExG, Wavelet_Mean	95.01 ± 4.71	94.97 ± 5.36	94.96 ± 4.67
10	Hue, Sat, VARI, ExG	95.34 ± 4.27	95.68 ± 5.05	95.56 ± 4.11

* A single mean value calculated by averaging the values of all the user-defined classes, and standard deviation value calculated among the accuracies for each user-defined class for each feature model.

** Abbreviations: ExG-Excess Greenness Index, PC-Principal component, Sat-Saturation, TGI-Triangular Greenness Index, VARI-Visible Atmospheric Resistant Index, Wavelet_mean-Wavelet transformed coefficient mean value, Wavelet_std-Standard deviation of Wavelet transformed coefficients mean value.

). The highest average F-score (95.5%) was achieved with the model that combined color transformed features (hue and saturation) with vegetation indices (Excess Green Index (ExG) and Visible Atmospheric Resistant Index (VARI)) for machine learning (Model #10 in Table 3), which was closely followed by the model that used Red, Blue, Sat, VARI, ExG and Wavelet_Mean (Model #9; F-score: 95.3%). However, the model #10 was chosen for mapping Italian ryegrass (Figure 4) since it was more parsimonious compared to #9.

For model #10, the user-defined classes Bareground and Shadow were classified with the highest precision, recall and F-score (>98%), compared to the other classes namely Ryegrass-A, Ryegrass-B and Non-Ryegrass (Figure 5). As explained by boxplots for different features (Figure 3), Bareground and Shadow had very distinct boundaries from other classes for several features. The lightly shaded portion of Italian ryegrass and wheat leaves were expected to be classified as Shadow (i.e., formed underneath the canopy) due to spectral similarities; however, a meticulous training of these regions greatly reduced potential misclassification, which is indicated by the high precision (>98%) and recall (>98%) values for Shadow. The classification for Non-Ryegrass vegetation had the lowest accuracy (Figure 5; F-score: 91%), which is likely since this class encompassed a mixture of primarily wheat and few other weed species, resulting in fuzzy, instead of distinct, boundaries for different features. As a result, there could have been several instances of misclassification with either Shadow and/or Ryegrass-A. Ryegrass-B had a higher F-score (94%) compared to Ryegrass-A (92.5%) and Non-Ryegrass vegetation (91%), which could be attributed to brighter pixels of Ryegrass-B compared to the rest of the vegetation pixels, leading to distinct separation for several features. However, Ryegrass-B and Bareground overlapped for several features, as a result of reflectance from debris present on the soil surface which often produced bright reflectance.

3.2. Prediction of Competitive Outcomes between Italian Ryegrass and Wheat

The canopy coverage area (%) for Italian ryegrass (predictor variable) was computed from each classified map of quadrat image (Figure 6) and regressed against the ground truth data (predicted variables). In general, Italian ryegrass biomass and seed production increased with an increase in their canopy coverage area (as determined through image analysis) for the densities simulated here, with a concurrent decline in wheat biomass production and grain yield. The highest coefficient of determination (R² = 0.87; RMSE = 66.03) was achieved for prediction of ryegrass biomass, followed by ryegrass seed yield (R² = 0.74; RMSE = 32.44), wheat biomass reduction (%) (R² = 0.73; RMSE = 9.27), and wheat grain yield reduction (%) (R² = 0.69; RMSE = 10.94) (Figure 7). Results showed that Italian ryegrass coverage had a linear relationship with its biomass, and a curvilinear relationship with its seed production as well as biomass and grain yield reduction of wheat.

3.3. Model Validation

The early-season ryegrass canopy coverage maps developed with the DNN model for validation area in each site (Figure 8, top panel) and the competition models described above were utilized together to produce heat maps (1 m × 1 m grid size). These heat maps provide a visual representation of weed/crop competitive outcomes at the end of the season in terms of biomass and seed yield (Figure 8, bottom panel). Validation results showed that the coefficient of determination based on predicted (heat map-based) and observed values (ground-based) was the highest (R² = 0.83; RMSE: 69.8) for Italian ryegrass biomass, followed by ryegrass seed yield (R² = 0.72; RMSE = 17.9), wheat biomass reduction (%) (R² = 0.63; RMSE: 10.57), and grain yield reduction (%) (R² = 0.60; RMSE = 16.23) (Figure 9). Thus, the validation analysis showed that the models developed in this study were generally robust in predicting end-of-season productivity for Italian ryegrass as well as wheat.

4. Discussion

The results provide strong evidence that a combination of multiple classification features is more effective in species detection compared to employing individual features, but the choice of features is important. In this study, color-transformed features (hue and saturation) and vegetation indices (VARI and ExG) were found to be the most effective combination in detecting Italian ryegrass in wheat. Hue and saturation are invariant to brightness variation [43] and, therefore, are least affected by illumination differences by ryegrass leaves. Given the pale green color of ryegrass leaves compared to that of wheat, the difference in the greenness level was obvious with hue and saturation values. Several studies have credited hue and saturation for their ability to differentiate plants based on the greenness level [44,45]. Additionally, ExG was shown to be useful in separating plant tissues from other backgrounds (soil and weathered plant residue) [46]. VARI was designed to be minimally sensitive to atmospheric effects, allowing precise estimation of the vegetative fraction of different plant species. Recently, VARI was found to be very useful in classifying real shadows from non-sunlit plant leaves in the canopy [47]. This property of the index may have helped in reducing misclassification between shadow and non-sunlit wheat/ryegrass plant canopies in our study, as there were several non-sunlit plant pixels with shadow-like appearance in both experimental sites.

A very limited number of studies have detected/classified grass weeds in wheat using digital images (either handheld camera or UAS-derived) to date. Golzarian and Frick [14] used very high-resolution true-color images (0.26 mm/pixel) for differentiating annual ryegrass and wheat, with an accuracy of 88%. It should be noted that the current study utilized relatively lower spatial resolution (3 mm/pixel; Figure 4) and still achieved a higher accuracy (F-score: 95%). This is particularly advantageous for optimizing computational costs and complexity when scaling up this approach for vast production fields. The reasons for the improvement in classification accuracy compared to that of Golzarian and Frick [14] could be the use of DNNs, which have been proven to solve increasingly complicated applications with increasing accuracy over time [30]. However, the current study classified ryegrass at a relatively larger seedling stage compared to what was studied by Golzarian and Frick [14], which may also have affected the learning capability of the classification model.

Kodagoda et al. [13] used an overhead imaging system fitted with color and near-infrared cameras to capture high-resolution digital images in order to differentiate between wheat and two weed species, cobbler’s peg (Bidens pilosa L.) and rigid ryegrass (Lolium rigidium L.). Hue, saturation, and texture information of plant leaves were extracted from the digital images and fed into traditional machine learning algorithms such as k-means clustering and Mahalanobis distance. Although their model worked fairly well for differentiating cobbler’s peg from wheat (accuracy: 85%), it failed to detect and classify ryegrass from wheat (accuracy: 26%). Similarities of these two species in the distributions of hue, saturation, and texture cues were concluded to be the prime reason for the very low performance of the model. The current study also observed an overlap in the distribution of hue and saturation between ryegrass and non-ryegrass vegetation (Figure 3); however, supplementing with vegetation indices such as ExG and VARI was beneficial for classification.

Recently, convolutional neural networks (CNNs) have been widely appreciated for their high potential for detecting and mapping weeds [48,49,50]. However, training a CNN model for segmentation generally requires a relatively large number of annotated labels for weeds and crop canopy boundaries, making the procedure labor-intensive and time-consuming [50]. The complexity in training data preparation manifolds due to the intricate labeling procedure for grasses caused by the extreme interlocking of leaves. Moreover, this annotation process is almost impossible if the resolution of the imagery is not high enough to clearly delineate leaf boundaries. Instead of a time-consuming and intricate weed annotation procedure, this study adopted the relatively easier training sample selection approach. The most representative pixels for each user-defined class were selected as the training samples and deep neural network system was trained over the samples to achieve higher accuracy. With the intensive feature optimization technique, this study generated various feature models and tested independently to obtain the most accurate model. Feature optimization processes, such as that described in the current research, are often reported to boost machine learning performance [34].

To the best of our knowledge, very few studies have utilized DNN-based predictive models for understanding weed–crop interaction and explored the feasibility of predicting biomass and seed yield using plant canopy coverage information. Most of the existing yield prediction studies have heavily relied upon vegetation indices, especially Normalized Difference Vegetation Index (NDVI) [51,52,53]. However, several studies have reported that NDVI becomes saturated at high leaf-area index levels, which in turn may lead to inaccurate prediction of biomass and yield [54,55]. This study, in contrast, utilizes vegetation indices and other promising features to classify the pixels pertaining to the class of interest and then uses the number of pixels as the predictor variable for biomass and seed yield. Thus, this method avoids the risk of saturation and seasonal variability of vegetation indices, and leads to a better prediction.

Relatively lower correlation for wheat biomass and grain yield reduction (%) compared to ryegrass biomass and seed yield in this study could be attributed to the use of ryegrass canopy coverage area (%) as the predictor variable. In particular, wheat grain yield reduction (%) had lower coefficient of determination compared to wheat biomass because grain yield is a complex trait and is affected by several factors including environmental and genetic factors [56,57]. Although biomass has been reported to be one of the primary determinants of grain yield, other factors such as grains per spike and spikelets per plant may also have an influence [57]. Thus, the competitive effect of ryegrass on wheat grain yield may not be proportional to the effect on wheat biomass, as explained by different coefficient of determination for wheat biomass and grain yield in this study.

The ryegrass infestation map developed during the early season may help facilitate management interventions, including site-specific weed management [6]. The infestation maps can also be useful for monitoring ryegrass distribution and dynamics spatially and temporally. The predictive models and the spatial heatmap representation of weed-crop competitive interactions as presented in this study can be highly useful for management decision making [58]. Predicting weed-crop interference early-on can inform weed control thresholds required to minimize yield loss [59]. These spatial heatmaps together with weed control thresholds can be utilized to create management grids. It should also be noted that heat map representation of competitive interactions at the 1 m × 1 m grid level in this study can be scaled-up to various grid sizes to fit different management needs. Furthermore, recommendations for the features and hyperparameters made in the study could be utilized in similar studies to improve the efficiency.

This study, however, has some limitations: (1) only ANNs were tested for weed detection where several machine-learning classifiers such as random forest and support vector machine were available and already used for weed detection and mapping in the past. Future research should test these classifiers independently or maybe fused with more advanced deep-learning methods such as the CNNs; (2) broader applicability of the classification model presented here in wheat fields with varying geographies and environmental conditions is unknown. Wheat varieties may widely differ in leaf color and composition and thus may exhibit different spectral signatures. The model can be generalized and empowered with diverse training samples; (3) the competition models developed here were based solely on ryegrass canopy coverage area estimated from the aerial imageries. As such, this study did not attempt to utilize/evaluate already established weed-crop competitive models that were based on variables such as weed density [60], biomass [61,62], and leaf-area index [63]. The effectiveness of the canopy/ground cover-based prediction compared to the previously established approaches is unknown. Future research should test and ensemble these approaches to improve the accuracy and feasibility of weed–crop interaction assessments; and (4) scaling this approach to large production fields may be challenging due to high computational demands.

5. Conclusions

This study successfully identified and demonstrated a UAS-based remote-sensing approach that combined both color transformed features and vegetation indices for improved detection and mapping of Italian ryegrass in wheat (Highest F-score: 95.56 ± 4.11%). In addition, this study provided evidence that deep learning-based estimation of early-season plant canopy coverage can be a better predictor for competitive interactions, with relatively higher R² values for developed models [0.87 for ryegrass biomass (g), 0.74 for ryegrass seed yield (g), 0.73 for wheat biomass reduction (%), and 0.69 for wheat seed yield reduction (%)]. This study also highlighted the value of affordable, computationally less complex, and less storage demanding RGB imageries in assisting farmers with weed assessment and precision weed management. The machine learning-based classification model and the weed–crop competition models developed and employed in the study will be helpful in devising suitable agronomic interventions.