A Grain Number Counting Method Based on Image Characteristic Parameters of Wheat Spikes

Abstract

In order to measure wheat yield and wheat spike phenotypes, the grain number of wheat spikes is counted manually at present, but acquiring the grain number of wheat spikes is laborious and time-consuming. Counting the grain number of wheat spikes with an image processing method is promising, yet the application of this method is flawed due to its low accuracy. In this work, images of wheat spikes were collected and processed with technical procedures, including image cropping, image graying, histogram equalization, image binarization, eroding operation, removing small objects, filling image holes, revolving vertical spikes, cutting off stems, and removing stems. Wheat stems in binary images were eliminated by the sum pixels method, and the morphological characteristic parameters of the image areas of wheat spikes and lengths of wheat spike axes were calculated. Mathematical models relating the image areas of wheat spikes and lengths of the wheat spike axes to the grain number were established, and the mathematical models were verified. The results showed that the characteristic parameters of the image areas of wheat spikes and the lengths of the wheat spike axes for the spike images were linear relative to the grain number, and the maximum determination coefficients R² were 0.9336 and 0.9012, respectively. The maximum determination coefficients R² for the practical and predicted grain numbers were 0.9552 and 0.9369, respectively, and the minimum average absolute error was 2.3, while the average relative error for the mathematical models was 5.65%. The mathematical models relating the image areas of wheat spikes and the lengths of the wheat spike axes to the grain number were practical and accurate, and the mathematical model comparing the image area of wheat spikes and the grain number was superior to that comparing the length of the wheat spike axis and the grain number. The grain number of wheat spikes could be acquired accurately and quickly by the image processing method extracting the characteristic parameters of wheat spikes.

1. Introduction

The grain number of wheat spikes is counted by breeders, farmers, and agronomists before wheat harvesting in order to predict crop yield and analyze the phenotype [1]. Crop yield can be obtained quickly and correctly according to the grain number of wheat spikes, the wheat spike number in a unit area, and the 1000-grain weight. Counting the grain number of wheat spikes is one of the most important and difficult steps for obtaining the theoretical wheat crop yield. The grain number of wheat spikes is obtained manually, and then crop yield is calculated from the counted grain number. Counting the grain number of wheat spikes manually is exhaustive and time-consuming, and is particularly unable to avoid definite errors. Limitations of the manual counting method are many; for example, results acquired from different operators may vary subjectively. It is necessary to develop a method or equipment that can quickly and correctly count the grain number of wheat spikes.

There have been a number of reports on the development of digital image processing technologies for phenotyping wheat grain traits. With the aid of this technology, classification of a wheat variety and its source growing region can be made with reference to the characteristic parameters of wheat grain morphology [2,3]; color [2,4,5]; texture [6]; and combined morphology, color, and texture [7]. However, the overall reports on phenotyping the characteristic parameters of wheat spikes using digital image processing technology are few and limited in precision. Image processing tools have been applied to phenotype the wheat spike awn number [8,9], the average awn length [8,9], spike shape and morphometric characteristics [8,10], and spike length [8,9,11]. Wheat spike length and spikelet number were measured synchronously from spike profile images, and the zero-error rate for detecting the spikelet number was only 16.0% [12]. Du Shiwei et al. [13] counted the spikelet number and grain number with the spikelet method by separating each spikelet from front-view images of wheat spikes. The zero-error rate for the counted spikelet number was 68.16%, the mean absolute error was 0.46, and the mean relative error was 2.99%. The grain number of wheat spikes was also measured from the total grain number of spikelets; the mean absolute error for the total grain number of spikelets was 2.11 and the mean relative error was 5.62%. In another alternative of image processing, Gong and Ji [14] established the relationship between the weight of wheat spikes and the image texture features of wheat spikes.

Many studies focusing on counting the number of unit-area wheat spikes in the field have been based on traditional digital image processing methods, such as color–texture image analysis [15]; color features, texture features, and a corner detection algorithm [16]; the Laplacian frequency filter, median filter, and finding maxima to segment local peaks [17]; and convolutional neural networks [18,19]. A neural network-based method with Laws’ texture energy was used to detect wheat spikes, achieving an identification accuracy higher than 80%. The final grain yield per spike was found to be highly correlated with the spike area, and the correlation R² between the average grain yield per spike and the spike area was 0.7889 [20].

Studies on counting the grain number of wheat spikes and predicting crop yield by recognizing the characteristic parameters of wheat spike images are even more scarce. The architecture of wheat spikes is composed of many spikelets distributed alternately on both sides of the spike axis, with each spikelet composed of a few grains. X-ray computed tomography was used for imaging the 3D architecture of wheat spikes to determine the number of grains. The grain number per spike was significantly correlated with the manual counting method (r² = 0.989) [21]. A deep learning method was also applied to count the number of spike grains using only one-sided and two-sided images of wheat spikes, and the counting precision for the two-sided images of wheat spikes was higher than that for the one-sided images of wheat spikes [22]. The determination coefficients R² between the predicted spike grain number and the true spike grain number for images of five different varieties of wheat using the one-sided and two-sided image methods were 0.81 and 0.91, respectively. The accuracy of counting the grain number of wheat spikes failed to fully satisfy practical requirements [20,22]. In comparison, the accuracy for counting the grain number of wheat spikes using X-ray computed tomography was better, yet the account imaging 3D architecture of wheat spikes was complex and time-consuming [21].

Breeders, farmers, and agronomists believe that the morphology of wheat spikes can reflect the grain number of wheat spikes, and the morphology of wheat spikes is represented by the characteristic parameters of the wheat spikes.

The characteristic parameters of wheat spikes were acquired in this paper by processing front-view images of wheat spikes. In order to provide a theoretical basis for quickly and non-destructively predicting crop yield and obtaining the wheat phenotype, the relationships between the characteristic parameters acquired from the wheat spike images—the spike area and length of the spike axis—and the grain number of wheat spikes were established.

2. Materials and Methods

2.1. Image Collection System

The image collection system is shown in Figure 1, which contained one CMOS Industrial Digital Camera (Model: MV-EM1400C; resolution: 3288 × 4608 Pixels; Shanxi Vision Image Technology Company, Xi’an, China), two LED strip light source (Model: AFT-WL21244-22W, White Light, Shanxi Vision Image Technology Company, Xi’an, China), one PC computer (Model: HP Win7 PC 64bit, Shanghai HP Company, Shanghai, China), one stage (Model: MV-BR601, Shanxi Vision Image Technology Company, Xi’an, China), and one light controller (Model: AFT-ALP2406-02, Shanxi Vision Image Technology Company, Xi’an, China).

2.2. Experimental Materials and Image Collecting Methods

Wheat spikes for the experiment were collected before harvesting in May 2022 at Babeiqiao, Luhe District, Nanjing, P.R. China. The 3 wheat varieties used were Ningmai 13, Ningmai 16, and Yangmai 13. One hundred wheat spikes of each wheat variety were collected for the digital images, among which 60 wheat spike images were randomly sampled for the establishment of a mathematical model. The remaining spike images were used to verify the mathematical model. All images were collected under the same luminosity, at a focal distance of 5 mm and a height (object distance) of 12.5 cm. A wheat spike of Ningmai 13 is used to illustrate the image processing method. The original image of the collected wheat spike is shown in Figure 2a. In order to illustrate the digital image processing method; only one image and its processed images of wheat spike are shown. The grain number of the wheat spike was counted manually after the digital image of the wheat spike had been collected.

2.3. Image Processing Method

The digital image processing was conducted through a number of distinctive steps, which included image cropping, image graying, histogram equalization, image binarization, eroding operation, removing small objects, filling image hole, revolving vertical spike, cutting off the stem and removing the stem, and acquiring characteristic parameters from the processed image [23]. A mathematical model was established and verified according to the acquired characteristic parameters of the wheat spike images.

• Step 1: Image cropping

In order to save computing time, the part of the image region that was outside of the wheat spike was cropped automatically by a defined process, retaining only the wheat spike region. The pixels and dimensions of the whole wheat spike image region were not affected by the cropping step (Figure 2b).

• Step 2: Image graying

After the cropping step, the wheat spike image was grayed, as shown in Figure 2c. The detail and the edges of the wheat spike are clarified in the resulting gray level image. Spikelets and the stem are also shown clearly in this gray level image. In addition, enhanced contrast between the wheat spike and the background is obvious.

• Step 3: Histogram equalization

To further intensify the image contrast between the wheat spike region and its background, the gray histogram was equalized. An example of the resulting histogram equalization image is illustrated in Figure 2d, showing that the wheat spike image region and background region are entirely intensified, as well as the boundary of the wheat spike.

• Step 4: Image binarization

The image thresholding value for binarization was acquired through the Otsu method (maximum interclass variance method), which is a simple and quick algorithm for image processing. Automatic Otsu thresholding was applied to the wheat spike image binarization. An example of the resulting binary image of the wheat spike is shown in Figure 2e, in which the wheat spike region is clearly separated from its background region.

• Step 5: Eroding operation

Image eroding was applied to the binary image of wheat spike. A disk structure element was used to process the binary images of wheat spikes. The radius of the disk structure element was set to 7. One resulting eroded binary image of a wheat spike is shown in Figure 2f.

If set A is eroded by set B, the operation is expressed as A⊝B, where ⊝ is the erosion operation. The image eroding operation was defined as follows:

$$\mathrm{A}⊝\mathrm{B}=\left\{x\left|{\left(B\right)}_x\subseteq A\right.\right\}$$

Let set A be eroded by set B; the result belongs to point X set of set A after set B being moved (translated) by all X in set A.

The noise in the binary image was removed when the area covered by the noise was smaller than that of the disk structure element.

• Step 6: Removing small objects and filling image hole

Redundant objects covering an area in pixel numbers below the p-value were removed using the area thresholding method. At the same time, filling of the image holes was processed by the eroding operation. The hole filling operation can fill up all the holes in the image by the setup parameters for holes. Examples of processed binary images of a wheat spike after removing redundant objects and filling holes are shown in Figure 2g,h.

After the wheat spike image was processed by removing small objects and filling any hole, all small objects were removed effectively, retaining only the grain part of the wheat spike in the image (Figure 2h). Then, the characteristic parameters of the wheat spike carrying grain information could be acquired reliably.

• Step 7: Revolving vertical spike

The geometric center of the processed image was determined, and the center line of the spike image was rotated to align parallelly with the vertical axis of the image (Figure 2i).

• Step 8: Cutting off stem and removing stem

The total pixel number of the spike region in the transverse axis direction was larger than that of the stem region (Figure 2j). When the sum of pixel numbers for the stem was less than a specified amount, the stem was cut off (Figure 2k) and the stem part was removed (Figure 2l) [13].

2.4. Acquiring Morphological Characteristic Parameters

The dimensions and shapes of the pre-processed image were used to depict the morphological characteristic parameters of the wheat spike. Two characteristic parameters, i.e., the image area of the wheat spike and the length of the wheat spike axis, were calculated mainly according to the rules of operation for the production technology of wheat basic seed [1].

A pixel number with a value of 1 was computed as the image area of wheat spike.

The minimum circumscribed rectangle of the wheat spike image was drawn, with the long edge parallel to the transverse axis. The pixel number sum of the long edge of the rectangle was considered as the length of the wheat spike axis. The minimum circumscribed rectangle and length of the wheat spike axis are shown in Figure 2m.

2.5. Data Analysis

The relationships between the image area and grain number of the wheat spike and between the length of the wheat spike axis and grain number of the wheat spike were drawn on a scatter plot for all varieties of wheat spike. Regression analysis was performed with the least squares method, and linear curve fitting was applied to derive the equation and determination coefficient for each variety. The absolute difference between the practical grain number and the predicted grain number was defined as the absolute error. The average absolute error was the average value of the absolute error for all wheat spikes. The absolute difference between the practical grain number and predicted grain number dividing the practical grain number was defined as the relative error. The average relative error was the average value of the relative error for all wheat spikes.

3. Results and Discussion

3.1. Establishing Mathematical Model

Scatter plots of the relationships between the image area of wheat spike and the grain number of wheat spike for each variety and all varieties are illustrated in Figure 3. The image area increased linearly, and in a perfect manner, with respect to the grain number. It thus indicates that by using the image processing tool proposed in this paper, a linear relationship between the image area and grain number of wheat spikes could be found, which allows a precise prediction of wheat grain number from digital images.

The results of the regression analysis and the curve fittings using linear equations between the image area and the grain number of the wheat spikes are also shown in Figure 3. The determination coefficients R² between the image area and grain number of the wheat spikes for the three varieties were 0.844, 0.9151, and 0.9336, respectively, showing apparent linear relationships between the area image and grain number of the wheat spikes. The regressive linear mathematical models were Y = 0.0014X − 0.6356, Y = 0.0015X − 4.2873, and Y = 0.0014X − 3.6968, respectively, for the three wheat varieties.

The determination coefficient R² for all wheat spike varieties between the image area and grain number was 0.8571. The linear mathematical model was the regressive fitting line equation Y = 0.0013X − 1.5499.

Scatter plots of the length of the spike axis and the grain number of the wheat spike for each variety and all varieties are shown in Figure 4. The length of the spike axis was also found to linearly increase with the increase in the grain number. It therefore indicates that the proposed image processing method facilitates a linear relationship between the length of the spike axis and the grain number, which is an aid for phenotyping purpose.

The regressively fitted linear equations and the determination coefficients between the spike axis length and wheat spike grain number are illustrated in Figure 4. The determination coefficients R² between the spike axis length and wheat spike grain number were 0.7832, 0.8925, and 0.9023, respectively, for the three wheat varieties, indicating a significant linear relationship between the length of the spike axis and the grain number of the wheat spike. The linear mathematical models for the three wheat varieties were Y = 0.1259X − 12.943, Y = 0.1436X − 22.616, and Y = 0.1265X − 22.706, respectively.

The determination coefficient R² of regression between the length of the spike axis and the grain number of the wheat spike for the all wheat varieties was 0.7495, and the resulting mathematical model was Y = 0.1039X − 6.5114.

The determination coefficient R² between the wheat spike image area and grain number was larger than that between the length of the spike axis and the grain number of the wheat spike. The linear mathematical models between the image area and grain number of wheat spikes were superior to the linear mathematical models between the length of the spike axis and the grain number of wheat spikes.

3.2. Verifying the Mathematical Model

The linear mathematical models for each wheat spike variety between the image area and grain number and between the spike axis length and grain number were verified by using the remaining untreated 40 wheat spike samples from the same variety. All images of wheat spike samples for verifying the mathematical model were acquired and processed under the same conditions as the images of wheat spike samples for establishing the mathematical model.

For each variety, the grain number that was counted manually was used as the practical grain number, and the grain number that was calculated from the mathematical model after the wheat spike image was processed was used as the predicted grain number. The absolute error and relative error were used to determine the accuracy of the established mathematical model of the image area and the grain number of the wheat spike. Here, the mathematical models of the image area and grain number of the wheat spikes were used for the verification. The image area of the wheat spike (as x-value) was acquired and substituted into the previously established equations, and the result (as y-value) was calculated for the grain number of the wheat spike. The calculated y-value was the model predicted grain number. The correlative linear curve fitting and the equations between the practical grain number and the predicted grain number are shown in Figure 5.

The determination coefficients R² between the practical grain number and predicted grain number of the three wheat varieties were 0.8632, 0.8824, and 0.9552, respectively (Figure 5). This means that the established image-based mathematical model of the image area and grain number of wheat spikes predicted the grain number correctly.

Table 1. Analysis of average absolute error and average relative error for verifying mathematical models.

	Ningmai 13		Ningmai 16		Yangmai 13		All Varieties
	Image Area of Spike	Length of Spike Axis	Image Area of Spike	Length of Spike Axis	Image Area of Spike	Length of Spike Axis	Image Area of Spike	Length of Spike Axis
Average absolute error (Grain)	2.4	2.8	2.6	2.8	2.3	2.4	3.1	4.0
Average relative error (%)	5.65	6.80	6.66	7.11	5.92	6.04	8.12	10.56

shows the average absolute error and average relative error for the mathematical model of the image area and grain number of wheat spikes. The average absolute error value of Yangmai 13 was the lowest at 2.3 grains, and the average absolute error value of Ningmai 16 was the highest at 2.6 grains. The average relative error value of Ningmai 13 was the lowest at 5.65%, and the average relative error value of Ningmai 16 was the highest, reaching 6.66%.

The mathematical model of wheat spike grain number versus image area was verified for all wheat varieties by using the same 120 wheat spike samples. The determination coefficient R² between the practical grain number and the predicted grain number was 0.8626 (Figure 5). The average absolute error and average relative error for the mathematical model of the wheat spike image area and grain number were 3.1 grains and 8.12%, respectively.

The established linear mathematical models of the length of the spike axis of the wheat spike image and the grain number of the wheat spike were verified using the same method mentioned above. Figure 6 shows the correlative linear curve fitting and the equations between the practical grain number and predicted grain number.

The determination coefficients R² between the practical grain number and predicted grain number for the three wheat varieties were 0.8122, 0.8352, and 0.9369, respectively (Figure 6). This means that the established mathematical model of the length of the spike axis and the grain number of wheat spikes accurately predicted the grain number by using digital image processing.

The average absolute error and average relative error for the mathematical models of the length of the spike axis and the grain number of the wheat spike are shown in Table 1. The average absolute error value of Yangmai 13 was 2.4 grains, which was the minimum, and the average absolute error value of Ningmai 13 and 16 was 2.8 grains, which was the maximum. The average relative error value of Yangmai 13 was 6.04% (the minimum), and the average relative error value of Ningmai 16 was 7.11% (the maximum).

The mathematical model for all varieties of wheat spike between the length of the spike axis and the grain number of the wheat spike was again verified using the same 120 wheat spike samples. The determination coefficient R² between the practical grain number and predicted grain number was 0.7182 (Figure 6). The average absolute error and average relative error for this mathematical model were 4.0 and 10.56%, respectively.

The average absolute error and average relative error for the mathematical model of the spike axis length and the grain number were larger than those for the model of the image area of wheat spike and the grain number. This indicates that the accuracy of the established mathematical model of the image area of wheat spike and grain number was superior to that of the model of the spike axis length and the wheat spike grain number.

3.3. Discussion

Wheat spikes contain a large number of spikelets, with one to five grains housed in each spikelet. The higher the grain number is, the larger the spikelet’s area is, so the wheat spike’s area can be used to determine the grain number. On the other hand, the higher the grain number is, the higher the spikelet number is and the larger the length of the spike axis is, so the length of the spike axis could also represent the grain number. The results from this study indicate that the grain number of a wheat spike is related to the characteristics parameters of the wheat spike.

The total grain number of wheat spikes can be calculated quickly, conveniently and simply through collecting a one-sided image of the wheat spike. The high accuracy of the predicted grain number thus satisfies the requirement of counting the grain number of wheat spikes for yield prediction and phenotyping.

Wheat yield is governed by the number of wheat spikes per square meter, the grain number of wheat spikes, and the 1000-grain weight. Among them, counting the grain number of wheat spikes is the most time-consuming and tedious work, and also the most important step in measuring the crop yield. A number of researchers have studied counting methods of wheat spike number according to area in square meters and the 1000-grain weight, but reports counting the grain number of wheat spikes are scarce. In one study, the grain weight of wheat spikes was measured by testing the texture feature of wheat spike images [14], and in another, the grain weight of wheat spikes was also measured by the image area of wheat spikes [17]. Yet these approaches provided low measuring accuracy and were thus unable to predict the grain number of wheat spikes, an important parameter for breeders and researchers in analyzing the wheat phenotype.

In a more recent report, the grain number was obtained by using a one-sided image of a wheat spike that was detected based on deep learning; the determination coefficient (R²) of 0.81 between the predicted grain number and the true grain number was lower than the minimum determination coefficient (R²) of 0.86 in this study [22]. The grain number of wheat spikes was also obtained using X-ray computed tomography imaging, and the determination coefficient (R²) between the virtual grain number per spike and the manually counted grain number per spike was 0.989 [21]. However, X-ray computed tomography scanning of the wheat spikes and reconstruction of the 3D architecture of the wheat spikes took 10 min. The grain number of wheat spikes amounted to each grain number of spikelets, the absolute and relative errors of wheat spike grain numbers from Du et al.’s study were roughly equal with this study, though the complexity of the image processing methods and the counting method of wheat spike grain numbers has limited applicability [13].

The accuracy of the established mathematical models of the image area and grain number of wheat spikes and of the spike axis length and grain number of wheat spikes for each variety was different. The accuracy for Yangmai 13 was the highest, and the accuracy for Ningmai 13 was the lowest. The accuracy of the verified mathematical model for each variety was the same as the accuracy of the established mathematical model. The accuracy of the established mathematical model of the image area and grain number of wheat spikes for all varieties was also superior to the accuracy of the established mathematical model of the spike axis length and grain number of wheat spikes for all varieties.

4. Conclusions

Wheat spike images were acquired using an image collection system. The digital image processing approach involved pre-processing the images and acquiring characteristic parameters from the processed wheat spike images. The characteristic parameters of the wheat spike image, the image area, and the length of the wheat spike axis were acquired. Two linear mathematical models were established and verified for the image area and grain number of the wheat spikes and for the length of the wheat spike axis and the grain number of the wheat spike. The maximum determination coefficients R² for the practical and predicted grain numbers were 0.9552 and 0.9369, respectively, and the minimum average absolute error and average relative error for the mathematical models were 2.3 grains and 5.65%, respectively. The accuracy of the established mathematical model of the image area and grain number of the wheat spikes was superior to that of the established mathematical model of the length of the wheat spike axis and the grain number of the wheat spike from the verified two mathematical models.