Determination of Moisture in Rice Grains Based on Visible Spectrum Analysis

Abstract

Rice grain production is important for the world economy. Determining the moisture content of the grains, at several stages of production, is crucial for controlling the quality, safety, and storage of the grain. This work inspects how well rice images from global and local descriptors work for determining the moisture content of the grains using artificial vision and intelligence techniques. Three sets of images of rice grains from the INIAP 12 variety (National Institute of Agricultural Research of Ecuador) were captured with a mobile camera. The first one with natural light and the other ones with a truncated pyramid-shaped structure. Then, a set of global descriptors (color, texture) and a set of local descriptors (AZAKE, BRISK, ORB, and SIFT) in conjunction with the dominate technique bag of visual words (BoVW) were used to analyze the content of the image with classification and regression algorithms. The results show that detecting humidity through images with classification and regression algorithms is possible. Finally, f1-score values of at least 0.9 were accomplished for global color descriptors and of 0.8 for texture descriptors, in contrast to the local descriptors (AKAZE, BRISK, and SIFT) that reached up to an f1-score of 0.96.

1. Introduction

The moisture parameters of rice are monitored from the harvest stage when the grain is normally harvested with moisture levels between 18 and 26% [1,2,3,4]. To ensure the safety of the grain and that it is free of fungi and insects, the grain is dried quickly after harvest at a moisture level below 15% [5,6,7]. During storage, the humidity of the rice must be maintained between 12 and 14%, with storage variables of 25 °C temperature and 13% humidity. Under this environment, the safety of the grain can be ensured for 17 months. On the other hand, with 14% humidity, the storage time can be reduced from 3 to 7 months, noting that storing the rice at 12% humidity ensures a longer storage time [8,9,10].

In the post-harvest processing stages, the moisture content (12–13%) of rice plays a very important role for the quality of the final product, mainly in the characteristics of whiteness and morphology of the final product [11,12]; however, during husking in rice milling, a humidity of 12 and 13% is inadequate, with the most appropriate humidity being 15–16%, where less rice breakage is observed [12]. The average per capita consumption of rice in the world and Ecuador is 58.4 and 53.2 kilos, respectively, being one of the most consumed products on the planet [13,14,15]. Rice consumption provides 20% of the world’s metabolic energy supply.

There are several methods for determining moisture in grain [16], but the most reliable methods are slow, and the fastest methods are not accessible to all farmers. Nevertheless, artificial intelligence has advanced by leaps and bounds in recent years. In this work, an alternative for training humidity classifier models is explored using computer vision algorithms and artificial intelligence.

Developments using computer vision to determine moisture in corn kernels, such as the work of [17], conclude that using this method has potential for determining moisture. In this study, an optimized set of weight, morphology, and color variables were used, and an R2 greater than 0.96 was obtained when linear regression was used.

It has also been used to quantify changes in grain morphological characteristics in Western Canadian wheat classes as a function of moisture content. This test was performed at moisture contents between 12 and 20%, using potassium hydroxide (KOH) concentrations to regulate relative humidity [18].

Changes in appearance and morphology of cereal grains attributed to moisture content were detected by the human eye and evaluated using computer vision techniques in a study examining three cereal species: Canada Western Amber Durum wheat (CWAD), Canada Western Red Spring wheat (CWRS), and barley. Grains were conditioned at moisture contents of 12, 14, 16, 18, and 20% to take photographs. Moisture content was found to significantly affect grain color and texture [19].

The quality of the grains has been also evaluated in terms of their physical factors such as moisture content, bulk density, grain size, hardness, grain density, and number of damaged grains. Using techniques, such as color image analysis, hyperspectral imaging, X-ray imaging, and thermal imaging, these techniques are being investigated for their advantages in controlling production chains and grain quality, as there is great interest in developing these technologies [20].

Figure 1 shows the stages used in the classification of grains according to the information in the overview in [21]. The first stage is image acquisition, which can be of different types; the second stage is preprocessing, in which the image is cropped, scaled, enhanced, and the colors are converted; the third stage is segmentation, in which the elements of interest in the image are obtained; and finally, there is a classification stage through several alternatives of algorithms for learning and classifying.

The technique called Bag of Visual Words (BoVW) has also been used to classify im-ages of weed species in some studies, as shown in [22]. In this work, the images were de-composed into a set of generalized features that corresponded to visual features. By ap-plying BoVW, relationships represented as frequency vectors and generalized features of reference images were generated.

The BoVW technique [23] consists of three steps: The first step consists in creating a visual dictionary based on a considerable number of images, which may be images of a different type and not necessarily plants, and from which information must be extracted using extractors and descriptors of keypoints. The second step consists of generating a set of tagged images for which the features of new images are extracted and declined, and from which their descriptors of keypoints refer to the visual dictionary according to their similarity. In a final step, an image classifier is trained using a support vector machine (SVM) [24]. Then, a new BoVW vector is generated for each image and related to the codebook. A generalized classifier is then used to decide to which specific category an image belongs.

In many applications based on the use of artificial vision, the process of image pairing is common, which includes five main stages: first, feature recognition and description; and second, determining the correspondence between the features of the images, rejecting atypical features, deriving the transformation function, and reconstructing the images [25]. For the detection and description process, the following algorithms can be used: SIFT [26], SURF [27], KAZE [28], AZAKE [29], ORB [30], and BRISK [31].

Once the features are constructed from the images, machine learning models, namely Random Forest (RF) and XGBoost, are used to perform a classification and regression analysis on the data. Both RF and XGBoost are ensemble machine learning models that fit several decision-based classifiers (trees) on various subsamples of the dataset and use averaging to improve the predictive accuracy and control overfitting. These algorithms perform well on small datasets; thus, they are suitable for the present problem [32,33].

The main goal of this work is to detect humidity in rice grain using computer vision techniques on visible spectrum images. In this work, both global and local descriptors are used for detecting moisture content in grains using images from mobile devices. Local descriptors are processed using the BoVW technique to define the vector of features. Once the features are engineered according to the discussed methods, Random Forest (RF) and XGBoost classifiers and regressor are constructed. The performance of the classifiers and regressor is explored and compared in detail.

The rest of the paper is organized as follows. The Methodology Section describes the procedures consideration of data acquisition preprocessing methods. The Results Section presents the classification and regression results with the different algorithms used, i.e., Random Forest and XGBoost classifiers and regressor. A comparison of their performances is presented with an analysis of variance. Finally, the Conclusion Section outlines the main conclusions of this experimental study and suggests future research.

2. Materials and Methods

This section describes the treatment and preparation of the rice grains used for im-aging, the techniques used to analyze the images, and the procedure for training the models to predict moisture content in rice grains.

2.1. Preparation of Rice Samples

The rice used in this study corresponds to the INIAP 12 variety harvested in August 2020 in Milagro Canton, Guayas Province, Ecuador. The rice fields are located at an altitude of 10 m above sea level. The harvesting process was carried out manually and the samples were stored in silos of 10 tons for a period not exceeding 10 h.

For the analysis of the images and the identification of their humidity, homogeneous samples with humidity between 10 and 15%, with a 1% increase, were prepared. That is, humidity is in {10, 11, 12, 13, 14, 15} humidity percent. The homogeneous samples were obtained using the gravimetric method of Gough [34], which allows for the application of Equation (1) for preparing the inhomogeneous batch at the corresponding humidifies.

Considering Gough’s method, homogeneous samples were prepared by conditioning 10 lots of 250 g each. Five moisture values were determined, corresponding to a moisture content between 10% and 15%, considering the data in [35]. Then, the grains were stored and kept in airtight bags at a refrigeration temperature of 5° C. To confirm the homogeneity of the samples, the moisture content of portions of the grains of each bag was determined. An analysis of variance was performed to determine if there was a significant difference between the samples of each standard according to their respective moisture content.

$$Q=\frac{\mathrm{A} \left(\mathrm{b} - \mathrm{a}\right)}{100-\mathrm{b}}$$

(1)

where Q corresponds to the weight of the water to be added in grams, A is the initial weight of the subsample in grams, a is a percentage of the initial moisture content of the subsample, and b is the percentage of the desired final moisture content in the subsample.

2.2. Image Acquisition

After the initial steps, independent samples corresponding to homogeneous moisture levels 10, 11, 12, 13, 14, and 15% were prepared. For each sample, a set of photographs was acquired for the analysis by using a mobile camera. In total, three different groups of samples were prepared and labelled as G01, G02, and G03 (Figure 2).

The samples in the G01 dataset were collected with natural white light and a cell phone fixed on a static base, which was a stand holder for the mobile phone. These samples consist of a multilayer image captured using natural white light (60 W). On the other hand, the G02 and G03 samples were taken using a truncated pyramid-shaped structure (see Figure 3). These last datasets consist of randomly scattered backlit images captured with white light source (60 W) in a single layer with similar light conditions.

In the truncated pyramid-shaped structure, a camera was set at the top while a container with a rectangular transparent bottom supporting the rice grains in such a way as to allow for the passage of white directional light from a lamp LED through the rice grains [36]. Additionally, to prevent the outside light from changing the intensity of the analyzed images inside the structure, a camera with outside light isolation was used. The height h between the camera lens and the lower position of the rice grains was 17 cm for acquiring all the sample images. Each time a photograph was taken, a refill was carried out. This means that the grain sample in the device was replaced by a new sample of the same class (moisture level).

A set of 75 photographs were taken for G01 with a resolution of 4032 × 3024 pixels, a flash illumination, and no image enhancement effect. The photographs were taken in an environment with 60 W of directional LED white light, without using the structure in Figure 2.

In group G02, a total of 70 images were taken with the device shown in Figure 2 and with a resolution of 1600 × 1200 pixels. A white LED directional lamp (60 W) was used for background illumination, and the grains were randomly scattered over the light source without checking whether there was a single layer of grains.

Finally, in group (G03), a total of 218 images were taken with a resolution of 3472 × 4624 pixels. The grains were also scattered in a single layer within the rectangular box and the same light source used in G02 was used.

2.3. Image Preprocessing for Feature Extraction

The photographs of the three groups were cropped before processing according to their resolutions to keep the central region information and to evaluate the power of dis-crimination of moisture based on the available information. Then, to obtain the image content, vectors of features with global and local descriptors were calculated.

2.3.1. Global Descriptors

The global features describe the whole image [37] as a vector. They are compact representations where each image corresponds to a point in a high-dimensional feature space, and they focus on color and texture.

• The color histogram is rotation and scale invariant and is used under the hypothesis that images with similar color model distributions are semantically similar. The color model is a mathematical representation that usually uses three or four different components [38]. Some common color models are Gray, HSV, and RGB.
  The grayscale color model (GRAY_CH) defines color by using only one component, lightness, which is measured in values ranging from 0 to 255. The RGB color model (RGB_CH) is a color model with three dimensions—red, green, and blue—that are mixed to produce a specific color. The HSV color model (HSV_CH) is a cylindrical color model that remaps the RGB primary colors into three dimensions that are easier for humans to understand.
  The color moments (HSV_CM) represent the color distribution by three moments: the average is the first-order moment, the variance is the second-order moment, and the skewness is the third-order moment.
• Local binary patterns (LBP) [39] look at points surrounding a central point and test whether the surrounding points are greater than or less than the central point. It is illumination and translation invariant. The image is converted to grayscale, and a histogram is computed with a mask of P = 16 and R = 10.
• Haralick (HRLK) [40] distinguishes between rough and smooth surfaces using the gray level co-occurrence matrix (GLCM), which uses the adjacency concept in images. It looks for pairs of adjacent pixel values that occur in an image and keeps recording it over the entire image.
• A gray level co-occurrence matrix (GLCM) is a histogram of co-occurring gray-scale values at a given offset over an image [41]. It is created in four directions with the distance between pixels as one. Texture features are extracted from the statistics of this matrix according to the correlation of a couple pixels’ gray-level value at different positions.

The global descriptors are evaluated with a 10-fold cross validation technique, which is a data resampling method (without replacement) to prevent overfitting and to assess the generalization ability of predictive models. Each time, one of the K subsets is used as the testing set and the other K−1 subsets are put together to form the training set and compute the f1-score on the test set [42]. The f1-score is an error metric that measures model performance by calculating the harmonic mean of precision and recall [43].

2.3.2. Local Descriptors

Besides, local features refer to image neighborhoods computed at multiple interest points in the image. They are consequently more robust to occlusion and clutter. Features consist of keypoints and descriptors [44]. Key points are the “stand out” points in an image, so no matter whether the image is rotated, shrunk, or expanded, its keypoints will always be the same. In addition, a descriptor depicts the keypoint in the image.

In this study, the algorithms of SIFT, BRISK, ORB, and AKAZE were used to detect local features.

• Scale Invariant Feature Transform (SIFT) [44] is robustly invariant to image rotations, scale, and limited affine variations, but its main drawback is high computational cost.
• Binary Robust Invariant Scalable Keypoints (BRISK) [45] is invariant to scale, rotation, and limited affine changes. It uses an easily configurable circular sampling pattern from which it computes brightness comparisons to form a binary descriptor string.
• Oriented FAST and Rotated BRIEF (ORB) [15] is invariant to scale, rotation, and limited affine changes.
• AKAZE [46] is invariant to scale, rotation, and limited affine changes and has more distinctiveness at varying scales because of nonlinear scale spaces.

Once local features were detected, the bag of visual word (BoVW) method [47] was used to quantify and to describe regions in and around keypoints of interest (Figure 4). Detected keypoints were treated like words and were used to construct a vocabulary with the k-means clustering algorithm to identify groups of similar local descriptors and to take some of them to represent an image. The keypoints were quantified and pondered by the Tf-Idf strategy to represent the images by a histogram (Equation (2)).

$$x_i=\frac{n_i}{N_D}log\left(\frac{N}{N_i}\right)$$

(2)

The Tf-Idf value increases proportionally to the number of times a word appears in an image but is offset by the frequency of the word in the image collection. It is represented in Equation III, where n_i is the number of repetitions of each characteristic (word) i in the image, N_D is the total number of characteristics in the image, N is the total number of images used to construct the vocabulary, and N_i is the number of the image where the characteristics appear. Afterwards, the resulting histogram is reduced into a vector for each humidity level, and each image is represented as a frequency histogram.

Then, histograms are normalized, and a supervised approach is used with two classifications and two regression algorithms. The classifiers are the Random Forest classifier (RFC) and XGBoost classifier (XGBC) and the regressors are the Random Forest regressor (RFC) and XGBoost regressor (XGBR). The Random Forest algorithm consists of several simple decision trees that use averaging to improve the predictive accuracy and control overfitting by building estimators from a bootstrap sample from the training set [48]. On the other hand, the XGBoost algorithm is also a decision tree ensemble designed to be highly scalable that provides a parallel tree boosting based on gradient boosting [49].

A common ratio for evaluating the models is a 70/30 split for the training and testing datasets [50], selected without replacement. It is because the visual words vocabulary for the BoVW method was created dynamically for each training set using a k-means clustering algorithm, and this process was computationally expensive. Finally, the effectiveness of these descriptors is also evaluated using the measured f1-score.

3. Results

3.1. Global Descriptors

The global descriptors capture the information from the whole image by a single feature vector, thus cutting the images to keep the central regions. Figure 5 schematically illustrates how the borders are gradually removed, and after a few steps, the final new image just contains the central area. It is anticipated that, as the region of the grains is smaller, the global descriptors have a higher and then a lower value of the f1-score. It is because the image becomes too small and there is not enough information, or the outer border region of the box is noise, given that the edges represent other information related to the surface.

Results of regression and classification algorithms for the different cropping are presented in Figure 6, Figure 7 and Figure 8. The color global descriptors, the most basic quality of the visual contents, are beneficial to describe the moisture levels independently of the analyzed color space (Gray, HSV, RGB). It is also shown that descriptors of the images taken in controlled conditions like G02 and G03 seem to give better performance than G01.

Besides, results show that texture global descriptors (LBP, Haralick, GLCM) also al-low for detecting humidity in G03 in classification and regression algorithms but do not perform well for discriminating the humidity levels when there is not enough information, like G01 and G02. This can be explained due to visual patterns in the images in those datasets that do not contain enough structural information of surfaces and their relationship to the surrounding environment to differentiate the humidity intra-class.

Classifiers and regressors allow for discriminating the grain moisture with global features. The f1-scores obtained with color and texture features suggest a good performance with both techniques; however, classifiers seem to work a little better. The results also suggest that, with enough information, a prediction model can be trained with an image size of 940 × 940 pixels based on their color features.

Figure 9 shows the average f1-score of these images with the different algorithms. It shows that there are visual patterns of color and texture to discriminate the different moisture levels, especially in controlled environments. It also shows that the color descriptors stand out for their good performance with little information.

Subsequently,

Table 1. Average f1-score performance of different classifier algorithms with global descriptors in 940 × 940 images.

Global Descriptors		Dataset	Feature Size	RFC	XGBC	RFR	XGBR
Color histogram 1D	CRAY_CH	G01	92	0.66	0.55	0.66	0.6
Color histogram 3D	HSV_CH	G01	1728	0.73	0.7	0.57	0.55
Color histogram 3D	RGB_CH	G01	8000	0.75	0.6	0.65	0.58
Color moments	HSV_M	G01	9	0.51	0.55	0.57	0.67
LBP	LBP	G01	34	0.72	0.69	0.68	0.62
GLCM	GLCM	G01	24	0.53	0.51	0.58	0.62
Haralick	HRLK	G01	13	0.53	0.55	0.6	0.49
Color histogram 1D	CRAY_CH	G02	20	0.81	0.66	0.63	0.7
Color histogram 3D	HSV_CH	G02	8000	0.72	0.7	0.46	0.63
Color histogram 3D	RGB_CH	G02	8000	0.8	0.72	0.65	0.65
Color moments	HSV_M	G02	9	0.73	0.72	0.65	0.71
LBP	LBP	G02	34	0.83	0.80	0.7	0.7
GLCM	GLCM	G02	13	0.73	0.71	0.69	0.69
GLCM	HRLK	G02	24	0.73	0.64	0.69	0.67
Color histogram 1D	CRAY_CH	G03	44	0.96	0.92	0.92	0.87
Color histogram 3D	HSV_CH	G03	8000	0.9	0.92	0.86	0.85
Color histogram 3D	RGB_CH	G03	8000	0.87	0.9	0.9	0.86
Color moments	HSV_M	G03	9	0.75	0.77	0.72	0.7
LBP	LBP	G03	32	0.75	0.78	0.65	0.58
GLCM	GLCM	G03	13	0.87	0.85	0.83	0.84
Haralick	HRLK	G03	24	0.85	0.81	0.81	0.79

shows the average f1-score for the evaluated prediction algorithms and global features with cropped images (940 × 940) of each dataset—G01, G02, and G03. The f1-score values suggest that G02 and G03 have a better f1-score with most global descriptors compared to G01. This can be related to the truncated pyramid-shaped structure used for capturing these datasets. A small group of images like G01, which were captured with natural light, is propene to noise due to external factors; consequently, finding patterns is more difficult, and a bigger dataset is required to handle this noise with other techniques.

Likewise, the results evidence that color descriptors have a better f1-score with values higher than 0.8 in images taken under controlled conditions and an f1-score of 0.7 when images were not taken under controlled conditions, as is shown with the descriptors RGB_GH and HSV_CH in G01. Moreover, LBP texture descriptors report f1-score values higher than 0.75 in G03.

Regarding the results, one-way ANOVA was performed with Tukey’s post hoc test over the f1-score values of each classification and regression algorithm, and a p-value of 0.00147 was reached, which means that there are statistically significant differences between them (

Table 2. Tukey’s multiple comparisons of classifiers’ and regressors’ f1 mean scores.

	Mean Difference	Lower Estimation	Upper Estimation	p-Adjusted
RFR-RFC	−0.037281746	−0.08877360	0.01421011	0.2423261
XGBC-RFC	−0.003769841	−0.05526170	0.04772201	0.9975838
XGBR-RFC	−0.069207058	−0.12069891	−0.01771520	0.0033528
XGBC-RFR	0.033511905	−0.01797995	0.08500376	0.3344428
XGBR-RFR	−0.031925312	−0.08341717	0.01956654	0.3782033
XGBR-XGBC	−0.065437216	−0.11692907	−0.01394536	0.0063473

). In addition, a Tukey’s test was used to perform a pairwise comparison (Figure 10) with a 95% confidence level to see differences between groups. The difference between groups XGBC-RFC and XGBR-XGBC is statistically significant because the intervals for the mean differences do not contain the zero. The pairwise results are reported in Table 2, and for the groups with a small p-value (<0.05), a similar conclusion can be yielded.

3.2. Local Descriptors

On the other hand, the local descriptors analysis, which identifies prominent image regions that have rich local information (such as color or texture) and are more robust to occlusion and clutter, also reported good results. Figure 10 shows the performance of them when cropping the borders of the image, and it shows that, when the image is very small, there is not enough information to identify local descriptors and to define a vocabulary and a feature vector. The smallest image required to identify the descriptors is 146 × 146 pixels for images taken in a controlled environment.

Figure 11 suggests that, with enough visual information, AKAZE, BRISK, and SIFT local descriptors are significantly better for moisture detection because most of their f1-score values were over 0.75, and they achieved values higher than 0.8 with an image size of at least 660 × 660 pixels.

The local features were also evaluated with cropped images in the same way as the global descriptors in

Table 3. F1-score over the local features with the BOWN technique.

Local Descriptors	Feature Size	RFC	XGBC	RFR	XGBR
BRISK	70	0.96	0.87	0.93	0.88
SIFT	80	0.93	0.85	0.79	0.74
AKAZE	70	0.98	0.78	0.82	0.72
ORB	70	0.67	0.59	0.68	0.58

. Handling these features requires more computational re-sources and time for training; however, the results indicate a better performance for deter-mining the different levels of humidity. The reported f1-score values were over 0.9 for BRISK, SIFT, and AKAZE local descriptors. In addition, a one-way ANOVA test was also performed, and a p-value of 0.00114 was obtained. This demonstrates again that there are statistically significant differences between the algorithms.

SIFT is robust local descriptor that can reliably identify objects even in messy situations and with partial occlusion [26]; however, its calculation speed is low, which makes it complicated to use in real-time applications.

On the other hand, AZAKE, ORB, and BRIEF are faster binary descriptors based on the intensity information because they encode information in a series of numbers like a fingerprint. In terms of f1-score, the AZAKE, BRISK, and SIFT local descriptors perform well for detecting humidity levels, and ORB is the worst.

4. Conclusions

In this work, global and local descriptors were used to detect humidity moisture in images taken with natural light and under controlled conditions. Various image features were engineered and used to fit Random Forest and Boost classifier and regressor models.

The best results for classification and regression algorithms were achieved with texture and global features due to the f1-score reaching above 0.85. The classifiers detect moisture with an f1-score of up to 0.96 for color descriptors (GRAY_H) and up to 0.87 for texture descriptors (GLCM). The regressors also reached values of 0.92 with color descriptors (GRAY_H) and 0.84 with texture descriptors (GRAY_H). It was evidenced that the color feature has a key role in humidity prediction.

The BoVW technique is also an efficient image representation in the humidity levels classification task; however, a significant amount of time and data is required to train a classifier. The f1-score values over 0.9 suggest that they are good for detecting moisture levels with the BRISK, SIFT, and AKAZE local features The classifiers reached an f1-score of 0.98 with AKAZE, 0.96 with BRISK, and 0.93 with SIFT; in a similar way, the regressors reached a value of 0.93 with BRISK, 0.82 with AKAZE, and 0.79 with SIFT.

For future work, images will be collected under different conditions for extracting a variety of color and texture patterns, as well as performing a segmentation analysis. Thus, robust patterns can be obtained to better handle noise and explore samples of grains un-der a combination of different moisture levels. It is also recommended to increase the dataset size due to exploring other robustness techniques like neural networks and deep learning classifiers.