Identification of Rice Freshness Using Terahertz Imaging and Deep Learning

Abstract

Retention of rice freshness is highly dependent on storage temperature. Timely and accurate identification of rice freshness is critical to ensure food security. Here, we characterize the freshness of rice in reference to storage temperature. Terahertz reflection imaging is a non-destructive and deeply penetrating technique that can be used for detecting rice freshness. Due to the shortcomings of traditional machine learning, such as limited processing of nonlinear problems and insufficient computing power. Deep learning has the advantages of strong learning ability and high portability. Therefore, for rice freshness identification, the VGG19 network and the Inception-ResNet-v2 network were used in this paper. Moreover, we propose an improved 1D-VGG19-Inception-ResNet-A network. This network possesses the advantages of low time consumption from the 1D-VGG19 network and high classification accuracy from the 1D-Inception-ResNet-V2 network. Compared with the traditional algorithms, the accuracy of the proposed network is significantly improved, with the rice freshness recognition accuracy of 99.80%. The experimental results indicate that terahertz spectral imaging and deep learning algorithms are viable tools for monitoring rice freshness.

1. Introduction

Rice is one of the most important grain foods in the world. More than half of the world’s population consumes rice [1]. It provides carbohydrates, proteins, fats, and lipids [2,3]. The chemical composition of rice is about 85% starch, 7% protein, and 0.3% lipids [4]. However, the quality of rice is highly dependent on its storage temperature. An inappropriate temperature will degrade the freshness of rice quickly. If it happens on a large scale, it will cause significant economic losses. Therefore, regular monitoring of the rice quality and freshness is mandatory. Several methods [5,6] have been introduced to measure rice freshness, such as near-infrared spectroscopy, electronic nose, gas chromatography, and mass spectroscopy. We believe that they have obvious advantages. For example, near-infrared spectroscopy (NIR) is a fast and non-destructive analytical technique with high resolution, utilization rate, and reliability. The electronic nose is a technology that simulates human olfactory organs. It has the advantages of fast response, sensitivity to multiple odors, and low maintenance costs. Gas chromatography (GC) is a separation and analysis technique for mixed gases or liquid samples, with high separation efficiency, sensitivity, and selectivity. Mass spectrometry is an analysis technique with high sensitivity, high resolution, and good reliability. Although these techniques are reasonably sensitive and accurate, they are time-consuming, expensive, and require sophisticated sample preparation before testing. Therefore, a rapid, accurate, non-destructive, and reliable method for measuring rice freshness is still desired.

Terahertz (THz) radiations [7], located between radio waves and infrared light, are electromagnetic waves ranging from 0.1 to 10 THz. The most important use of THz radiation was in the development of THz spectroscopy which has evolved into a tool of tremendous applications due to the development of modern ultrafast optics [8,9]. Terahertz imaging technology has unique advantages in both spectrum and imaging, containing a large amount of image and spectral information. By combining the spectral characteristics of terahertz images with their spatial characteristics, target feature recognition can be achieved from multiple angles. In addition, terahertz images provide more visual and pictorial information. Compared to spectral data, terahertz images can present features such as structure, morphology, and distribution within the sample.

Machine learning algorithms are widely used for classifying THz spectra data and imaging data [10,11]. However, machine learning algorithms can mostly classify and identify targets from shallow features [12], while deep learning algorithms have different convolutional and pooling layers, which can extract deeper features of the target [13]. Therefore, deep learning can compensate for the shortcomings of machine learning in some situations. VGG19 and Inception-ResNet-v2 are classical deep learning algorithms with excellent feature extraction performance [14,15]. However, they have different advantages. For example, the VGG19 network has a simple structure and low time consumption. The Inception-ResNet-v2 network has high accuracy. A new deep learning network based on VGG19 and Inception-ResNet-v2 networks is proposed to obtain fast and efficient classification results. It is called 1D-VGG19-Inception-ResNet-v2. The proposed network improves accuracy compared to the VGG19 network and reduces model training time compared to the Inception-ResNet-V2 network.

In this work, the THz reflection imaging technique was used to obtain different images of rice with different levels of freshness. Then the proposed deep learning network, 1D-VGG19-Inception-ResNet-v2, was used to recognize the freshness levels of the rice. The results demonstrate that. the improved network can shorten the network training time while ensuring an increase in accuracy. Meanwhile, the results prove the feasibility of detecting rice freshness using THz imaging and deep learning.

2. Materials and Methods

2.1. Sample Preparation

According to the literature [16], it presents that the fatty acid value is one of the important indicators to judge whether aging occurs during rice storage, and the higher the fatty acid value, the lower the freshness of the rice. The reference [17] presents that the fatty acid values increased fastest when samples were stored at a high temperature, and the rising rate can be effectively delayed at low temperatures. Therefore, for comparative experiments, different storage temperatures were chosen to prepare the rice samples, including 10 °C low temperature, 25 °C room temperature, and 38.5 °C high temperature storage environments in this study. These three temperatures were chosen because the three temperatures are typical and representative.

The Japonica rice used in the experiment was purchased from Guangzhou Inspection and Testing Technology Co., Ltd(Guangzhou, China). We know that storage temperature has a significant effect on the freshness of the rice. Therefore, the rice samples were stored in wide-mouthed bottles and kept at three different temperatures, 10 °C, 25 °C, and 38.5 °C for one month.

2.2. Image Acquisition

The rice samples with different storage temperatures were placed on the moving platform of the THz system for reflective imaging. The maximum scanning area of the platform was 50 mm × 50 mm, and the spatial resolution was set to 2 mm. A complete THz waveform can be acquired at each pixel position with 9000-domain points in the time range of 90 ps. The terahertz image contains spatial information and spectral information. The original data of the terahertz image obtained using terahertz reflection imaging belongs to the three-dimensional structure, so the original three-dimensional image of the rice is presented here, as shown in Figure 1.

2.3. Spectral Information Extraction

The external changes in the freshness of rice are not so obvious that we can not observe them with the naked eye. However, the resolving properties of terahertz spectroscopy are more sensitive to small changes in the substance. Therefore, in this paper, rice with different storage temperatures was used to represent different freshness levels.

In Figure 2a, we can see that the terahertz image of the rice sample has a three-dimensional data structure. However, this THz image has a large amount of background interference and poor image quality. In order to acquire better image quality, peak-to-peak imaging was carried out, as shown in Figure 2b. It can also be seen that the color of the rice sample is darker than the background color. We used a three-dimensional map shown in Figure 2c. It can be seen that the amplitude of the sample is small, and the amplitude of the background is large. It is because the rice significantly absorbs the terahertz wave. In Figure 2c, it can be seen that amplitude changes sharply where color has a distinct change. The change is evident at 1200 a.u. Therefore, the amplitudes corresponding to fewer than 1200 a.u are selected as the region of interest for rice grains.

After the region of interest is determined, the spectral information corresponding to each pixel is extracted, and the spectra of each rice grain are averaged, as shown in Figure 3. In the 25 ps time domain, the terahertz spectrum shows typical characteristics that the higher the storage temperature, the higher the amplitude peak. The reason for this phenomenon is that temperature increase has an effect on the crystal water in the grain, which in turn leads to changes in the amplitude values of the terahertz time-domain spectra [18].

2.4. Partial Least Squares Regression Analysis

Partial least squares regression (PLSR) analysis, a multivariate data analysis algorithm, combines the advantages of correlation analysis, multiple linear regression, and principal component analysis. It can perform regression modeling for small samples and multiple correlations [19,20]. Therefore, PLSR analysis is used to analyze the spectrum of the region of interest and the background region. It is to determine the effectiveness of the spectrum selection. In this experiment, the independent variable X is the 2800 amplitude values corresponding to each THz spectrum, and the dependent variable Y is the rice and the background spectral labels.

2.5. 1D-VGG19 Network

The VGG19 network [21], a deep learning algorithm, has a simple structure. Using a deep learning network structure for spectral data can increase the number of network calculations. Therefore, a conversion from the 2D convolutional operation of VGG19 into a 1D convolutional operation is introduced. The specific structure of the 1D-VGG19 network is shown in Figure 4. The improved 1D-VGG19 network has the same structure as the 2D-VGG19 network, which is composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The difference between the 1D-VGG19 network and the 2D-VGG19 network is the scale of input data, convolutional kernel, and maximum pooling. For example, in a 1D-VGG19 network, the input data is 2 dimensions, and the convolution kernel and maximum pooling dimensions are 3 × 1 and 2 × 1, respectively. The input data in the 2D-VGG19 network is 3D, and the convolution kernel and maximum pooling dimensions are 3 × 3 and 2 × 2, respectively. The one-dimensional convolutional operation is shown in Figure 5, and their function is expressed in the following equation.

$$\left\{\begin{array}{c}{x}_j^l=f\left(u_j^l\right)\\ u_j^l={\displaystyle \sum _{i\in M_j}^n}{x}_i^{l-1}\ast k_{ij}^l+b_j^l\end{array}\right.$$

(1)

where $l$ represents the $l$-th layer convolution; $j$ represents the $j$-th channel of the $l$ convolution layer; $f(.)$ represents the activation function of the network model, which is usually the ReLU function; $x_j^l$ represents the output value of the activation function; $b_j^l$ represents the bias value of the $j$-th channel of the $l$-th layer convolution; $k_{ij}^l$ represents the convolutional kernel vector of the $i$-th input feature vector corresponding to the $j$-th channel of $l$th layer convolution.

2.6. 1D-Inception-Renest-v2 Network

Inception-ResNet-V2 was developed by introducing the residual network idea into the Inception-V4 network [22]. However, the deep learning network is not suitable for the classification of one-dimensional data. Therefore, the two-dimensional convolution operation of Inception-ResNet-V2 was converted into a one-dimensional convolution operation. The network structure is shown in Figure 6.

The input layer in the 1D-Inception-ResNet-V2 network is used to read data. The middle layers are mainly used for feature extraction, which includes convolutional and pooling operations. The corresponding modules are Inception-ResNet-A, Inception-ResNet-B, and Inception-ResNet-C. Their structures are shown in Figure 7. Since the input spectral data is a 2-dimensional structure, in the Inception-ResNet-B and Inception-ResNet-C modules, the convolutional with 7 × 7 and 3 × 3 kernel sizes are decomposed into 1 × 1, 7 × 1, 1 × 1, and 3 × 1 kernel sizes. Average-pooling layers with kernel size set to 1 × 3 are used to select features and spectral information. Dropout layers are used to prevent the model from overfitting. After passing through the above operations, the softmax function as the activation function is used to determine the rice spectrum. After determining the rice spectrum by the softmax function, the output data format is in 3 × 1 format.

2.7. 1D-VGG19-Inception-ResNet-A Network

The Inception-ResNet-V2 network involves complex structures, many calculations, and lengthy data processing time [23]. Moreover, the Inception-ResNet-A module in the Inception-ResNet-V2 network enables feature extraction under different receptive fields [24]. The VGG19 network, on the other side, has the advantage of simple structures. Nevertheless, it only extracts limited features. Therefore, this study introduces the Inception-ResNet-A module in Inception-Restnest-v2 and VGG19 network structures. The purpose is to increase the number of convolutional layers, to extract features under different receptive fields for effective feature extraction. The specific structure of the network is shown in Figure 8.

2.8. Model Evaluation Indicators

The evaluation indicators [25] for multi-classification problems mainly include precision, recall, and accuracy. The calculation formulas of various evaluation indicators are as follows.

$$\mathrm{Pr}ecision=\frac{TP}{TP+FP}$$

(2)

$$\mathrm{Re}call=\frac{TP}{TP+FN}$$

(3)

$$F1-score=\frac{2\ast \mathrm{Pr}ecision\ast \mathrm{Re}call}{\mathrm{Pr}ecision+\mathrm{Re}call}$$

(4)

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(5)

where $TP$ is for the true positive; $TN$ stands for the true negative, $FP$ for the false positive, and $FN$ for the false negative.

In order to compare the effectiveness and advantages of the proposed model, the experimental results are compared with the improved VGG19 algorithm proposed by other scholars [26].

3. Results and Discussion

3.1. Discrimination of Spectral Validity

Using the amplitude value to select the region of interest for rice granules is a random process. With this method, the selection region of the rice granule can be too large or too small. The selection region that is too large or too small will result in the selected spectrum, which is not a true representative of the entire population. Therefore, to verify the validity of the selected spectral data, PLSR analysis was used to perform regression analysis on the selected rice and background spectra. Before regression analysis, rice and background spectra were labeled 1 and 2, respectively. All data were divided into training and test datasets in the 7:3 ratio. The experimental results show that the error between the actual and the predicted values is small. According to Figure 9, it can be seen that the sample labels are primarily distributed around y = 1 and y = 2. PLSR analysis was used to calculate the correlation coefficient R, mean square error, and root-mean-square error which are 0.91, 0.01, and 0.1, respectively. In order to accurately obtain the error value of the samples, the actual value was subtracted from the predicted value. The result is shown in Figure 10. It can be seen that the error between the predicted value and the actual value is mainly distributed from −0.2 to 0.2. Through the PLSR analysis, it can be seen that PLSR analysis can clearly distinguish between the selected rice and the background spectra. It means the spectra of the selected rice granule can be effectively used for classifying and identifying rice.

3.2. Discrimination Results of Different Classification Networks

In order to achieve accurate and rapid identification of rice with 10 °C, 2 °C and 38.5 °C storage temperatures, 1D-VGG19, 1D-Inception-Renest-v2, and 1D-VGG19-Inception-ResNet-A networks are used to determine rice freshness. The best model is determined by comparing their four evaluation indicators and training times for the test set. During the experiment, the 2849 spectral data were acquired and divided into a training set and test set in a ratio of 3:1. To ensure that the network is in the same variable environment, the network learning rate was set to 0.0001, and the number of epochs was set to 100. Moreover, the loss function was set per the cross-entropy loss function, and the optimizer was adjusted with the Adam optimizer.

3.2.1. Identification of Rice Freshness by 1D-VGG19 Network

Experimental results, after 1D-VGG19 network analysis, are shown in Figure 11. It can be seen that the training was completed when the number of epochs reached 28. Fluctuations of the loss value and accuracy rate were minimal. After 100 epochs, the loss and accuracy of the test set were 0.0917 and 0.9759, respectively. It took 113s to complete the entire testing process. Corresponding to the confusion matrix, as shown in Figure 11b, the rice stored at low temperatures can be accurately identified.

3.2.2. Identification of Rice Freshness by 1D-Inception-ResNet-V2 Network

After 1D-Inception-ResNet-V2 network analysis, the experimental results are shown in Figure 12. It can be seen that the network acquired convergence after 30 epochs. After 100 epochs, the loss and accuracy of the 1D-Inception-ResNet-V2 network on the test set were 0.0453 and 0.9839, respectively. It took 466 s to complete the entire testing process. Corresponding to the confusion matrix, the rice stored at low temperatures can be accurately identified, as shown in Figure 12b.

3.2.3. Identification of Rice Freshness by 1D-VGG19-Inception-ResNet-A Network

The improved 1D-VGG19-Inception-ResNet-A network based on 1D-VGG19 combines the advantages of 1D-VGG19 and 1D-Inception-ResNet-V2. After 30 epochs, the loss value and the accuracy rate of the network show a state of convergence. After 100 epochs, the test set loss value and accuracy rate were 0.0051 and 0.9980, respectively. It took 204 s to complete the entire testing process. As the corresponding confusion matrix shown in Figure 13b, rice stored at 10 °C low temperature and 25 °C room temperature can be accurately identified.

According to the experimental results, the network has a clear improvement in testing time and classification accuracy compared to other networks.

3.3. Discussion

After training with the three deep learning networks, the corresponding accuracy, precision rate, recall rate, F1 score, and testing time on the test set were all acquired for each network. The specific data is shown in

Table 1. Experimental results of different networks.

Network Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	Testing-Time (s)
1D-VGG19	97.59	97.70	97.13	97.39	113
1D-Inception-ResNet-V2	98.39	98.20	98.37	98.28	466
1D-VGG19-Inception-ResNet-A	99.80	99.83	99.74	99.78	204
LiteVGGNet [27]	91.57	90.82	91.40	91.05	115

. It can be seen that the test set accuracy of the three networks can reach more than 97%. However, the significant difference is in the test times. The 1D-Inception-ResNet-V2 network took the longest time of 466 s with an accuracy of 98.39%. The 1D-VGG19 network took the shortest time of 113 s with a lower classification accuracy than the 1D-Inception-ResNet-V2 network. The accuracy rate and testing time of the proposed ID-VGG19-Inception-ResNet-A network is 99.80% and 204 s, respectively. Moreover, in terms of time, the proposed algorithm in this study has obvious advantages compared with the improved algorithms proposed by other scholars.

In summary, by comparing different methods and evaluation indexes, it can be found that the deep learning improved algorithm proposed in this paper has better classification accuracy and efficiency. Furthermore, experimental results demonstrate that the improved network in this paper is suitable for identifying the simple and small number of rice samples.

4. Conclusions

This study presents a way of classifying rice freshness, which combines THz imaging and 1D-VGG19-Inception-ResNet-A deep learning network. The terahertz images of rice at different storage temperatures were obtained. After pretreatment, the time-domain spectra of rice at different storage temperatures were input to the 1D-VGG19-Inception-ResNet-A network. The result demonstrates that the classification recognition accuracy of the proposed network can achieve 99.80% with a testing time of 204 s. By comparing the results, we found that the 1D-VGG19-Inception-ResNet-A network has improved accuracy compared to the 1D-VGG19 network. It also has a shorter testing time compared to the 1D-Inception-ResNet-V2 network. The terahertz reflective imaging combined with the improved deep learning algorithm can accurately and rapidly identify rice freshness. This research is of great significance to ensure the quality of agricultural products.