1. Introduction
“Hupingzao” jujubes (
Zizyphus jujuba Mill.) can be eaten fresh, made into dried jujubes, or used as a raw material for many functional foods, due to their high nutritional value and medicinal function [
1]. The maturity of the fruit is affected by many factors (such as light and flowering time). Fruits of various maturity levels coexist among the harvested “Hupingzao” jujubes. There are differences in the component content of jujubes at different maturity levels, which leads to different tastes and uses for the fruit. In addition, the postharvest respiration intensity and the rate of water loss and weight loss are different when the ripeness period of fresh jujubes is different. The higher the maturity, the higher the respiration rate. The lower the maturity, the faster the water loss [
2,
3]. The maturity directly affects the shelf life and the storage conditions of fresh jujubes. Therefore, maturity is an important indicator in the grading and sorting of fresh jujubes, and thus has important significance for improving the added value of products and meeting the needs of processing, sales, and storage.
The peel color is an important feature for evaluating the ripeness of fresh jujubes [
4,
5]. The manual grading of the maturity of jujubes using visual methods is time-consuming and laborious, and is highly intensive with a low efficiency. Computer vision technology intuitively obtains the geometric structure and the appearance characteristics of samples [
6,
7]. This technology has the advantages of being noncontact, fast, nondestructive, and low-cost, and has been widely used in fruit quality detection and grading. Image processing is a core component of machine vision, and statistical analysis is one of its important theoretical foundations. In the current research, two image processing technologies, mainly based on the traditional machine learning and deep learning techniques, were used.
Image processing technology using the traditional machine learning technique has received much attention in the research on maturity classification for fruits. Based on the extracted color features of apricots, Khojastehnazhand et al. [
8] used linear and quadratic discriminant analyses to classify maturity, with accuracies of 0.904 and 0.923, respectively. Wan et al. [
9] realized an average accuracy of 99.31% for tomato maturity classification, using the color feature values and the backpropagation neural network. In the classification of the maturity of oil palm fresh fruit bunches, using the principal component analysis method, a data dimensionality reduction based on the color and texture features was performed [
10]. The accuracy using an artificial neural network with a backpropagation algorithm was 98.3%. Based on the selected geometric attributes and texture and color features produced by a dimensionality reduction, Azarmdel et al. [
11] used artificial neural networks and a support vector machine to classify three maturity stages of mulberry fruits, and the optimal classification accuracy of the prediction set was 99.1%. In the above research, the models established by machine learning achieved good prediction performances. However, in all cases, the required feature information was extracted manually from a small sample set. In the process of feature extraction, the number of features that can be generated is large. If the number of features exceeds a critical value, the performance of the classifier will decrease. For high-dimensional data, a dimensionality reduction process needs to be performed. This method is only suitable for specific objects, has a low efficiency, and a weak repeatability.
Deep learning automatically learns the features from the dataset, and the structure is flexible. In this method, the linear and nonlinear features are mined by effectively integrating the information between channels to adapt to the different learning strategies. The convolutional neural network (CNN) is a deep learning method that includes nonlinear transformation functions. CNN shares the convolution kernels and automatically extracts the features. The built model is transferable using CNN. CNN includes a variety of network structures, such as AlexNet [
12], VGG 16 [
13], ResNet [
14], Inception V3 [
15], and GoogLeNet [
16]. Gulzar et al. [
17] improved VGG 16, which was compared with machine learning methods in the classification of seeds, and the proposed model achieved the best results, with an accuracy of 99.9%. Loddo et al. [
18] compared the performance of models using several advanced CNN architectures, the proposed SeedNet model was the best, and the CNN models were all better than traditional machine learning methods. Taner et al. [
19] designed a CNN model (Lprttnr1) to classify hazelnut varieties, the accuracy of the proposed model was 98.63%. Hamid et al. [
20] used MobileNet V2 to classify 14 different levels of seeds, and the accuracies of the training and test sets were 98% and 95%, respectively. CNN has been widely used in the classification of varieties [
21,
22,
23], defects [
24], freshness [
25,
26], and maturity [
27,
28] of fruits, and achieved good results. However, the classification of maturity for “Hupingzao” jujubes based on CNN is rarely reported.
Transfer learning can shorten the training time, because it can transfer the relevant knowledge from the learned model to the new task model [
29,
30]. In the classification of the maturity of fruits, Xiang et al. [
31] used a CNN based on transfer learning to classify the maturity of mangoes, and the accuracy was 96.72%. Behera et al. [
32] performed the classification of papaya maturity using the traditional machine learning and the CNN of transfer learning, and the classification accuracy of the VGG19 based on transfer learning was 100%. The early typical CNN contains relatively deep layers and a network structure with high complexity, which mainly improve the classification accuracy of the model. Meanwhile, network efficiency has become another consideration. The lightweight network [
33,
34] is designed based on structural simplification or model compression, has a small volume, and increases speed; for example, ShuffleNet [
35] and MobileNet [
36].
In the actual images collected, problems with an insufficient data scale and unbalanced datasets are inevitable. When the amount of data in one category is obviously more than that in another category, the category with the large amount of data is given priority to learn, and the category with the small amount cannot be fully learned during the CNN model training process [
37]. The imbalance of data between the different categories affected the classification performance of the CNN training model [
38,
39,
40].
Therefore, this study will establish CNN classification models with different network structures based on class-balanced loss (CB) and transfer learning to achieve an efficient and accurate classification of maturity for “Hupingzao” jujubes. The main objectives of this research include: (1) To establish a MobileNet V2 lightweight model with a class-balanced loss (CB-MobileNet V2). (2) To discuss the effect of learning rates and optimizers on the performance of the CB-MobileNet V2 model based on transfer learning. (3) To analyze the impact of CB on the performance of the MobileNet V2 model, and compare the performance of CNN models with different network structures. (4) To develop a detection system for ‘Hupingzao’ jujube maturity, and test the performance of the model to realize a robust and accurate classification.
4. Conclusions
In this study, class balance loss was used to improve the MobileNet V2 network, and a CB-MobileNet V2 model with transfer learning was used to classify the maturity of “Hupingzao” jujubes. The optimizer and learning rate affected the performance of the model. Under the same training conditions, the model with a learning rate of 0.0001 achieved the best classification result. At a learning rate of 0.0001, the AdamW optimizer was better than Adam, ASGD, and SGD optimizers for classification results. Data imbalance among classes in datasets affected the performance of models. The application of class balance loss and transfer learning improved the performance of the MobileNet V2 model, and classification accuracy and F1 score were increased. Compared to MobileNet V2 based on transfer learning, the white maturity sample had the highest improvement in the validation results among samples of the five maturity levels using CB-MobileNet V2. The recall and F1 score were increased by 6.25% and 0.033, respectively. Class balance loss performed well in discriminating classes with a small sample size. Compared with CB-AlexNet, CB-GoogLeNet, CB-ShuffleNet, CB-Inception V3, CB-ResNet 50, and CB-VGG 16 network models with transfer learning, the CB-MobileNet V2 model achieved better classification results. The validation loss and accuracy were 0.055 and 99.058%, respectively. The precision was 96.800~100.000%, the recall was 95.833~100.000%, and the F1 score was 0.963~1.000. A maturity detection system of “Hupingzao” jujubes was developed, and the testing accuracy of the CB-MobileNet V2 model was 99.294%. The testing precision, recall, and F1 score were 97.917~100.000%, 95.918~100.000%, and 0.969~1.000, respectively. The CB-MobileNet V2 model exhibited a good overall performance. Therefore, this study achieved the maturity classification of “Hupingzao” jujubes. In the future, we plan to develop a multitask classification model for the maturity and defects of jujubes at the same time, and to develop software and hardware for automatic grading equipment for their comprehensive classification.