Next Article in Journal
Automated System-Level Malware Detection Using Machine Learning: A Comprehensive Review
Previous Article in Journal
Hydration, Reactivity and Durability Performance of Low-Grade Calcined Clay-Silica Fume Hybrid Mortar
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Nectarine Disease Identification Based on Color Features and Label Sparse Dictionary Learning with Hyperspectral Images

1
College of Information Science and Engineering, Shanxi Agricultural University, Jinzhong 030801, China
2
College of Agricultural Engineering, Shanxi Agricultural University, Jinzhong 030801, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(21), 11904; https://doi.org/10.3390/app132111904
Submission received: 25 September 2023 / Revised: 19 October 2023 / Accepted: 24 October 2023 / Published: 31 October 2023
(This article belongs to the Section Agricultural Science and Technology)

Abstract

:
Fruit cracking and rust spots are common diseases of nectarines that seriously affect their yield and quality. Therefore, it is essential to construct fast and accurate disease-identification models for agricultural products. In this paper, a sparse dictionary learning method was proposed to realize the rapid and nondestructive identification of nectarine disease based on multiple color features combined with improved LK-SVD (Label K-Singular Value Decomposition). According to the color characteristics of the nectarine itself and the significant color differences existing in the three categories of nectarine (diseased, normal, and background parts), multiple color spaces of RGB, HSV, Lab, and YCbCr were studied. It was concluded that the G channel in RGB space, Y channel in YCbCr space, and L channel in Lab space can better distinguish the diseased part from the other parts. At the model-training stage, pixels of the diseased, normal, and background parts in the nectarine image were randomly selected as the initial training sets, and then, the neighboring image blocks of the pixels were selected to construct the feature vectors based on the above color space channels. An improved LK-SVD dictionary learning algorithm was proposed that integrated the category label into the process of dictionary learning, and thus, an over-complete feature dictionary with significant discrimination was obtained. At the model-testing stage, the orthogonal matching pursuit (OMP) algorithm was used for sparse reconstruction of the original data, which can obtain the classification categories based on the optimized feature dictionary. The experimental results show that the sparse dictionary learning method based on multi-color features combined with improved LK-SVD can identify fruit cracking and rust spot diseases of nectarines quickly and accurately, and the average overall classification accuracies were 92.06% and 88.98%, respectively, which were better than those of k-nearest neighbor (KNN), support vector machine (SVM), DeepLabV3+, and Unet++; the identification results of DeepLabV3+ and Unet++ were also relatively high, but their average time costs were much higher, requiring 126.46~265.65 s. It is demonstrated that this study can provide technical support for disease identification in agricultural products.

1. Introduction

With high nutritional value, nectarines contain a variety of amino acids that are essential for the human body. At the same time, they can enhance immunity, which has high medicinal value. Hence, using hyperspectral imaging technology to realize the nondestructive identification of nectarines is a pivotal step in the process of nectarine industrialization [1]. In the process of the picking, preservation, and storage of nectarines, they are vulnerable to pests, diseases, and microbial pollution, which can lead to a great decline in product quality. Fruit surface defects caused by diseases have always been an important factor affecting the quality and price of fresh fruits. Fruit cracking and rust spot diseases are the most common physiological diseases of nectarines during their growth, and they are prone to occur from their break stage. Fruit cracking is a disease in which the fruit is exposed due to vertical or horizontal cracking nearing maturity, and the cracked fruit decays easily and become acidic [2]. The main causes of fruit cracking include deformational flowers, strong radiation, high temperature and drought, improper use of plant growth regulators, and excessive pinching. Rust spot is a fungal disease in which a layer of rust spots will appear in the fruit surface, and then, it begins to lose water and shrink as it is infested by the pathogen, resulting in a large number of fruit drops and a large-scale reduction in the yield. The main causes of rust spots include improper cultivation, an unsuitable climate and environment, pesticide and harmful dust pollution, mechanical damage, and pest infestation [3]. These diseases can seriously reduce the yield or result in a loss of commercial value. Therefore, we focus on these two diseases for research, and disease identification in nectarines is of great significance to improve their quality and market competitiveness [4].
Recently, with abundant spatial and spectral information, hyperspectral imaging systems have been widely used in the quality assessment for agricultural products [5,6]. The ripeness of strawberries has been estimated using a hyperspectral imaging system in field and laboratory conditions, which showed better performance than other methods [7]. Al-Alimi et al. [8] proposed a novel hyperspectral image classification framework using the meta-learner technique to train multi-class and multi-size datasets by concatenating and training the hybrid and multi-size kernels of convolutional neural networks (CNNs). At present, several studies on the internal and external quality assessment of nectarines have been conducted. The RPI (ripening index) and IQI (internal quality index) [9] have been used to assess the internal physicochemical properties and sensory perception of two cultivars of nectarine using VIS-NIR (visible near-infrared) hyperspectral images, showing great potential for further monitoring the evolution of intact nectarine ripeness in industrial setups. Various dimensionality reduction methods [10] have been used to extract the feature vectors from the dielectric spectrum and near-infrared spectrum, which can systematically reflect the advantages and disadvantages of the two spectrums in the quality assessment of nectarines, and the correlation coefficient and root mean squared error (RMSE) of the prediction set were 0.887 and 0.782, respectively. Hyperspectral image technology [11] combined with CARS-ELM (competitive adaptive reweighted sampling, CARS; extreme learning machine, ELM) has been successfully applied to realize the variety identification for nectarines, and the correlation coefficient and RMSE of the prediction set were 0.942 and 0.205, respectively, which provides the technical basis for the internal quality assessment of fruits. Nevertheless, most of the above studies aimed at the assessment of internal quality and variety identification of nectarines, and studies on disease identification are still in the theoretical analysis phase [12], let alone identification based on hyperspectral image technology. Since disease identification based on hyperspectral images belongs to the field of pattern recognition, sparse representation combined with dictionary learning algorithms has been widely used in face recognition [13], image classification [14], and other fields. Song et al. [15] established an HSI denoising algorithm by applying dictionary learning and sparse coding theory, extended into the spectral domain, and their experiments illustrated that the denoising result obtained using the proposed algorithm is at least 1 dB better than those of the comparison algorithms. Li et al. [16] proposed two multitemporal dictionary learning algorithms, expanding on their K-SVD and Bayesian counterparts, and the results were effective. Therefore, how to effectively apply the sparse representation and dictionary learning methods to identify the diseased area is the focus of our study.
The objective of this work was to develop a method for identifying different nectarine diseases rapidly and nondestructively. “Zhongyou NO.9” variety nectarines with fruit cracking and rust spots were taken as study objects, and visible hyperspectral images of the two diseases were collected using a hyperspectral imaging system. Firstly, addressing the problems of the high dimensionality and linear inseparability of hyperspectral image data, a recognition model for common nectarine diseases based on dictionary learning and sparse representation was studied. Secondly, as feature extraction is the most important and difficult part in pattern recognition [17]; moreover, there were significant color differences among the diseased, normal, and background parts. According to the influence of different color features on the recognition results we can obtain an optimal feature vector so as to enhance the reliability and robustness of the model. Finally, in the process of dictionary learning, as the recognition area included three categories, diseased, normal and background, an improved LK-SVD algorithm was proposed that integrated the category label into the dictionary learning process, and then a discriminant over-complete feature dictionary was obtained for the sparse reconstruction of the original data to obtain the classification categories. This study will provide a basis for nondestructive and on-line identification in nectarine and other agricultural products.
The core contributions of this paper can be summarized as follows.
We deeply analyze the color features in the nectarine hyperspectral images according to the color characteristics of the nectarine itself and the significant color differences that exist in the three categories of nectarine (diseased, normal, and background parts).
We propose an improved LK-SVD dictionary learning method that differs from the traditional machine learning method. Multiple color features combined with the improved LK-SVD are proposed.
Through dictionary optimization, a dictionary with a strong category discrimination ability and corresponding linear classifiers can be obtained, which optimizes the algorithm while reducing the amount of computation. Applications for disease identification in nectarine using hyperspectral imaging are lacking. As the research object of this study is disease identification, samples with fruit cracking and rust spots were selected for the experiment, and then, the samples were arranged based on the hyperspectral-image-acquisition system for hyperspectral data collection.

2. Materials and Methods

2.1. Data Collection

The experimental nectarines were picked from the Yuncheng orchard, Shanxi Province, in June 2021. To ensure the reliability of disease identification, the nectarine samples were similar in shape and uniform in maturity and size (each weighing 160–205 g) [18]. A total of 290 samples with four types were collected, including 70 fruit cracking samples, 72 rust spot samples, and 148 intact samples.
The HyperSIS (USA) hyperspectral-image-acquisition system used in this experiment was mainly composed of a CMOS camera, a spectrograph, an electronically controlled displacement platform with an array detector, a computer and a darkroom, as shown in Figure 1. This setup can be used for the hyperspectral data collection of agricultural products. The spectral range was 874–1734 nm, the resolution was 2.8 nm, and the sampling interval was 0.59 nm. Figure 2 shows the spectral curve of the fruit cracking samples. The light was a 150 W quartz halogen lamp. A total of 56 hyperspectral sample images were collected by the imaging system. The image size was 320 × 349, each with 256 bands. The visible nectarine sample images are shown in Figure 3, which are pseudo-color images synthesized using multiple bands.

2.2. Color Feature Analysis

The hyperspectral data and color feature analyses were conducted using ENVI 4.7 and MATLAB 2019, respectively. Since nectarines possess red and green colors, there were obvious color differences among the diseased, normal, and background parts. Therefore, we analyzed the multiple color spaces of RGB, HSV, YCbCr, and Lab [19] for feature extraction. Figure 4 shows the results of the different color spaces with fruit cracking.
As can be seen from Figure 4, since the HSV color space is composed of hue, saturation, and lightness [20], none of its channels can be used to identify the diseased part. The G channel in RGB (Figure 4b), Y channel in YCbCr (Figure 4g), and L channel in Lab (Figure 4j) can better distinguish the diseased part from the other parts; as for the remaining channels, the diseased parts were similar to the normal ones in color, and the boundaries were blurred, which cannot be used for identification. Therefore, the G channel in RGB, Y channel in YCbCr, and L channel in Lab were selected for color feature extraction. In general, the color features were pixel-level and the feature dimension of each channel was 256 × 256; thus, feature dimensionality reduction was necessary. Moments can describe the image features: low-order moments can reflect the low-frequency (main) information, and high-order moments can reflect the high-frequency (detailed) information [21]. Therefore, this study intended to extract the first, second, and third moments of the above-mentioned color channels as feature vectors for subsequent analysis. Then, the feature vector “Clolor_features” was defined as RGB_G_FM, YCbCr_Y_FM, Lab_L_FM, RGB_G_SM, YCbCr_Y_SM, Lab_L_SM, RGB_G_TM, YCbCr_Y_TM, and Lab_L_TM, which contains nine components.

2.3. Improved LK-SVD Sparse Dictionary Learning Method

2.3.1. Sparse Dictionary Learning

The main idea of dictionary learning is to use the dictionary matrix to linearly sparsely represent the original samples [22], as shown in Equation (1):
Y = D X
where D R m × k represents the over-complete dictionary; X R k × n represents the sparse matrix; Y R m × n represents the original samples; m and n are the dimension and number of samples, respectively; and k is the number of dictionary atoms. The essence is to find X , making D as sparse as possible.
Generally, sparse representation mainly includes two parts: sparse coding and dictionary updating [23]. In this study, OMP [24] was used to sparsely decompose the input information and calculate the reconstruction error; the improved LK-SVD algorithm was used to construct and update the dictionary.

2.3.2. Improved LK-SVD Dictionary Learning

The initial dictionary is not usually the optimal one, and there will be considerable error between the data represented by the sparse matrix that meets the sparseness with the original data. The K-SVD algorithm takes the principle of minimum error as the basic idea [25] to update the dictionary, and its objective equation is Equation (2):
min D . X { Y D X F 2 }   s . t . i , x i 0 T 0
where T 0 represents the upper limit of the non-zero sparse coefficient.
The K-SVD algorithm can effectively reduce the within-class deviation, but its learning process is only effective for a certain category and cannot increase the between-class variance for multi-class problems. Therefore, an LK-SVD algorithm based on category labeling was proposed, which integrates the category information to modify the K-SVD. This study involved a three-category classification issue: diseased, nectarine, and background parts. For this, sparse reconstruction mainly judges the type of test samples by solving the position where the minimum value of the residual appears in the sparse expression. Thus, the above problems can be replaced by a linear classifier, and then, the classification can be expressed as Equation (3):
H = W X + b
where H [ 0 , 1 , 2 ] represents the three categories, W represents the linear classification matrix, and b is the bias term. By integrating this into the process of dictionary learning, the optimization of solving W can be converted to Equation (4):
min H , W H W X b 2 2 + β W F 2
Combined with the formula in Equation (2), the above formula can be converted to Equation (5):
min D , X , H , W Y D X 2 2   + λ H W X b 2 2 + β W F 2     s . t . i x i 0 T 0
where W F 2 is the regular term and λ and β are the contribution values of the corresponding terms. A dictionary can be considered a combination of the primitive atom Y and thus can be expressed as D = Y Ω , in which Ω is the transformation matrix. Then, the above formula can be simplified as Equation (6):
min D , X , H , W [       Y λ H ] [ Y Ω λ W ] X F 2       s . t . i x i 0 T 0
By solving the above problems, the obtained H is the measured category. According to the above description, we obtain the implementation pseudocodes of the LK-SVD algorithm, as shown in Algorithm 1.
Algorithm 1 LK-SVD algorithm
Input: original sample matrix Y , transformation matrix Ω , label matrix H .
Output: over-complete dictionary D, linear classification matrix W .
Initialization: initialize Ω and W .
1: Stage 1: Model building
2:  Build a sparse representation model: Y = D X
3:  Build a Linear classification model: H = W X + b
4: Stage 2: Dictionary and classifier optimization
5:  Model fusion: Combining two optimization problems into one
6:   min D , X , H , W [       Y λ H ] [ Y Ω λ W ] X F 2       s . t . i x i 0 T 0
7:  Apply K-SVD dictionary learning to solve the above optimization problem

2.4. Algorithm-Implementation Process

Nectarine disease identification based on sparse dictionary learning is a method of supervised learning. Figure 5 shows the identification flow chart for this study. The recognition process mainly included two parts: the model training process and the model testing process. The images with fruit cracking and rust spot disease were divided into training sets (351 fruit cracking and 372 rust spot) and testing sets (117 fruit cracking and 124 rust spot) at a ratio of 3:1, and the recognition results are the statistical values of the testing sets. In order to evaluate and analyze the recognition results, the “Image Labeler” tool in MATLAB 2019 was used to label the disease images, which generated the ground truth maps. Figure 6b is the ground truth map of a sample image of fruit cracking disease, among which, “1” represents the background part, “2” represents the normal nectarine part, and “3” represents the diseased part.

2.4.1. Model Training Process

The training process for the identification of nectarine disease included four processes [26]: feature point extraction, feature vector construction, feature dictionary initialization, and dictionary learning. The operating system was Windows 10, the CPU model was an Intel (R) Core (TM) i7-12700F [email protected], the GPU model was an NVIDIA GeForce RTX 3080, the running memory was 32 GB, and the hard drive was 1 T. The programming environment used for the improved algorithm in this study was MATLAB 2019.
  • Feature point extraction: N feature points were respectively selected from the diseased, normal, and background areas as the initial training sets, and then, the initial dimension of the training sets was 3 N . For each pixel, the M × M neighborhood image block was extracted for recognition, and then, the sizes of the training sets were M × M × 3 N .
  • Feature vector construction: The data of the G channel in RGB, Y channel in YCbCr, and L channel in Lab of each image block were extracted, respectively. Then, the first, second, and third moment features of each channel were extracted. Finally, the dimension of the feature vectors was 9 × 3 N .
  • Feature dictionary initialization: Certain columns of the initial sample were selected as the initial feature dictionary. In this study, the K features of each category in 2 were randomly selected as the initial dictionary. By generating the transformation matrix between the dictionary and the initial sample randomly, the initial feature dictionary D was constructed, and its dimension was 9 × 3 K .
  • Dictionary learning: It was generally considered that there was a linear classification relationship between the features and categories, so a linear classification model was constructed using the category label H with the feature dictionary D . Here, the over-complete dictionary and the linear classifier were obtained using the LK-SVD algorithm iteratively.

2.4.2. Model Testing Process

The testing process of the nectarine disease identification included four processes [27]: feature point extraction, feature vector construction, sparse representation, and sparse reconstruction.
  • Feature point extraction: Referring to Section 2.4.1 (1.), all of the pixels of the disease image were extracted, and then, the size of the training sets was M × M × 256 × 256 .
  • Feature vector construction: The construction process was as in Section 2.4.1 (2.), and then, the dimension of the feature vectors was 9 × 256 × 256 .
  • Sparse representation: The sparse representation matrix was obtained by adopting the OMP algorithm using the over-complete dictionary obtained in Section 2.4.1 (3.) and the feature vectors in 2.
  • Sparse reconstruction: By inputting the sparse expression obtained in Section 2.4.2 (3.) into the linear classification model, the test category can be obtained.

2.5. Model Evaluation

In general, the confusion matrix, overall accuracy of classification, user accuracy, producer accuracy, and kappa coefficient were used to evaluate the classification results. The confusion matrix is mainly used to compare the classification results with the actual measured values [28]. Here, r is the category; X i j represents the percentage of category i judged as category j by the classifier in the total number of categories i ; X i i is the number of pixels in row i and column i in the confusion matrix (the number of correct classifications); X i + and X + i are the total number of pixels in row i and column i , respectively; and N is the total pixels.
The overall classification accuracy [29] (OA) is equal to the sum of the correctly classified pixels divided by the total pixels, as shown in Formula (7):
O A = i = 1 r X i i i = 1 r j = 1 r X i j
User accuracy [30] (UA) indicates the probability that a certain type of sample is correctly classified, as shown in Formula (8):
U A = X i i X i +
Producer accuracy [31] (PA) represents the probability that a certain type of sample in the classification diagram is correctly classified, as shown in Formula (9):
P A = X i i X + i
The kappa coefficient [32] can make full use of the information from the confusion matrix. It can be used as a comprehensive index for the evaluation of classification accuracy. Table 1 shows the relationship between classification quality and kappa statistics, and the calculation formula of the kappa coefficient is (10) as follows:
K = N i = 1 r X i i i = 1 r ( X i + X + i ) N 2 i = 1 r ( X i + X + i )

3. Results

The initial experimental parameters set during the training process were as follows: (1) in the feature point extraction, 1500 (500 points per category) pixels were randomly selected as the training points; (2) the initial size of the neighborhood image block was 7 × 7; (3) the initial dimension of the feature vector was 9; (4) the initial dictionary size was 300, and 100 features of each category were randomly selected as the initial dictionary.

3.1. Disease Recognition Results

Figure 6 shows the recognition results of the nectarine disease identification. Figure 6c shows the fruit cracking recognition result with a feature vector dimension of 9. Table 2 shows the corresponding confusion matrix results, which are respectively expressed in numerical form and percentage form. As can be seen from Table 2, the sum of the rows is the total pixels of the predicted result for each category, the sum of the columns is the total pixels of the ground truth values for each category, and the diagonal elements are the correct prediction pixels of all categories. By calculating the sum of the diagonal elements, it can be concluded that the correct prediction pixels of all categories is 61910, and the OA is 94.47%. The classification accuracies of diseased, normal, and background parts are relatively high, achieving more than 91%, and the classification accuracy of the background parts is the highest at 98.10%, proving that this method has a better effect.
The UA and PA of each category can also be calculated using the confusion matrix. Table 3 shows the UA and PA corresponding to Figure 6c. It can be seen that the UA and PA of the background and normal parts are both higher, reaching over 91%; the UA of the diseased part is the lowest at 55.34%. This is mainly because the border between the nectarine and the background is blurred with shadows, and it is easy to be confused with the diseased part; therefore, most of the edge of the nectarine parts and a few background parts are identified as being diseased.

3.2. Acquisition of Optimal Testing Parameters

In order to verify the influence of the feature vector dimension, the number of feature points extracted, and the size of the neighborhood image block on the classification results, a comparative analysis was conducted to obtain the optimal parameters. In this study, three features with the first moment, six features with the first and second moments, and nine feature vectors with the first, second, and third moments were constructed, respectively. The feature points of 1500, 2400, and 3000 were respectively selected, and the neighborhood image block sizes of 3 × 3, 5 × 5, and 7 × 7 were respectively selected. Figure 6 shows the results of the disease recognition using different feature vector dimensions, and Table 4 presents the statistical results of the different testing parameters. As can be seen from Figure 6, the classification results shown in Figure 6c,i are the best. It can also be seen from Table 5 that among the statistical results of fruit cracking and rust spot disease, when six feature vector dimensions are used, the recognition results are the highest at 90.92% and 86.86%, respectively, and the corresponding kappa coefficient classification quality is considered “excellent”. The number of feature points extracted has little effect on the recognition results; therefore, the influence on the recognition result can be ignored. The size of the neighborhood block has a significant impact on the recognition results, among which the size of 7 × 7 is the best.

3.3. Identification Results Using Different Dictionary Sizes

In order to verify the influence of the dictionary size on the classification results, this study compared the results of different dictionary sizes, as shown in Table 5, as an average of 100 trials. Figure 7 is a line chart of the average OA of different dictionary sizes. It can be seen from Table 5 and Figure 7 that the average OA of fruit cracking disease is higher than that of rust spot; the effect of the dictionary size on fruit cracking is also more obvious than that on rust spot. When the dictionary size for fruit cracking disease is 1350, the average OA and kappa coefficient are both the highest, at 92.06% and 0.92%, respectively. When the dictionary size for rust spot disease is 450, the average OA is the highest at 88.98%. With an increase in the size of the dictionary, the average OA and kappa coefficient for rust spot disease change smoothly.

3.4. Identification Results of Different Methods

Table 6 presents the classification results compared with SVM, KNN, DeepLabV3+ [33], and Unet++ [34], in which the distance of KNN was the Euclidean distance. The kernel function of SVM is the Gaussian kernel, and the optimal values of the penalty parameter and γ were obtained using the network-searching algorithm and the 10-fold cross validation method, which were 32 and 0.005. The learning rate in the training process of DeepLabV3+ and Unet++ was 0.0001, the value of the batch_size was 8, the value of momentum was 0.9, and the number of iterations was 300. Compared with the other methods, the identification results of our method in terms of identifying fruit cracking and rust spot disease were the highest, with an average OA of 92.06% and 88.98%, respectively. Compared with deep learning methods, the average time costs of SVM, KNN, and our method are obviously smaller, and the three are similar. Though the identification results of DeepLabV3+ and Unet++ are also relatively high, their average time costs are much higher at 126.46~265.65 s. As the datasets in this study were relatively small, they are not suitable for deep learning models. Therefore, the insufficient number of samples also affects the deep learning results. The experimental results show that the method proposed in this paper can effectively identify disease from the nectarine images and that the reconstructed fruit cracking images are better than those of rust spot disease.

4. Discussion

In the disease recognition results, the classification accuracies of diseased, normal, and background parts are relatively high, achieving more than 91%, and the classification accuracy of the background parts is the highest at 98.10%, proving that this method has a better effect. In the process of the acquisition of optimal testing parameters, the size of the neighborhood block has a significant impact on the recognition results, among which the size of 7 × 7 is the best. In the identification results with different dictionary sizes, the average OA of fruit cracking disease is higher than that of rust spot, and the effect of the dictionary size on fruit cracking is more obvious than that of rust spot. In terms of the identification results of the different methods, the method proposed in this paper can effectively identify the disease nectarine images, and the reconstructed image of fruit cracking is better than that for rust spot disease. We found that the sparse dictionary learning method based on multi-color features combined with improved LK-SVD can identify fruit cracking and rust spot in nectarine quickly and accurately, which can provide technical support for disease identification in agricultural products.
Reference [35] proposed an improved discriminative K-singular value decomposition (D-KSVD) algorithm for face recognition. In the D-KSVD algorithm, the initialization of dictionary D was implemented manually, which should guarantee that all categories were included in the dictionary. When human error occurs, the classification ability of the dictionary is incomplete. In the algorithm used in this study, we used D = Y Ω to generate the initialized D randomly in the early stage of dictionary optimization; thus, the dictionary selection was dynamically adjusted according to the limited conditions. Furthermore, the dictionary optimization process can be converted from semi-manual to fully automatic, so we were able to develop a dictionary with strong category discrimination ability and linear classifiers.
We added comparisons between machine learning methods and the recently advanced deep learning models of DeepLabV3+ and Unet++. Deep learning models need a GPU environment with a deep learning framework and additional acceleration libraries to conduct complex calculations, while our method does not rely on high-performance devices. Furthermore, the datasets in this study were relatively small, which are not suitable for deep learning models. In order to meet the requirements of deep learning for datasets, we used flipping, rotation, and other methods to expand the datasets. Finally, the OA results that we obtained were close to our method. Though the identification results of DeepLabV3+ and Unet++ were also relatively high and the average OAs for fruit cracking were 91.58% and 91.95%, their average time costs were much higher. Therefore, considering the high equipment costs, small sample, and long running time, the method proposed in this paper provides a basis for more effective fruit disease identification.
Research has been conducted on the identification of root rot disease in grapevine leaves [36], bacterial spot [37], yellow leaf curl disease in tomato [38], and leaf diseases in strawberry [39] using hyperspectral imaging. But, in Reference [37], only the spectral dimension information was used for disease identification, and there were no visualization identification results; in references [36,38,39], the research objects were leaves, which had no direct effect on fruit flavor. In this study, we propose an improved LK-SVD dictionary learning method focused on nectarine fruit disease identification that will affect the taste and sales of fruits directly. A dictionary with a strong category discrimination ability and corresponding linear classifiers can be obtained. We also obtained good visualization identification results, and these results can provide a basis for disease identification in other fruits.

5. Conclusions

In this paper, we proposed a sparse dictionary learning method to realize the rapid and nondestructive identification of nectarines disease based on multiple color features combined with the improved LK-SVD. Combined with the color characteristics of the nectarine itself and the diseased parts, a feature-vector-construction method based on multi-color space was proposed. At the same time, the statistical characteristics of image blocks were fully considered in the process of feature dimension reduction, which lays the foundation for subsequent experiments. An improved LK-SVD algorithm was proposed to integrate the category labels of the diseased, normal, and background parts into the process of dictionary learning so as to obtain an over-complete dictionary. As a result, the reconstructed image of fruit cracking is better than that of rust spot. The shortcoming was that the improved LK-SVD dictionary learning method can only optimize linear classifiers, but the results of the identification of nectarine diseases can meet our requirements. As the research object in this study was the identification of fruit cracking and rust spot disease, the experimental datasets may have certain limitations. In order to determine the generalizability of the model by evaluating its performance based on samples of unseen data, diseased fruits infected by other pathogens, like brown rot, bacterial spot, or peach scab ant, could be added into the training process as further research. In the future, we will continue to extend this method for other, more complex applications, and the realization of online disease identification is an important area of work as well. We also want to identify different phases of fruit disease.

Author Contributions

Conceptualization, R.M.; methodology, J.W.; software, R.M. and J.W.; validation, H.Y.; formal analysis, H.Y.; investigation, F.H.; resources, F.H.; data curation, R.M. and J.W.; writing—original draft preparation, R.M.; writing—review and editing, H.Y. and F.H.; visualization, J.W.; supervision, H.Y.; project administration, F.H.; funding acquisition, H.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Shanxi Province Applied Basic Research Youth Project, grant number 202203021212428; Shanxi Province Applied Basic Research Youth Project, grant number 202203021212414; Shanxi Agricultural University Youth Science and Technology Innovation Fund, grant number J141902200.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to thank the anonymous reviewers for their constructive comments, which considerably improved the quality of the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pantelidis, G.; Mavromatis, T.; Drogoudi, P. Consecutive wet days may impede fruit quality of peach and nectarine and cause fruit drop. Sci. Hortic. 2021, 282, 110011. [Google Scholar] [CrossRef]
  2. Lal, N.; Kumar, A.; Pandey, S. Screening of Litchi Genotypes for Fruit Cracking and the Relationship of Cracking to Fruit and Leaf Traits. Erwerbs-Obstbau 2022, 65, 479–485. [Google Scholar] [CrossRef]
  3. William, H.M.; Singh, R.P.; Huerta-Espino, J.; Palacios, G.; Suenaga, K. Characterization of genetic loci conferring adult plant resistance to leaf rust and stripe rust in spring wheat. Genome 2006, 49, 977–990. [Google Scholar] [CrossRef] [PubMed]
  4. Casagrande, E.; Génard, M.; Lurol, S.; Charles, F.; Plénet, D.; Lescourret, F. A process-based model of nectarine quality development during pre- and post-harvest. Postharvest Biol. Technol. 2021, 175, 111458. [Google Scholar] [CrossRef]
  5. Tao, H.; Feng, H.; Xu, L.; Miao, M.; Yang, G.; Yang, X.; Fan, L. Estimation of the Yield and Plant Height of Winter Wheat Using UAV-Based Hyperspectral Images. Sensors 2020, 20, 1231. [Google Scholar] [CrossRef]
  6. Sabar; Saputro, A.H.; Imawan, C. Moisture Content Prediction System of Dried Sea Cucumber (Beche-de-mer) Based on Visual Near-Infrared Imaging. In Proceedings of the 2019 6th International Conference on Instrumentation, Control, and Automation (ICA), Bandung, Indonesia, 31 July–2 August 2019. [Google Scholar]
  7. Gao, Z.; Shao, Y.; Xuan, G.; Wang, Y.; Liu, Y.; Han, X. Real-time hyperspectral imaging for the in-field estimation of strawberry ripeness with deep learning. Artif. Intell. Agric. 2020, 4, 31–38. [Google Scholar] [CrossRef]
  8. AL-Alimi, D.; Al-qaness, M.A.A.; Cai, Z.H.; Dahou, A.; Shao, Y.X.; Issaka, S. Meta-Learner Hybrid Models to Classify Hyperspectral Images. Remote Sens. 2022, 14, 1038. [Google Scholar] [CrossRef]
  9. Munera, S.; Amigo, J.M.; Blasco, J.; Cubero, S.; Talens, P.; Aleixos, N. Ripeness monitoring of two cultivars of nectarine using VIS-NIR hyperspectral reflectance imaging. J. Food Eng. 2017, 214, 29–39. [Google Scholar] [CrossRef]
  10. Shang, L.; Gu, J.; Guo, W. Non-destructively detecting sugar content of nectarines based on dielectric properties and ANN. Trans. Chin. Soc. Agric. Eng. 2013, 29, 257–264. [Google Scholar]
  11. Zhao, X.T.; Zhang, S.J.; Liu, J.L.; Sun, H.X. Study on Varieties Discrimination of Nectarine by Hyperspectral Technology Combined with CARS-ELM Algorithm. Mod. Food Sci. Technol. 2017, 33, 281–287. [Google Scholar]
  12. Yang, L.J.; Xu, Y.X.; Zhou, J.; Li, S.F.; Lu, M.G. Molecular characterization of nectarine stem-pitting-associated virus from China. J. Plant Prot. 2020, 47, 143–149. [Google Scholar]
  13. Xing, D.; Fda, B.; Hsab, C.; Yj, D. A multi-scale three-dimensional face recognition approach with sparse representation-based classifier and fusion of local covariance descriptors. Comput. Electr. Eng. 2020, 85, 106700. [Google Scholar]
  14. Kong, Y.; Wang, T.; Feng, Z.; Chu, F. Discriminative dictionary learning based sparse representation classification for intelligent fault identification of planet bearings in wind turbine. Renew. Energy 2020, 152, 754–769. [Google Scholar] [CrossRef]
  15. Song, X.R.; Wu, L.D.; Hao, H.X.; Xu, W.P. Hyperspectral Image Denoising Based on Spectral Dictionary Learning and Sparse Coding. Electronices 2019, 8, 86. [Google Scholar] [CrossRef]
  16. Li, X.; Shen, H.; Zhang, L.; Yuan, Q.; Yang, G. Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning. IEEE Trans. Geosci. Remote Sens. 2014, 52, 7086–7098. [Google Scholar]
  17. Sharma, S.; Tiwari, S.K. A novel feature extraction method based on weighted multi-scale fluctuation based dispersion entropy and its application to the condition monitoring of rotary machines. Mech. Syst. Signal Process. 2022, 171, 108909. [Google Scholar] [CrossRef]
  18. Huang, F.; Zhang, S.; Yang, Y.; Man, Z.; Zhang, X.; Wu, Y. Application of hyperspectral imaging for detection of defective features in nectarine fruit. Trans. Chin. Soc. Agric. Mach. 2015, 46, 252–259. [Google Scholar]
  19. Dong, L.; Zhang, W.; Xu, W. Underwater image enhancement via integrated RGB and LAB color models. Signal Process. Image Commun. 2022, 104, 116684. [Google Scholar] [CrossRef]
  20. Mh, A.; Vg, B. Statistical Measurements of Multi Modal MRI—PET Medical Image Fusion using 2D—HT in HSV color Space. Procedia Comput. Sci. 2019, 165, 209–215. [Google Scholar]
  21. Benouini, R.; Batioua, I.; Zenkouar, K.; Zahi, A.; Najah, S.; Qjidaa, H. Fractional-order Orthogonal Chebyshev Moments and Moment Invariants for image representation and pattern recognition. Pattern Recognit. 2018, 86, 332–343. [Google Scholar] [CrossRef]
  22. Anada, S.; Nomura, Y.; Hirayama, T.; Yamamoto, K. Sparse coding and dictionary learning for electron hologram denoising. Ultramicroscopy 2019, 206, 112818. [Google Scholar] [CrossRef] [PubMed]
  23. Chen, S.; Cheng, Z.; Liu, C.; Xi, F. A blind stopping condition for orthogonal matching pursuit with applications to compressive sensing radar. Signal Process. 2019, 165, 331–342. [Google Scholar] [CrossRef]
  24. Cui, L.; Sun, Y.; Zhang, J.; Wang, H. Adapted dictionary-free orthogonal matching pursuit and 0–1 programming to solve the isolation and diagnosis of bearing and gear compound faults. Measurement 2021, 178, 109331. [Google Scholar] [CrossRef]
  25. Jiang, F.; Chen, Z.; Nazir, A.; Shi, W.; Lim, W.; Liu, S.; Rho, S. Combining Fields of Experts (FoE) and K-SVD methods in pursuing natural image priors. J. Vis. Commun. Image Represent. 2021, 78, 103142. [Google Scholar] [CrossRef]
  26. Liu, C.; Liu, J. Research on power quality signals reconstruction method based on K-SVD dictionary learning. In Proceedings of the 39th Chinese Control Conference (CCC), Shenyang, China, 27–29 July 2020. [Google Scholar]
  27. Meng, Q.; Li, D.; Chen, S. Sparse representation and reconstruction of image based on K-SVD dictionary learning. In Proceedings of the International Conference on Intelligent Computing, Automation and Systems, Chongqing, China, 11–13 December 2020. [Google Scholar]
  28. Xu, J.F.; Zhang, Y.J.; Miao, D.Q. Three-way confusion matrix for classification: A measure driven view. Inf. Sci. 2020, 507, 772–794. [Google Scholar] [CrossRef]
  29. Wang, Y.; Jia, Y.; Tian, Y.; Xiao, J. Deep reinforcement learning with the confusion-matrix-based dynamic reward function for customer credit scoring. Expert Syst. Appl. 2022, 200, 117013. [Google Scholar] [CrossRef]
  30. Salla, R.; Wilhelmiina, H.; Sari, K.; Mikaela, M.; Jaakko, M. Evaluation of the confusion matrix method in the validation of an automated system for measuring feeding behaviour of cattle. Behav. Processes 2018, 148, 56–62. [Google Scholar]
  31. Trajdos, P.; Kurzynski, M. Weighting Scheme for a Pairwise Multi-label Classifier Based on the Fuzzy Confusion Matrix. Pattern Recognit. Lett. 2018, 103, 60–67. [Google Scholar] [CrossRef]
  32. Luque, A.; Carrasco, A.; Martín, A.; Ana, D. The impact of class imbalance in classification performance metrics based on the binary confusion matrix. Pattern Recognit. 2019, 91, 216–231. [Google Scholar] [CrossRef]
  33. Chen, L.C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 40, 834–848. [Google Scholar] [CrossRef]
  34. Zhou, Z.; Siddiquee, M.M.R.; Tajbakhsh, N.; Liang, J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In Proceedings of the 4th Deep Learning in Medical Image Analysis (DLMIA) Workshop, Granada, Spain, 20 September 2018. [Google Scholar]
  35. Zhang, Q.; Li, B.X. Discriminative K-SVD for dictionary learning in face recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010. [Google Scholar]
  36. Calamita, F.; Lmran, A.H.; Vescovo, L.; Mekhalfi, M.L.; Porta, N.L. Early Identification of Root Rot Disease by Using Hyperspectral Reflectance: The Case of Pathosystem Grapevine/Armillaria. Remote Sens. 2021, 13, 2436. [Google Scholar] [CrossRef]
  37. Yin, X.J.; Zhu, H.H.; Zhang, Q.; Liu, J.; Zhao, Q.Z.; Wang, C.J.; Ning, C. The disease severity estimation of bacterial spot disease of processing tomato based on hyperspectral remote sensing. In Proceedings of the International Conference on Geoinformatics IEEE, Wuhan, China, 19–21 June 2015. [Google Scholar]
  38. Lu, J.Z.; Cui, D.; Jiang, H.Y. Discrimination of Tomato Yellow Leaf Curl Disease using Hyperspectral Imaging. In Proceedings of the 2013 American Society of Agricultural and Biological Engineers, Kansas City, MO, USA, 21–24 July 2013. [Google Scholar]
  39. Wu, G.S.; Fang, Y.L.; Jiang, Q.Y.; Cui, M.; Ou, Y.M.; Diao, Z.H.; Zhang, B.H. Early identification of strawberry leaves disease utilizing hyperspectral imaging combing with spectral features, multiple vegetation indices and textural features. Comput. Electron. Agric. 2023, 204, 107553. [Google Scholar] [CrossRef]
Figure 1. Hyperspectral image acquisition system. 1. Camera. 2. Spectrograph. 3. Camera lens. 4. Stepping motor. 5. Sample. 6. Conveyor. 7. Lighting controller. 8. Computer. 9. Darkroom.
Figure 1. Hyperspectral image acquisition system. 1. Camera. 2. Spectrograph. 3. Camera lens. 4. Stepping motor. 5. Sample. 6. Conveyor. 7. Lighting controller. 8. Computer. 9. Darkroom.
Applsci 13 11904 g001
Figure 2. Spectral curve of fruit cracking samples.
Figure 2. Spectral curve of fruit cracking samples.
Applsci 13 11904 g002
Figure 3. Sample images of nectarines in near-infrared. (a) Sample images with fruit cracking. (b) Sample images with rust spot.
Figure 3. Sample images of nectarines in near-infrared. (a) Sample images with fruit cracking. (b) Sample images with rust spot.
Applsci 13 11904 g003
Figure 4. Different color space images of fruit cracking. (a) R channel of RGB color space. (b) G channel of RGB color space. (c) B channel of RGB color space. (d) H channel of HSV color space. (e) S channel of HSV color space. (f) V channel of HSV color space. (g) Y channel of YCbCr color space. (h) Cb channel of YCbCr color space. (i) Cr channel of YCbCr color space. (j) L channel of Lab color space. (k) a channel of Lab color space. (l) b channel of Lab color space.
Figure 4. Different color space images of fruit cracking. (a) R channel of RGB color space. (b) G channel of RGB color space. (c) B channel of RGB color space. (d) H channel of HSV color space. (e) S channel of HSV color space. (f) V channel of HSV color space. (g) Y channel of YCbCr color space. (h) Cb channel of YCbCr color space. (i) Cr channel of YCbCr color space. (j) L channel of Lab color space. (k) a channel of Lab color space. (l) b channel of Lab color space.
Applsci 13 11904 g004
Figure 5. Flow chart of nectarine disease identification based on sparse dictionary learning.
Figure 5. Flow chart of nectarine disease identification based on sparse dictionary learning.
Applsci 13 11904 g005
Figure 6. Classification results of nectarine disease identification using different feature vector dimensions. (a) Sample image of fruit cracking. (b) Ground truth map. (c) Classification results using nine feature vectors. (d) Classification results using six feature vectors. (e) Classification results using three feature vectors. (f) Sample image of rust spot. (g) Ground truth map. (h) Classification results using nine feature vectors. (i) Classification results using six feature vectors. (j) Classification results using three feature vectors.
Figure 6. Classification results of nectarine disease identification using different feature vector dimensions. (a) Sample image of fruit cracking. (b) Ground truth map. (c) Classification results using nine feature vectors. (d) Classification results using six feature vectors. (e) Classification results using three feature vectors. (f) Sample image of rust spot. (g) Ground truth map. (h) Classification results using nine feature vectors. (i) Classification results using six feature vectors. (j) Classification results using three feature vectors.
Applsci 13 11904 g006
Figure 7. Average overall classification accuracy of different dictionary sizes.
Figure 7. Average overall classification accuracy of different dictionary sizes.
Applsci 13 11904 g007
Table 1. Classification quality and kappa statistics.
Table 1. Classification quality and kappa statistics.
Kappa CoefficientClassification Quality
0.0–0.2Difference
0.2–0.4Commonly
0.4–0.6Good
0.6–0.8Very Good
0.8–1.0Excellent
Table 2. Confusion matrix of classification results for nine feature vectors (numerical/percentage).
Table 2. Confusion matrix of classification results for nine feature vectors (numerical/percentage).
Confusion MatrixGround Truth
DiseaseNectarineBackgroundTotal of Row
Predicted resultDisease2504/97.661884/5.59137/0.474525/103.72
Nectarine49/1.9130,692/91.07418/1.4331,159/94.41
Background11/0.431127/3.3428,714/98.1029,852/101.88
Total of column2564/100.0033,703/100.0029,269/100.10065,536/300.00
Table 3. User accuracy and producer accuracy of classification prediction results for nine feature vectors.
Table 3. User accuracy and producer accuracy of classification prediction results for nine feature vectors.
Classification ResultsUA (%)UAPA (%)PA
Disease55.342504/452597.662504/2564
Nectarine98.5030,692/31,15991.0730,692/33,703
Background96.1928,714/29,85298.1028,714/29,269
Table 4. Statistical results of different parameters.
Table 4. Statistical results of different parameters.
Parameter IndexesDisease CategoriesAverage OA (%)Average Kappa Coefficient
Feature vector dimension9 feature vectorsFruit cracking90.810.90
6 feature vectors90.920.91
3 feature vectors53.530.32
9 feature vectorsRust spot84.980.78
6 feature vectors86.860.82
3 feature vectors79.830.72
Feature points Number1500 (500 points per type)Fruit cracking90.810.90
2400 (800 points per type)90.610.87
3000 (1000 points per type)90.320.85
1500 (500 points per type)Rust spot84.980.78
2400 (800 points per type)83.780.75
3000 (1000 points per type)84.950.78
Neighborhood block size3 × 3Fruit cracking60.200.45
5 × 590.610.87
7 × 790.810.90
3 × 3Rust spot78.580.70
5 × 583.600.77
7 × 784.980.78
Table 5. Classification results of different dictionary sizes.
Table 5. Classification results of different dictionary sizes.
Disease CategoriesEvaluation IndexesDictionary Size
3004506007509001050120013501500
Fruit crackingAverage OA (%)90.9289.6489.1188.3489.3191.6792.0092.0691.65
Average kappa coefficient0.910.890.880.820.890.870.920.920.87
Rust spotAverage OA (%)86.8688.9887.1687.5287.4087.5287.3587.6286.20
Average kappa coefficient0.820.870.830.830.830.830.830.830.82
Table 6. Classification results of different methods.
Table 6. Classification results of different methods.
MethodsFruit CrackingRust Spot
Average OA (%)Average Kappa CoefficientAverage Time Cost (s)Average OA (%)Average Kappa CoefficientAverage Time Cost (s)
SVM88.810.8642.4680.450.7436.75
KNN89.280.8738.1478.560.7146.17
DeepLabV3+91.580.90126.4685.680.84185.37
Unet++91.950.91265.6586.380.85215.38
Our method92.060.9235.9388.980.8748.37
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Miao, R.; Wu, J.; Yang, H.; Huang, F. Nectarine Disease Identification Based on Color Features and Label Sparse Dictionary Learning with Hyperspectral Images. Appl. Sci. 2023, 13, 11904. https://doi.org/10.3390/app132111904

AMA Style

Miao R, Wu J, Yang H, Huang F. Nectarine Disease Identification Based on Color Features and Label Sparse Dictionary Learning with Hyperspectral Images. Applied Sciences. 2023; 13(21):11904. https://doi.org/10.3390/app132111904

Chicago/Turabian Style

Miao, Ronghui, Jinlong Wu, Hua Yang, and Fenghua Huang. 2023. "Nectarine Disease Identification Based on Color Features and Label Sparse Dictionary Learning with Hyperspectral Images" Applied Sciences 13, no. 21: 11904. https://doi.org/10.3390/app132111904

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop