Next Article in Journal
Effect of Biostimulants and Glyphosate on Morphophysiological Parameters of Zea mays (L.) Seedlings under Controlled Conditions
Previous Article in Journal
Crucial Factors Influencing the Efficiency of Androgenesis in Oat (Avena sativa L.) Through Anther and Microspore Cultures
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Agronomy
Volume 14
Issue 10
10.3390/agronomy14102395
agronomy-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

From Outside to Inside: The Subtle Probing of Globular Fruits and Solanaceous Vegetables Using Machine Vision and Near-Infrared Methods

by
Junhua Lu
1,†,
Mei Zhang
2,†,
Yongsong Hu
3,
Wei Ma
3,*,
Zhiwei Tian
3,
Hongsen Liao
1,
Jiawei Chen
1 and
Yuxin Yang
1
1
School of Mechanical Engineering, Xihua University, Chengdu 611743, China
2
School of Mechatronics, Chengdu Agricultural University, Chengdu 611130, China
3
Institute of Urban Agriculture, Chinese Academy of Agricultural Sciences, Chengdu 610213, China
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Agronomy 2024, 14(10), 2395; https://doi.org/10.3390/agronomy14102395
Submission received: 26 September 2024 / Revised: 12 October 2024 / Accepted: 15 October 2024 / Published: 16 October 2024
(This article belongs to the Special Issue Intelligent Detection and Classification of External Traits in Crop Plants, Fruits, and Vegetables)
Browse Figures
Versions Notes

Abstract

:
Machine vision and near-infrared light technology are widely used in fruits and vegetable grading, as an important means of agricultural non-destructive testing. The characteristics of fruits and vegetables can easily be automatically distinguished by these two technologies, such as appearance, shape, color and texture. Nondestructive testing is reasonably used for image processing and pattern recognition, and can meet the identification and grading of single features and fusion features in production. Through the summary and analysis of the fruits and vegetable grading technology in the past five years, the results show that the accuracy of machine vision for fruits and vegetable size grading is 70–99.8%, the accuracy of external defect grading is 88–95%, and the accuracy of NIR and hyperspectral internal detection grading is 80.56–100%. Comprehensive research on multi-feature fusion technology in the future can provide comprehensive guidance for the construction of automatic integrated grading of fruits and vegetables, which is the main research direction of fruits and vegetable grading in the future.

1. Introduction

According to the statistics of the Forward-looking Industry Research Institute, the global fruit planting area used to be 64.86 million hectares, and the output used to be 88.7 million tons in 2020. China, India and the United States were among the top fruit producers in the world [1,2]. The global vegetable output used to be 1.15 billion tons in 2020. China, India and Brazil were among the top vegetable producers in the world. With the development of global agriculture and information technology [3] and the rise of related industries, it is very important to test and grade the quality of fruit and vegetables. The augmented production of fruit [4] and vegetables [5] has resulted in a progressive increase in the demand for labor worldwide. However, owing to the scarcity of the labor force globally, the traditional fruit and vegetable grading is readily influenced by human psychological factors [6], characterized by high intensity [7,8], low efficiency [9] and low precision. Hence, the application of novel detection and grading technology is urgently required [10]. The research of machine vision technology, near-infrared spectroscopy technology and hyperspectral technology has effectively solved the above problems of traditional grading.
Nowadays, machine vision technology, near-infrared light technology and hyperspectral technology are rapidly maturing and being applied to fruit and vegetables grading. Machine vision can improve the efficiency of the fruit and vegetables sorting process, reduce labor costs and pave the way for automation in agriculture by introducing faster, cost-effective and non-destructive methods. By analyzing the spectral characteristics of fruits and vegetables, near-infrared light technology and hyperspectral technology can quickly detect the maturity [11], quality and intrinsic attributes of fruits and vegetables such as moisture, sugar and nutrition. Varghese, Renju Rachel et al. [12] proposed a real-time fruit and vegetables grading system based on machine vision for the shelf life of fruits and vegetables to help users choose ideal vegetables and fruits for consumption. Tang Yunchao et al. [13] used machine vision technology to identify and locate fruits in complex environments, and reviewed two main methods for fruit recognition and location, including digital image processing technology and algorithms based on deep learning. Ismail, Nazrul et al. [14] used a variety of models to grade apples and bananas, and found that the Efficient Net model had the highest accuracy of apple and banana grading, 96.7% and 93.8%, which made a contribution to the future automatic-grading accuracy. Fan Shuxiang et al. [15] independently developed a rapid-response portable Vis/NIR device prototype. The experimental results showed that the ratio of the predicted determination coefficient (R-p (2)), the predicted root mean square error (RMSEP) and the standard deviation of the reference destructive SSC to RMSEP (RPD) was 0.690, 0.604% and 1.794, respectively, proof that the device can detect Apple internal features. Yuan Yuhui et al. [16] used three object detection algorithms, Faster R-CNN, Yolov3-Tiny and Yolov5s, to extract the bruised area of apples for damage detection to grade the bruises of apples. The results showed that the accuracy of the three algorithms for early bruised and non-bruised apples was more than 99%. The accuracy was above 96% for apples with no bruising, mild bruising, and severe bruising, and the shortest detection speed for a single image was 6.8 ms. When Unal, Zeynep. et al. [17] used NIR data for training, AlexNet, InceptionV3, and VGG16 models had high accuracy in bruise detection, with 99.33%,100%, and 100%.
In view of this, in order to achieve accurate grading of the exterior and interior of fruits and vegetables, researchers have carried out extensive research in machine vision technology, near-infrared light technology, and hyperspectral technology. In this paper, the advantages and disadvantages of different experiments are summarized and analyzed. This paper reviewed the research progress of the three technologies, discussed how to effectively improve the grading efficiency, and analyzed the limitations and challenges of the existing technologies, as well as the development trend of fruits and vegetable grading. It is pointed out that automatic grading in a multi-source environment, improving the recognition accuracy of fruits and vegetables, and automatic integrated grading are important directions for future research.

2. External Inspection of Fruits and Vegetables

Machine vision is one kind of comprehensive technology that emulates the human eye using computers and can identify specific images through training. It mainly includes five steps: image acquisition, image preprocessing, feature extraction, size grading and hierarchical output. It is widely applied in various fields of social production. Due to its advantages of high precision, fast speed, and user-friendly operation, products from international industry leaders such as Cognex (Boston, MA, USA), Basler (Arnsberg, Germany), Keyence (Japanese large version), and Omron (Kyoto, Japan) have found extensive use in external inspection and grading of fruit and vegetable products [18]. Autoline’s fruit grading equipment from the United States holds a leading position globally [8]. The process of fruit and vegetable grading based on machine vision technology mainly includes the following core steps: image acquisition, image preprocessing, feature extraction, size grading, and graded output. This section presents a comprehensive study on the application of machine vision in fruits and vegetables grading, as shown in Table 1, and elaborates the methods, conclusions, advantages and disadvantages of each study. A detailed review is made for the follow-up research.

2.1. Fruit and Vegetable External-Size Detection

In terms of citrus color and grade detection [33,34], Zou Wei [19] transformed RGB images into HSV images and processed them with image noise reduction and binarization. The Canny algorithm was used for edge detection and the Find Contours function was used to identify the contours of citrus fruits. The area of the contour is calculated and graded by the counter area function (Figure 1). Experimental results show that the proposed method can accurately distinguish citrus fruits of different sizes and qualities. It has made a contribution to the target area recognition, but this method is only suitable for static, not for dynamic situations. Li Lang et al. [20] developed an online citrus detection system using machine vision. (Figure 2) In this system, each frame image is segmented, sorted, denoised, and two-dimensional coloring ratio is extracted to reduce the influence of coloring rate in the image. Experimental results show that the maximum error tolerance is 5%, and the overall grading accuracy is 90.54%. Compared with Zou Wei [19], the accuracy of this experiment is higher, and it is suitable for grading in dynamic environments.
For the apple image recognition and grading process (Figure 3), Shi Ruiyao [21] used the Support Vector Machine (SVM) to segment and fill the apple image, calculated the threshold according to the OTSU algorithm, and reconstructed the image (Figure 4). In the experiment, the penalty factor C = 0.01 and the number of iterations 1000 have the best effect, reaching 95% accuracy. The experiment carried out a comprehensive detection, and can identify the defects by extracting the image, but there will be errors in the stem or calyx of the apple, which will be improved in the future. Peng Yankun et al. [22] used machine vision and a robotic arm to detect and grade apples [35,36]; the grading accuracy reached 95%, and the grading time was about 5.2 s. Using the PP-YOLO target-detection algorithm and the control software developed in Python to train and test images has high reliability and practicability. However, the grading speed will be affected by the manipulator. Liu Jiahao et al. [23] used the improved algorithm to fit and extract the edge image of apple, and then converted it into HIS Canny edge-detection image to calculate its roundness. Compared with the traditional manual identification of the apple diameter, the error is within ±1.5 mm, which meets the actual grading requirements. This method effectively solves the problem of high false detection rate and low detection efficiency of spherical fruit.
Rao Jian et al. [24] employed an online detection system for the identification and grading of spherical fruits (Figure 5). The proposed method utilizes the minimum external matrix to determine the fruit shape index, while also analyzing fruit surface defects through morphological region-filling techniques. In this experimental study, a total of 120 test sets were utilized for grading detection, resulting in the detection of 18 special-grade fruits, 39 first-grade fruits, 39 s-grade fruits, and 24 externally identical fruits. The experimental results demonstrate an average recognition rate of 94.4%. This research possesses the advantages of extracting surface-defect features accurately and achieving high grading accuracy. However, it has limitations in terms of not being able to detect the entire fruit surface and potential impact from fruit surface-defect characteristics on the detection results.
Zhang Yuhua et al. [25] employed multi-sensor information fusion technology for the detection of various fruits. The findings demonstrate that fruit grading should be approached from multiple perspectives, rather than a singular standpoint. This study enables accurate fruit grading, although it is limited by single-point detection and predominantly static images collected in the university laboratory, which do not fully capture the dynamic nature of real-world scenarios. Su Boni et al. [26] identified and graded strawberries [37,38]. The images are collected by SLR camera, and the images are processed by median filter for denoising, gray enhancement and binarization. The experimental results demonstrate that a white background enables clear segmentation of strawberry fruit, while the median-filtering algorithm effectively removes salt-and-pepper noise present in the image collection process. The five typical algorithms successfully segment the image contour. This experiment offers online, lossless, and real-time advantages; however, it does not consider factors such as strawberry rot or breakage, nor does it address complex background scenarios when completing strawberry fruit segmentation and maturity recognition. The machine vision technology was employed by Li Jianhang et al. [27] to accurately identify and classify blueberries (Figure 6). The image segmentation was performed using the maximum between-cluster variance method, followed by morphology-based removal of connected regions and fitting using the least squares method [39]. Experimental results demonstrate that the average accuracy for calculating blueberry fruit area is 98.93%, while the average accuracy for calculating blueberry fruit circumference is 87.74%. Comparing to manual segmentation as the standard calculation accuracy, area grading proves to be more accurate than perimeter grading. Blueberry fruits are classified based on their area and perimeter calculations, offering a low-complexity and cost-effective approach. However, it should be noted that accurate segmentation of fruit stems remains challenging, particularly in cases where images are unimodal.
The potato peel images (Figure 7) were preprocessed using MATLAB R2016a in Tang Zhensan’s [28] study on potato grading technology [40]. Additionally, support vector machine and BP neural network models were developed to effectively grade and classify the roughness of the potato peel. In this particular experiment, a total of 79 potato samples were employed, with 55 samples allocated as the training set and 24 samples designated as the test set. The experimental findings demonstrate that SVM achieves an average accuracy rate of 87.5%, surpassing that achieved by the BP neural network in terms of potato peel grading. Consequently, a highly effective grading model was established utilizing support vector machines alongside feature comparison using GLCM, enabling recognition and grading of potato peels. However, it is worth noting that one limitation is the relatively lower recognition accuracy.
Zhang Yanjun [29] optimized the yolov6 neural network to accurately predict the Huanghua joint node, and added the heat map branch to enhance the non-maximum suppression algorithm. The experimental results show that the recognition accuracy is 91.02%, and the positioning accuracy is 99.8%. The Agaric recognition grading system (Figure 8) was developed by Wang Mengxin et al. [30], using image processing techniques of the DMV-VGR software and PLC data monitoring. The system is able to perform initial grading and provide efficient and accurate grading solutions. The disadvantages are low grading accuracy and lack of omnidirectional features.
Bhargava, A. et al. [31] used Gaussian filtering for denoising, fuzzy C-means clustering and grab-cut for image segmentation in their experiments on detection and grading of various vegetables and fruits, respectively. Subsequently, they compared the accuracy with SVM and four other decision methods. The experiments were conducted on five vegetables and four fruits. Experimental results show that the accuracy of SVM decision detection reaches 97.63%, and the accuracy of grading reaches 96.59%. The SVM with the highest accuracy is selected for validation. Renju Rachel Varghese et al. [12] used SVM, K-NN and the Anna algorithm to design a system that can identify and grade a variety of fruits and vegetables without a network, and the overall accuracy of the experiment reached 70%. Anuja Bhargava and Atul Bansal [41] developed a fruit grading system using K-NN, SVM, SRC, ANN, and other methods. The system shows superior accuracy compared to existing techniques, being able to analyze and identify fruits based on color, geometry, statistics, and texture.

2.2. External-Defect Detection of Fruits and Vegetables

A set of image acquisition devices was developed by Chen Yingxue et al. [32] (Figure 9) for lossless feature recognition and grading, using mangoes. In the experiment, Sobel operator and Canny operator are used to detect the edge, which solves the problem of unclear and discontinuous edge in the previous defect recognition. The results indicate that the accuracy of first-grade fruits is 93.3%, while second-grade fruits exhibit an accuracy of 95%, and third-grade fruits demonstrate a similar accuracy level of 95%. This research successfully addresses issues related to unclear and discontinuous edges, while also mitigating noise interference in the images captured. However, it should be noted that variations in illumination during experimentation can impact image quality.
Due to the good grading performance of the VGG network, Zhou HaiYan [42] used the Random Weighted Average (SWA) optimizer and w-softmax loss function to improve the VGG network to identify Qingmei defects. The accuracy of defect grading and recognition is increased by 9.8% and 16.6%, and the test speed is increased by 1.87 ms and 6.21 ms, respectively. This experiment can well identify the external defects of green plum, but cannot identify the stem of green plum and the whole green plum. This experiment is not suitable for dynamic identification. For dynamic defect identification, Chen Yaohui [43] developed the SORT algorithm to track and identify citrus defects, as shown in Figure 10, which made up for the shortcomings of static identification. The overall recall, precision, and F1 score are 0.87, 0.88, and 0.871, respectively. The accuracy of defect-grading judgment in dynamic recognition is low, and high accuracy is needed to locate the position of the orange.
Non-destructive identification of defects in fruits and vegetables is still a difficult problem in the field. Zhang Xinxing [44] uses the optimized YOLO-V4 model for detection and the Efficient Net model for grading. The accuracy and F1 score are 0.890 and 0.872, respectively. The YOLO-V4 model before and after optimization identifies defects well, and after optimization it can be classified more accurately (Figure 11), but it will lose confidence, location, and grading. Improving the model accuracy and simplifying the model are the main research directions in the future.
The YOLOv5 network uniformly changes the size of the input image to 640 × 640, which will improve the speed of inference and detection. The YOLOv5 model was optimized by Hu Wenxin [45], and the training comparison with the YOLOv5x model is shown in Figure 3. The average precision, precision and recall reach 95.5%, 94.0% and 95.1%, respectively, which are 5.8%, 3.6% and 7.6% higher. At the same time, the image detection speed is increased by 22.1 ms. Experiments show that the improved network can achieve good performance in citrus skin defect detection, but there is a problem, in that the defect data set is difficult to collect.
We examined recent studies on fruit defect detection, which are organized into tables as shown in Table 2. In recent years, most works on fruit defect detection are based on convolutional neural networks. The datasets used in the literature mainly consist of an image with a single object in a laboratory environment as the dataset and a dataset with multiple objects in an orchard environment as the background. However, the datasets used in the above studies are different, so it is not possible to directly compare the performance of the detection networks used. For fair evaluation, the image content and background, network and detection performance of the dataset were selected for comparative study. As can be seen from Table 2, the detection effect of the optimized detection network is better than that of the original network, and the detection difficulty of the data set collected in the orchard environment is usually greater than that in the laboratory environment. This indicates that it is effective and necessary to improve the network when the image contains multiple target objects and a complex background.
The accuracy of machine vision in grading the external dimensions and defects of fruits and vegetables is 70–99.8% and 88–95%, and it has good grading ability under specific lighting conditions. However, there are still some shortcomings, such as high hardware and lighting requirements, a relatively single environment, complex image processing, and being unable to detect multiple defects at the same time. In the future, deep-learning and computer-vision technologies should be combined to develop more efficient image processing algorithms to improve the recognition ability of surface defects in fruits and vegetables. Using the AI Vision cloud platform improves the simultaneous detection of multiple defects through online annotation and learning, reducing the complexity and cost of custom development.

3. Internal Inspection of Fruits and Vegetables

The near-infrared spectrum refers to the range of wavelengths between the red edge of the visible spectrum and the infrared spectrum, spanning from 780 to 2526 nm [46]. Detection technology utilizes electromagnetic waves within the wavelength range of 700 to 1100 nm [47]. In recent years, the Perkin Elmer factory and Thermos factory in the United States, and the Bruker company in Germany have ventured into research and development of near-infrared spectrometers. Infrared spectroscopy is widely used in many fields, such as food detection, chemical analysis, and biological research. For fruits and vegetables specifically, near-infrared light technology enables non-destructive detection of various indicators such as sugar content and moisture levels. Its advantages can be harnessed for damage-free fruits and vegetable grading, with significant implications. The United States conducted the initial study on imaging spectrometers in 1983. AVIRIS and DAIS in the United States, FLI and CASI in Canada, ROSIS in Germany and HyMap in Australia have successively studied fruit and vegetable grading in the hyperspectral field. The “atlas and spectrum integration” technology of hyperspectral analysis utilizes a wide range of continuous-wavelength spectral data to extract object characteristics. This technology captures information within the visible-light-to-infrared spectrum, enabling change detection and target tracking [48]. It provides more detailed and comprehensive data compared to human eyes or conventional photography. Hyperspectral “map and spectrum integration” technology has found extensive applications in diverse fields such as agriculture, geological exploration, environmental monitoring, and medical diagnosis. Table 3 shows the application of machine vision and NIR spectroscopy and hyperspectral technology in fruit and vegetable grading in recent years.
The main process of fruits and vegetables grading by near infrared spectroscopy technology includes the selection of a calibration sample set and prediction sample set, spectrum collection, chemical value detection and, finally, the establishment and testing of the mathematical model. The process of fruit and vegetable grading based on hyperspectral technology mainly includes the following core steps: data acquisition and preprocessing, data dimensionality reduction, model construction and grading. The application of NIR spectroscopy and hyperspectral technology in fruit and vegetable grading is reviewed in detail below.
Liao Zhiqiang [49] used six preprocessing methods (Figure 12) for internal detection and grading of apples, and used the ridge model for preprocessing; the accuracy of grading ranged from 88.38% to 90.84%. Experimental results show that the ridge regression model has better stability and higher grading accuracy. Peng Yankun et al. [22] developed a robot hand system with lossless perception and grading (Figure 13). The PP-YOLO target-detection algorithm is used to improve the detection speed, the normalized spectral ratio method is used for brix modeling, and NSR + CARS is used as the robot-hand spectrum model. The accuracy of the method is 0.9655, the recall rate is 1, the accuracy rate is 0.998 4, and the model detection speed is 38 frames/s, which further improves the grading accuracy. It has high reliability. However, there is a lack of sufficient internal indicators, and the dynamic acquisition of the spectrum may have an impact on the model. The application of hyperspectral imaging technology solves the above problems. Wang Jifang [50] used hyperspectral imaging and the BP neural network to detect three indicators of apple sugar content, pH value and hardness. The R values of the experiment are 0.85, 0.46, and 0.36, and the RMSE values are 0.69, 0.76, and 0.86, respectively, which has a strong predictive ability and is suitable for the grading of apples according to their sugar content. It should be noted that the accuracy of using single-channel spectra needs to be further optimized. For the application of multichannel spectroscopy, Huang Yuping et al. [51] used multi-channel hyperspectral imaging technology to detect and classify apples, and the use of spectral combination improved the recognition rate of the SR spectrum, reaching a grading accuracy of 0.994. However, the spectral combination cannot improve the results of the best single SR spectra in the visible region, which is a difficult point to be solved in the future. Çetin, N. et al. [62] predicted apple hardness and soluble solid content and pointed out that the potential use of ANN and DT methods for hyperspectral imaging was more effective for hardness, while DT and MLR were more effective for SSC. These methods are very feasible for industrial applications.
Wang Yali et al. [52] designed a set of maize vitality detection and grading devices (Figure 14), and the partial least squares method can be used to better analyze the vitality. Experimental results show that SG convolutional smoothing shows superior performance, the calibration accuracy reaches 98.7%, the prediction set accuracy reaches 96%, and the overall vitality prediction accuracy reaches 97%. This experiment effectively solves the problems of low efficiency, low modeling accuracy and instability associated with single granulation. In the actual agricultural application, there are some problems such as the mismatch between the sliding speed and the identification rate, and the changeable environment. In research by Chia et al. [53], a low-cost artificial neural network based on k-fold cross validation was developed to classify the sugar content of white radish. Experimental results show that the accuracy of this method is 80.56%. Since the internal mass variation is small, pre-dispersive reflectance spectrum acquisition is used to improve the accuracy of the method. However, the grading accuracy of this method for white radish is relatively low, and it is more susceptible to other factors, such as illumination, in the actual grading process.
In terms of hyperspectral imaging technology, Li Bin et al. [54] used random forest, least squares support vector machine, and other four methods to establish the spectral characteristics of loquat to classify loquat contusions (Figure 15). The hybrid image model has superior prediction performance, and the least squares support vector machine method can achieve 100% prediction accuracy. It provides a high precision scheme for the future damage identification. The fruit stem and pulp are relatively difficult to distinguish in grading. Lu Wei et al. [55] used hyperspectral ensemble learning (Figure 16) to grade anthocyanin content in black Lycium barbarum (Figure 17). Stacking ensemble learning is used to improve the grading accuracy from 0.9417 to 0.9833. The spectral extraction and grading of fruit stems and pulp are carried out, which verifies the feasibility of hyperspectral ensemble training for rapid grading of Lycium ruthenicum.
The degree of granulation of fruit affects its taste. For fruits with thick skin, it can be challenging to predict internal fruit mass using imaging techniques [63]. Ref. [64] developed a method to estimate the ripening time of avocado using hyperspectral imaging combined with deep learning, with an average error of 1.17 days per fruit in the test set. When Liu Yande et al. [56] graded the granulation degree (Figure 18), the UVE-LS-SVM model based on RBF-Kernel was used for detection, and the misjudgment rate was only 0.78%. The established model effectively reduces the false positive rate and spectral space dimension, which provides a solid foundation for navel orange detection and grading. The model is only suitable for the granulation grading of thin-skinned fruits. When grading thick-skinned fruits, Sun Xiaopeng et al. [57] used the continuous projection-k-nearest neighbor algorithm to predict the accuracy, sensitivity and specificity of the model to reach more than 0.97, 0.9231 and 0.9784, respectively (Figure 19). It is suitable for grades 1–3, and the recognition accuracy of grades 4 and 5 is low. Miguel Vega-Castellote et al. [58] utilized near-infrared spectroscopy technology to detect the maturity and soluble solid content (SSC) of watermelon based on its penetration characteristics. In this experiment, NIR spectral data were obtained from two squares of one watermelon, while spectra were acquired from five different measurement points of 19 other watermelons. The results revealed that the penetration depth for both intact watermelon and pulp at four measurement points was 11 mm. Moreover, the optimal performance in terms of SSC percentage exhibited R2cv = 0.73, RMSECV = 0.39%, R2p = 0.81, and RMSEP = 0.30%. This study holds potential for watermelon grading; however, it is worth noting that the accuracy of the model heavily relies on optical geometry measurements, necessitating further optimization of the instrument. Sharma, S. et al. [65] used the GA-PLSR model and SPA-PLSR model to measure DM, TSS and FC of durian pulp, providing a quality inspection and grading system for durian packaging companies.
Lin Zhang et al. [59] used VISSA-GWO-SVM model to detect and classify saccharin jujube, and achieved a grading accuracy of 91.67%. Three different algorithms are used to tune the Support Vector Machine (SVM). The experimental method effectively overcomes the limitations of traditional manual detection methods, and has the advantages of being fast, accurate and lossless. It has made a contribution to food safety. Ci Jiangtao et al. [60] used hyperspectral technology [61], the CARS-MLR model, to detect and classify the external features of eggplant, and the accuracy of the prediction set reached 96.82%. The grading accuracy is improved, and the eggplant damage can be effectively graded and evaluated. However, the CARS-MLR model has strong dependence on data, and feature selection will affect its stability. The content of potato can be accurately determined using hyperspectral technology, as demonstrated by Jing Zhang et al. [66]. They developed the OSC-CARS–PLSR model based on partial least squares regression and orthogonal signal correction techniques. The obtained R2 values were 0.9606 and 0.8925, with corresponding RMSE values of 0.070% and 0.1385%. The OSC-CARS-PLSR model improves the prediction accuracy and stability, but it increases the computational complexity and needs to use more computing resources. Qi Hengnian et al. [67] found in the non-destructive determination of soluble-solid content of crown pear that the combination of Vis/NIR spectroscopy and the MLP-CNN-TCN method can quickly and non-destructively detect the SSC of crown pear, providing a new regression option for predicting the SSC of fruit.

4. Challenges and Trends

In the past few years, significant advances have been made in optical technology, including complex areas such as spectral imaging, near-infrared light technology, and computer vision [68]. However, there are still many challenges in the application of this technology. These challenges mainly include how to perform efficient and accurate fruit and vegetable recognition and grading in complex environments. Realizing multi-source, efficient and accurate identification and grading of fruits and vegetables is of great significance to improve the current agricultural labor shortage faced by the world, and to realize the unmanned and integrated development of planting and picking of fruits and vegetables. Therefore, future research needs to work on the following aspects:
(1) High yield. The increase in the world population will lead to an increase in the demand for fruits and vegetables, and the global production of fruits and vegetables will further increase. Fruit and vegetable grading is particularly important. Therefore, the accurate grading of fruits and vegetables under complex conditions is still a focus. At present, the research on fruit and vegetable grading requires less quantity; in the case of high yield and long duration, it is a major direction and challenge in the future to grade fruits and vegetables in real time and without error.
(2) Structure of orchard. An unstructured environment usually has the characteristics of irregular terrain, many dynamic obstacles and complex information collection, which adversely affect the fruit and vegetable grading in an unstructured environment. Therefore, the challenge of fruit and vegetable grading in the future integration application stems from the complexity of unstructured conditions, and the operation of the fruit and vegetable grading system and robot in a complex environment is still the focus of research. In the future, the construction of a structured environment is the orchard, and the development of the greenhouse is more conducive to the integrated management of fruits and vegetables, which is also a major direction of future development.
(3) Grading technique. It is still difficult to classify the size of fruits and vegetables by machine vision, taking into account factors such as high cost, complex technology, strong dependence on the environment, and difficulty in optimization. Near-infrared spectroscopy (NIRS) has some technical problems, such as difficult modeling, low sensitivity and poor reliability. There are problems such as high cost of hyperspectral technology and complex data processing of equipment structure. In the future, the theory and practice of machine vision technology need to be further refined to improve the accuracy of recognition and grading. Machine vision technology is applied to the whole growth process of crops to give full play to its non-destructive testing ability to assess the appearance quality of crops. NIR technology will enable miniaturization and lightweight design to be tested directly through mobile phone software or small devices. This conversion will result in a versatile and portable lossless internal-test instrument with multiple metrics that can be used at any given time. One direction in the future is to improve the processing and analysis capabilities of hyperspectral images, which will show great potential in fruit and vegetable grading and detection. With the continuous development of hyperspectral technology, sensors and devices will further improve their performance, thus promoting the role of hyperspectral technology in various fields.

5. Conclusions

In order to solve the problems existing in fruit and vegetable grading, the research status of fruit and vegetable grading was reviewed. At present, in the field of unmanned grading, the research on automatic grading systems and machines is not sufficient. However, current scoring techniques are rapidly evolving, providing a wide range of algorithms and techniques that can be used to enhance the capabilities of automated scoring systems and machines. This is the main contribution to fully automatic grading of fruits and vegetables.
This paper reviews the application of machine vision technology, near-infrared light technology and hyperspectral technology in fruit and vegetable grading in the past five years, and expounds the advantages and disadvantages of found in each research study. The results show that the accuracy of machine vision technology in fruit and vegetable size grading is 70% to 99.8%. The accuracy with respect to external defects is 88–95%. The accuracy of near-infrared light technology and hyperspectral technology in the detection and grading of internal content of fruits and vegetables is 80.56% to 100%, which is a high accuracy.
The machine vision technology, near-infrared light technology and hyperspectral technology are applied to the grading of fruits and vegetables, which realizes accurate and efficient sorting, and solves the problem of inconsistent quality. Further research on hyperspectral-based grading devices can facilitate online detection and grading of fruits and vegetables, compensating for the limitations of manual inspection and grading processes. Ultimately, this will lead to automated, unmanned, and mechanized operations. In the future, extensive efforts should be dedicated to studying non-destructive testing methods and theories for fruits and vegetables, as well as designing more advanced algorithms and software solutions. Integration of various beneficial advanced technologies is crucial in developing grading equipment suitable for diverse environmental conditions. This will ensure automation, safety, environmental friendliness, and advanced portability capabilities, among others advantages, while enhancing the quality, not only within fruit and vegetable grading, but also across other industries. Reliable support, along with technological advancements, are essential for fostering a healthy development within the fruit and vegetable industry chain.

Author Contributions

Conceptualization: W.M. and M.Z.; writing—original draft preparation: J.L. and M.Z.; article ideas: W.M. and Y.H.; methodology: J.L. and Y.Y. and Z.T.; article search: J.L. and M.Z.; article collation: J.C. and H.L.; visualization: J.L.; supervision: W.M. and Y.H.; project administration: W.M. All authors have read and agreed to the published version of the manuscript.

Funding

The work was funded by a number of projects, including the Chengdu Agricultural Science and Technology Center Project (NASC2022KR08) and (NASC2021KR07), and the Chengdu Agricultural College Youth Project (22ZR112).

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Tan, J.J.; Yang, S.D.; Zhou, L.Q.; Tan, H.W. High-quality citrus industry development strategies of Guangxi based on China-ASEAN fruit trade. Nanning J. South. Agric. 2020, 51, 470–476. [Google Scholar]
  2. Weng, M. The status quo and characteristics of China’s import and export of agricultural products. Int. Trade 2013, 6, 30–35. [Google Scholar]
  3. Tian, Y.; Yang, G.; Wang, Z.; Wang, H.; Li, E.; Liang, Z. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agric. 2019, 157, 417–426. [Google Scholar] [CrossRef]
  4. Wu, Z.Y.; Li, Y.R.; Dong, Z.D. Research on the Development Status and Countermeasure of Chinese fruit market. Chin. Fruits Veg. 2023, 43, 79–83+87. [Google Scholar]
  5. Wang, H.; He, J.; Aziz, N.; Wang, Y. Spatial Distribution and Driving Forces of the Vegetable Industry in China. Land 2022, 11, 981. [Google Scholar] [CrossRef]
  6. Wu, L.; Zhou, Y.Q. Research on imperfect grain detection technology of wheat based on improved YOLOX. J. Henan Univ. Technol. Nat. Sci. Ed. 2023, 44, 109–117. [Google Scholar]
  7. Gill, H.S.; Murugesan, G.; Mehbodniya, A.; Sajja, G.S.; Gupta, G.; Bhatt, A. Fruit type grading using deep learning and feature fusion. Comput. Electron. Agric. 2023, 211, 107990. [Google Scholar] [CrossRef]
  8. Bai, Z.W.; Yan, F.W.; Yuan, P.H.; Zhang, H.M.; Yang, R.S. Design and experiment of fruit sorting robot based on embedded machine vision. J. Intell. Agric. Mech. 2023, 4, 61–70. [Google Scholar]
  9. Qin, Y.C.; Li, T.; Li, X.; Wang, M.L.; Tan, Z.Y. Design of apple damage automatic detection system based on machine vision. Food Mach. 2024, 40, 138–142. [Google Scholar]
  10. Wen, J.; He, J. Agricultural development driven by the digital economy: Improved EfficientNet vegetable quality grading. Front. Sustain. Food Syst. 2024, 8, 1310042. [Google Scholar] [CrossRef]
  11. Liu, C.H.; Li, X.; Zhu, Z.J.; Bao, Z.J.; Xie, X.N.; Yang, Z.Z.; Wang, Y.; Liu, F. Progress of Non-destructive Testing Technology in Mango Quality. Sci. Technol. Food Ind. 2021, 42, 413–422. [Google Scholar] [CrossRef]
  12. Varghese, R.R.; Jacob, P.M.; Sooraj, S.; Ranjan, D.M.; Varughese, J.C.; Raju, H. Detection and Grading of Multiple Fruits and Vegetables Using Machine Vision. In Proceedings of the 2021 8th International Conference on Smart Computing and Communications (ICSCC), Kochi, Kerala, India, 1–3 July 2021; pp. 85–89. [Google Scholar]
  13. Tang, Y.; Chen, M.; Wang, C.; Luo, L.; Li, J.; Lian, G.; Zou, X. Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review. Front. Plant Sci. 2020, 11, 510. [Google Scholar] [CrossRef] [PubMed]
  14. Ismail, N.; Malik, O.A. Real-time visual inspection system for grading fruits using computer vision and deep learning techniques. Inf. Process. Agric. 2022, 9, 24–37. [Google Scholar] [CrossRef]
  15. Fan, S.X.; Wang, Q.Y.; Tian, X.; Yang, G.Y.; Xia, Y.; Li, J.B.; Huang, W.Q. Non-destructive evaluation of soluble solids content of apples using a developed portable Vis/NIR device. Biosyst. Eng. 2020, 193, 138–148. [Google Scholar] [CrossRef]
  16. Yuan, Y.H.; Yang, Z.R.; Liu, H.B.; Wang, H.B.; Li, J.H.; Zhao, L.L. Detection of early bruise in apple using near-infrared camera imaging technology combined with deep learning. Infrared Phys. Technol. 2022, 127, 104442. [Google Scholar] [CrossRef]
  17. Ünal, Z.; Kizildeniz, T.; Ozden, M.; Aktas, H.; Karagoz, O. Detection of bruises on red ap-ples using deep learning models. Sci. Hortic. 2024, 329, 113021. [Google Scholar] [CrossRef]
  18. Chen, B.Q.; Wu, Z.H.; Li, H.Y.; Wang, J. Research of machine vision technology in agricultural application: Today and the future. Sci. Technol. Rev. 2018, 36, 54–65. [Google Scholar]
  19. Zou, W. Research on citrus fruit size grading based on machine vision technology. Agric. Technol. 2023, 43, 48–51. [Google Scholar]
  20. Li, L.; Wen, T.; Dai, X.Y.; Wang, Z.Y. Online detection and grading system of citrus whole surface color. Food Mach. 2022, 38, 121–126. [Google Scholar]
  21. Shi, R.Y.; Tian, Y.W.; Lai, X.T.; Gu, W.J. Development of apple intelligent on-line inspection and grading system based on machine vision. China Agric. Sci. Technol. Rev. 2018, 20, 80–86. [Google Scholar]
  22. Peng, Y.K.; Sun, C.; Zhao, M. Dynamic non-destructive perception and grading robotic hand system for apple quality. Trans. CSAE 2022, 38, 293–303. [Google Scholar]
  23. Liu, J.H.; Gao, J.W.; Zhang, B.X.; Wang, J.C. Design of fruit grading system based on machine vision. Food Mach. 2023, 39, 112–118. [Google Scholar]
  24. Rao, J.; Lu, Z.Y. Research on external quality grading method of spherical fruit based on machine vision. Technol. Innov. 2022, 63–65. [Google Scholar]
  25. Zhang, Y.H.; Meng, Y.; Zhang, M.G.; Jiang, P.H.; Chen, D.J.; Guo, F.J. Fruit comprehensive quality detection based on Near infrared, machine vision and Information Fusion. Food Ind. 2018, 39, 247–250. [Google Scholar]
  26. Su, B.N.; Hua, X.Y. Research on strawberry image processing based on machine vision. J. Cap. Norm. Univ. (Nat. Sci. Ed.) 2018, 39, 42–45. [Google Scholar]
  27. Li, J.H.; Yi, K.C.; Liu, L.; Zhang, X.W.; Sun, Y.R. Research on blueberry fruit grading based on machine vision. Wuhan J. Jianghan Univ. 2021, 49, 64–69. [Google Scholar]
  28. Tang, Z.S.; Yuan, J.L.; Kang, L.H.; Cheng, L.X.; Lu, T.; Yang, C.; Zhang, F. Potato tuber skin roughness grading analysis based on image characteristics recognition. Sci. Agric. Sin. 2023, 56, 4428–4440. [Google Scholar]
  29. Zhang, Y.J.; Zhao, J.X. Research on daylily joint detection algorithm based on multiple neural networks. J. Chin. Agric. Mech. 2024, 45, 228–234. [Google Scholar]
  30. Wang, M.X.; Meng, Q.X.; Wang, M.J. Design of grading Control system of black Fungus Based on machine Vision. China Sci. Technol. Inf. 2023, 21, 91–94. [Google Scholar]
  31. Bhargava, A.; Bansal, A.; Goyal, V. Machine Learning-Based Detection and Sorting of Multiple Vegetables and Fruits. Food Anal. Methods 2022, 15, 228–242. [Google Scholar] [CrossRef]
  32. Chen, Y.X.; Yang, Y. Research on appearance quality of mango based on machine vision. Electron. Manuf. 2023, 31, 68–71. [Google Scholar]
  33. Wu, Y.L.; Chen, Y.Y.; Lian, X.Q.; Liao, Y.; Gao, C.; Guan, H.N.; Yu, C.C. Study on the Identification Method of Citrus Leaves Based on Hyperspectral Imaging Technique. Spectrosc. Spectr. Anal. 2021, 41, 3837–3843. [Google Scholar]
  34. Zhang, X.X.; Li, P.; Yu, M.; Jiang, L.W.; Liu, X.; Shan, Y. Progress in non-destructive citrus quality detection using near-infrared spectroscopy. Beijing Shipin Kexue/Food Sci. 2022, 43, 260–268. [Google Scholar]
  35. Zhang, Z.J.; Fu, X.Y.; Chen, K.M.; Zhao, Z.L.; Zhang, Z.X.; Zhao, J. Research on non-destructive detection method of moldy apple core by fusing density and spectral features. Beijing Food Ferment. Ind. 2022, 48, 281–287. [Google Scholar]
  36. Chandel, A.K.; Khot, L.R.; Sallato, B.C. Apple powdery mildew infestation detection and mapping using high-resolution visible and multispectral aerial imaging technique. Sci. Hortic. 2021, 287, 110228. [Google Scholar] [CrossRef]
  37. Xiao, J.-R.; Chung, P.-C.; Wu, H.-Y.; Phan, Q.-H.; Yeh, J.-L.A.; Hou, M.T.-K. Detection of Strawberry Diseases Using a Convolutional Neural Network. Plants 2021, 10, 31. [Google Scholar] [CrossRef]
  38. Liu, X.G.; Fan, C.; Li, J.N.; Gao, Y.L.; Zhang, Y.Y.; Yang, Q.L. Identification method of strawberry based on convolutional neural network. Trans. Chin. Soc. Agric. Mach. 2020, 51, 237–244. [Google Scholar]
  39. Wang, W.; Fang, Y.J.; Liu, J.Q.; Wang, Z.Q. Blueberries grading detection algorithm based on machine vision. Mod. Electron. Technol. 2018, 41, 38–42. [Google Scholar]
  40. Xue, Y.J. Research status and prospect of potato grading technology. Xinjiang Agric. Mech. 2024, 54–57. [Google Scholar]
  41. Bhargava, A.; Bansal, A. Automatic Detection and Grading of Multiple Fruits by Machine Learning. Food Anal. Methods 2020, 13, 751–761. [Google Scholar] [CrossRef]
  42. Zhou, H.; Zhuang, Z.; Liu, Y.; Liu, Y.; Zhang, X. Defect Classification of Green Plums Based on Deep Learning. Sensors 2020, 20, 6993. [Google Scholar] [CrossRef] [PubMed]
  43. Chen, Y.H.; An, X.S.; Gao, S.M.; Li, S.J.; Kang, H.W. A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting. Front. Plant Sci. 2021, 12, 622062. [Google Scholar] [CrossRef]
  44. Zhang, X.; Xun, Y.; Chen, Y. Automated identification of citrus diseases in orchards using deep learning. Biosyst. Eng. 2022, 223, 249–258. [Google Scholar] [CrossRef]
  45. Hu, W.X.; Xiong, J.T.; Liang, J.H.; Xie, Z.M.; Liu, Z.Y.; Huang, Q.Y.; Yang, Z.A. A method of citrus epidermis defects detection based on an improved YOLOv5. Biosyst. Eng. 2023, 227, 19–35. [Google Scholar] [CrossRef]
  46. Manley, M. Near-infrared spectroscopy and hyperspectral imaging: Non-destructive analysis of biological materials. Chem. Soc. Rev. 2014, 43, 8200–8214. [Google Scholar] [CrossRef]
  47. Ma, J.J.; Wang, K.Q. Research Progress of Optical Nondestructive Testing Technology for Fruit Quality. Sci. Technol. Food Ind. 2021, 42, 427–437. [Google Scholar]
  48. Ding, C.; Chen, J.Y.; Zheng, M.M.; Zhang, L.; Wei, W.; Zhang, Y.N. Research progress of 529 hyperspectral image change detection technology. Chin. J. Image Graph. 2024, 530, 1714–1729. [Google Scholar] [CrossRef]
  49. Liao, Z.Q.; He, C.X. Research on prediction and grading of sugar content of apple based on Near-infrared spectroscopy. Inf. Technol. Informatiz. 2023, 93–98. [Google Scholar]
  50. Wang, J.F.; Du, C.L.; Zou, X.Q.; Lin, C.Y.; Hao, Y.B. Research on non-destructive testing technology of apple based on hyperspectral and BP neural network. J. Jiamusi Univ. (Nat. Sci. Ed.) 2023, 41, 98–101+105. [Google Scholar]
  51. Huang, Y.P.; Yang, Y.T.; Sun, Y.; Zhou, H.Y.; Chen, K.J. Identification of Apple Varieties Using a Multichannel Hyperspectral Imaging System. Sensors 2020, 20, 5120. [Google Scholar] [CrossRef]
  52. Wang, Y.L.; Peng, Y.K.; Zhao, X.L.; Shen, L.Y. Design and experiment of Non-destructive testing and grading device for corn seed vigor. Trans. Chin. Soc. Agric. Mach. 2020, 51, 350–356. [Google Scholar]
  53. Chia, K.S.; Jam, M.N.H.; Gan, Z.; Ismail, N. Pre-dispersive near-infrared light sensing in non-destructively classifying the brix of intact pineapples. J. Food Sci. Technol. 2020, 57, 4533–4540. [Google Scholar] [CrossRef]
  54. Li, B.; Han, Z.Y.; Wang, Q.; Sun, Z.X.; Liu, Y.D. Research on scratch grade detection of loquat based on hyperspectral imaging technology. Spectrosc. Spectr. Anal. 2023, 43, 1792–1799. [Google Scholar]
  55. Lu, W.; Cai, M.M.; Zhang, Q.; Li, S. Hyperspectral and ensemble learning for fast grading of Lycium. Spectrosc. Spectr. Anal. 2021, 41, 2196–2204. [Google Scholar]
  56. Liu, Y.D.; Li, M.P.; Hu, J.; Xu, Z.; Cui, H.Z. Study on detection of navel orange granulation by near infrared hyperspectral spectroscopy. Spectrosc. Spectr. Anal. 2022, 42, 1366–1371. [Google Scholar]
  57. Sun, X.P.; Liu, C.C.; Lu, H.Z.; Xu, S. Detection of honey pomelo in different granulation levels based on near-infrared transmittance spectroscopy combined with machine vision. J. Food Sci. Technol. 2021, 39, 37–45. [Google Scholar]
  58. Miguel, V.-C.; María-Teresa, S.; Petter, W.J.; Kristian, A.N.; Dolores, P.-M. Near infrared light penetration in watermelon related to internal quality evaluation. Postharvest Biol. Technol. 2023, 204, 112477. [Google Scholar]
  59. Zhang, L.; Sun, J.; Zhou, X.; Nirere, A.; Wu, X.; Dai, R. Grading detection of saccharin jujube based on hyperspectral imaging technology. J. Food Process. Preserv. 2020, 44, e14591. [Google Scholar] [CrossRef]
  60. Chi, J.T.; Zhang, S.J.; Ren, R.; Lian, M.R.; Sun, S.S.; Mu, B.Y. Detection of eggplant external defects using hyperspectral technology. Mod. Food Sci. Technol. 2021, 37, 279–284+178. [Google Scholar]
  61. Zhang, J.W.; Zhi, J.H.; Guan, X.M.; Zhang, S.; Su, T.; Lin, S.Y.; Yu, P.L.; Li, M.B. Identification Method of Understory Crops Quality Grade Using Hyperspectral Image Technology: A Case of Astragalus membranaceus. J. Northeast. For. Univ. 2024, 52, 79–84. [Google Scholar]
  62. Çetin, N.; Karaman, K.; Kavuncuoglu, E.; Yildirim, B.; Jahanbakhshi, A. Using hyperspectral imaging technology and machine learning algorithms for assessing internal quality parameters of apple fruits. Chemom. Intell. Lab. Syst. 2022, 230, 104650. [Google Scholar] [CrossRef]
  63. Magwaza, L.S.; Tesfay, S.Z. A review of destructive and non-destructive methods for determining avocado fruit maturity. Food Bioproc. Technol. 2015, 8, 1995–2011. [Google Scholar] [CrossRef]
  64. Davur, Y.J.; Kämper, W.; Khoshelham, K.; Trueman, S.J.; Bai, S.H. Estimating the Ripeness of Hass Avocado Fruit Using Deep Learning with Hyperspectral Imaging. Horticulturae 2023, 9, 599. [Google Scholar] [CrossRef]
  65. Sharma, S.; Sirisomboon, P.; Sumesh, K.C.; Terdwongworakul, A.; Phetpan, K.; Kshetri, T.B.; Sangwanangkul, P. Near-infrared hyperspectral imaging combined with machine learning for physicochemical-based quality evaluation of durian pulp. Postharvest Biol. Technol. 2023, 200, 112334. [Google Scholar] [CrossRef]
  66. Zhang, J.; Guo, Z.; Ren, Z.; Wang, S.; Yin, X.; Zhang, D.; Wang, C.; Zheng, H.; Du, J.; Ma, C. A non-destructive determination of protein content in potato flour noodles using near-infrared hyperspectral imaging technology. Infrared Phys. Technol. 2023, 130, 104595. [Google Scholar] [CrossRef]
  67. Qi, H.N.; Shen, C.; Chen, G.; Zhang, J.Y.; Chen, F.N.; Li, H.Y.; Zhang, C. Rapid and non-destructive determination of soluble solid content of crown pear by visible/near-infrared spectroscopy with deep learning regression. J. Food Compos. Anal. 2023, 123, 105585. [Google Scholar] [CrossRef]
  68. Zhang, B.; Gu, B.; Tian, G.; Zhou, J.; Huang, J.; Xiong, Y. Challenges and solutions of optical-based nondestructive quality inspection for robotic Fruits and vegetables grading systems: A technical review. Trends Food Sci. Technol. 2018, 81, 213–231. [Google Scholar] [CrossRef]
Agronomy 14 02395 g001
Figure 1. Machine-vision technology features.
Figure 1. Machine-vision technology features.
Agronomy 14 02395 g001
Agronomy 14 02395 g002
Figure 2. Schematic diagram of the whole machine structure [19]. 1. Hoister. 2. Sorting mechanism. 3. Chain delivery mechanism. 4. Image acquisition system. 5. Screening unit. 6. Electric cabinet. 7. Cup. 8. Belt flipping module. 9. PT-A. 10. Orange. 11. PT-B. 12. LED. 13. Ken. 14. Industrial camera.
Figure 2. Schematic diagram of the whole machine structure [19]. 1. Hoister. 2. Sorting mechanism. 3. Chain delivery mechanism. 4. Image acquisition system. 5. Screening unit. 6. Electric cabinet. 7. Cup. 8. Belt flipping module. 9. PT-A. 10. Orange. 11. PT-B. 12. LED. 13. Ken. 14. Industrial camera.
Agronomy 14 02395 g002
Agronomy 14 02395 g003
Figure 3. System hardware [21].
Figure 3. System hardware [21].
Agronomy 14 02395 g003
Agronomy 14 02395 g004
Figure 4. Core methods [21].
Figure 4. Core methods [21].
Agronomy 14 02395 g004
Agronomy 14 02395 g005
Figure 5. Overall dimension detection system of spheroid fruit.
Figure 5. Overall dimension detection system of spheroid fruit.
Agronomy 14 02395 g005
Agronomy 14 02395 g006
Figure 6. Schematic diagram of picture acquisition. 1. PC. 2. Blueberry. 3. Light source. 4. Industrial camera. 5. Camera support.
Figure 6. Schematic diagram of picture acquisition. 1. PC. 2. Blueberry. 3. Light source. 4. Industrial camera. 5. Camera support.
Agronomy 14 02395 g006
Agronomy 14 02395 g007
Figure 7. Technical roadmap of potato surface recognition [19].
Figure 7. Technical roadmap of potato surface recognition [19].
Agronomy 14 02395 g007
Agronomy 14 02395 g008
Figure 8. Overall design scheme of the system.
Figure 8. Overall design scheme of the system.
Agronomy 14 02395 g008
Agronomy 14 02395 g009
Figure 9. Image acquisition device. 1. Objective table. 2. Mango. 3. Light source. 4. Light source. 5. Industrial camera. 6. Date line. 7. Computer.
Figure 9. Image acquisition device. 1. Objective table. 2. Mango. 3. Light source. 4. Light source. 5. Industrial camera. 6. Date line. 7. Computer.
Agronomy 14 02395 g009
Agronomy 14 02395 g010
Figure 10. Comparison of mobile-citrus network results and development trace results. The tracking series of five identified defective mandarins are shown in figure. Green box stands that this mandarin is identified as normal case, while red box stands that this mandarin is identified as defect [43].
Figure 10. Comparison of mobile-citrus network results and development trace results. The tracking series of five identified defective mandarins are shown in figure. Green box stands that this mandarin is identified as normal case, while red box stands that this mandarin is identified as defect [43].
Agronomy 14 02395 g010
Agronomy 14 02395 g011
Figure 11. Recognition and grading before and after optimization [44].
Figure 11. Recognition and grading before and after optimization [44].
Agronomy 14 02395 g011
Agronomy 14 02395 g012
Figure 12. Six kinds of spectral preprocessing methods [49].
Figure 12. Six kinds of spectral preprocessing methods [49].
Agronomy 14 02395 g012
Agronomy 14 02395 g013
Figure 13. The diagram illustrates the structure of a robotic hand system.
Figure 13. The diagram illustrates the structure of a robotic hand system.
Agronomy 14 02395 g013
Agronomy 14 02395 g014
Figure 14. Schematic diagram of seed-vigor detection and grading equipment. 1. Computer. 2. Near-infrared spectrometer. 3. Optical fiber probe. 4. Light source. 5. Seed box (energetic). 6. Seed box (not energetic). 7. Sorting pipeline. 8. Running pipeline. 9. Single granulation device. 10. Fiber optic sensor.
Figure 14. Schematic diagram of seed-vigor detection and grading equipment. 1. Computer. 2. Near-infrared spectrometer. 3. Optical fiber probe. 4. Light source. 5. Seed box (energetic). 6. Seed box (not energetic). 7. Sorting pipeline. 8. Running pipeline. 9. Single granulation device. 10. Fiber optic sensor.
Agronomy 14 02395 g014
Agronomy 14 02395 g015
Figure 15. Schematic diagram of hyperspectral imaging system.
Figure 15. Schematic diagram of hyperspectral imaging system.
Agronomy 14 02395 g015
Agronomy 14 02395 g016
Figure 16. Hierarchical flow chart of hyperspectral ensemble learning of Lycium ruthenicum [55].
Figure 16. Hierarchical flow chart of hyperspectral ensemble learning of Lycium ruthenicum [55].
Agronomy 14 02395 g016
Agronomy 14 02395 g017
Figure 17. Hyperspectral imaging system. 1. CCD-camera. 2. Spectrograph. 3. Lens. 4. Lamps. 5. Black goji berry. 6. Translation platform. 7. Dark box. 8. Motor controller. 9. Computer.
Figure 17. Hyperspectral imaging system. 1. CCD-camera. 2. Spectrograph. 3. Lens. 4. Lamps. 5. Black goji berry. 6. Translation platform. 7. Dark box. 8. Motor controller. 9. Computer.
Agronomy 14 02395 g017
Agronomy 14 02395 g018
Figure 18. Equipment imaging using hyperspectral technology. (a) Schematic diagram, (b) picture [56].
Figure 18. Equipment imaging using hyperspectral technology. (a) Schematic diagram, (b) picture [56].
Agronomy 14 02395 g018
Agronomy 14 02395 g019
Figure 19. Acquisition system of image information.
Figure 19. Acquisition system of image information.
Agronomy 14 02395 g019
Table 1. Research and comparison of machine vision technology in fruit and vegetable grading.
Table 1. Research and comparison of machine vision technology in fruit and vegetable grading.
AuthorObjectsCharacteristicsConclusionAdvantagesDisadvantages
[19]CitrusPython does the processing
Canny edge detection.
Find contours of citrus
counter area to calculate the contour area.		It has high accuracy, saves time and effort, and reduces the interference of human factors.~~
[20]The 2D projection image is used for integral calculation.The error is 5%.
The accuracy is 90.54%.		Comprehensive and concise information.Accuracy needs to be improved.
[21]AppleThreshold segmentation.
Fisher support vector machine.		Apple’s overall accuracy is 95%.Accurately divides the damage area.
Improves grading efficiency		Minor defects cannot be accurately identified
[22]Machine vision is combined with a robotic arm.The grading accuracy is 95%.
The time for grading is about 5.2 s.		Machine vision is combined with a robotic arm.
It has reliability and practicability.
Multiple metrics.		The number of indicators is relatively small.
Detection speed limited the manipulator speed.
[23]PP-YOLO Object Detection algorithm
writes the control software in Python (PyQt5)		The error is within ±1.5 mm.PP-YOLO object detection: low false detection rate and high efficiency.~
[24]Spherical fruitMinimal enclosing matrix.
Morphological region-filling analysis of fruit surface defects.		The average recognition rate is 94.4%.Surface-defect features can be extracted.
The grading accuracy is high.		Full surface inspection is not possible.
Fruit surface defect characteristics have a certain effect.
[25]A variety of fruitsMulti-sensor information fusion technology.Fruit grading should be considered in many ways.A precise grading of the fruit can be made.Lacks multi-scene and is static.
[26]StrawberryThe SLR camera performs image acquisition and processing.
Median filtering denoising, gray enhancement and binarization processing.		The white background can clearly segment the strawberry fruit.
The median-filtering algorithm can better remove the salt-and-pepper noise mixed in the strawberry image collection process.
Five typical algorithms can segment the image contour clearly.		On-line, lossless and good real-time performance.Effects such as strawberry rot were not considered.
The segmentation and maturity recognition were completed without considering the complex background.
[27]BlueberriesMaximum between-cluster variance-method
morphology.
Least squares method.		The area accuracy is 98.93%.
The accuracy of the perimeter is 87.74%.		Grading was performed based on area and perimeter.
The complexity and cost are low.		Fruiting stems cannot be segmented accurately.
Some images are unimodal and cannot be segmented out.
[28]PotatoSupport vector machine.
BP neural network model.		The average accuracy of SVM is 87.5%.The texture structure of the image processed by the weighted average method is clear.The recognition accuracy is low.
[29]Day lily flowerHeatmap branch.
Improved non-maximum suppression algorithm.
Joint-point prediction.		The recognition accuracy is 91.02%.
The positioning accuracy is 99.8%.		Satisfies most models.
High positioning accuracy.
Detection-box prediction is changed to joint-point prediction.		The recognition accuracy is low.
Recognition and localization in complex environments is not considered.
[30]Black fungusThe DMV-VGR software processes the images.
PLC data monitoring.		A preliminary grading can be performed.Provides a hierarchical scheme.Low accuracy.
No omnidirectional acquisition.
There are uncertainties.
[31]A variety of fruits and vegetablesGaussian filtering.
Fuzzy C-means clustering.
Grab-cut.		The detection and grading accuracy of SVM are 97.63% and 96.59%, respectively.Consider the impact of multiple factors.
The accuracy is relatively high.		~
[12]SVM, K-NN, AnnaThe accuracy of the proposed system is 70%.It works without a network.The accuracy is relatively low, and more fruits and vegetables should be introduced for experimentation.
[32]MangoSobel operator and Canny operator.
MATLAB is used for image processing.		The accuracy of the first-grade fruit is 93.3%.
The accuracy of the second-grade fruit is 95%.
The accuracy of the third-grade fruit is 95%.		Solves the problem of unclear edge and discontinuity.The lighting conditions are not uniform.
~ The representative information is not mentioned in the text.
Table 2. Some research studies on fruit defect detection in recent years.
Table 2. Some research studies on fruit defect detection in recent years.
ObjectNetworkDefect ConditionDataset ConditionNetwork Performance
Image ContentImage BackgroundRecallAccuracyF1 ScorePrecisionmAP
Defective mangoes [32])MATLAB softwareRot, spots, scarsA mangled mangoLaboratory~95%~~~
Green plum defects [42]Improved VGG networkRot, spots, scars, cracksA damaged green plumLaboratory~93.8%~~~
Defective citrus [43]Mobile-CitrusMechanical damage and skin lesionsMultiple defective citrus fruitsLaboratory87.0%88.0%87.1%~~
Defective citrus [44]YOLOv4 and EfficientNetCanker, anthracnose, sunscald, greening, and melanoseMultiple defective citrus fruitsOrchard~89.0%87.2%~~
Citrus epidermal defects [45]Based on the improved YOLOv5Injury and scarA defective citrus fruitLaboratory95.1%~~94.0%95.5%
~ The representative information is not mentioned in the text.
Table 3. Comparative study of near-infrared light technology in fruit and vegetable grading.
Table 3. Comparative study of near-infrared light technology in fruit and vegetable grading.
AuthorObjectsCharacteristicsConclusionAdvantagesDisadvantages
[49]AppleNear-infrared spectroscopy.
PLS model.
PCA dimensionality reduction.
Ridge processing.		The accuracy of grading ranged from 88.38% to 90.84%.The ridge regression model has good stability.
Multiple-preprocessing and dimensionality-reduction algorithms.		Low accuracy-
The data are not good enough and there is overfitting.
[22]Machine vision.
Normalized spectral ratio method.		The grading accuracy is 95%.Nir spectroscopy is combined with robotic arms and machine vision.
More indicators.		Lack of internal metrics.
The dynamic acquisition of the spectrum will have an impact on the model.
[50]Hyperspectral technology is combined with BP neural network.The correlation coefficient R of the prediction set reaches 0.86, and the root mean square error is 0.69.The computational complexity of the model is reduced without losing the main information.Fruit stem and calyx removed, the area is small.
[51]Multi-channel hyperspectral.The accuracy is 0.994.Spectral combination gives better accuracy in variety detection.Spectral combination was not able to improve the results of the best single SR spectra in the visible region.
[52]CornNear-infrared reflectance spectroscopy.
Partial least squares.		The SG convolution accuracy is 98.7% and the prediction set accuracy is 96%.
The overall accuracy of liveness prediction is 97%.		High efficiency of single granulation.
The modeling accuracy and stability are good.		The detection efficiency will be affected by the wheel speed or angle.
[53]White radishPre-dispersive near-infrared light technologyThe accuracy of grading is 80.56%.Pre-dispersive NIR light technology is used.
New detection method, low cost.		It is only applicable when the internal mass changes are small.
[54]IoquatHyperspectral technology.
RF.
Builds models with multiple colors		The accuracy is 100%.Multiple model methods are compared.
High accuracy.		No other defects were identified.
[55]Black wolfberryFD, FFT, HT, SG, Normalize and SNV preprocesses.
PCA, SPA, and CARS extract wavelengths.
LIBSVM, LDA, KNN, RF and NB build the model.
Stacking ensemble learning.		The precision is improved from 0.9417 to 0.9833.
Fast grading can be achieved by hyperspectral ensemble training.		It can obtain spectral and image information at the same time. and fast.
Comparison of multiple methods.		The steps are cumbersome and only suitable for indoor use.
[56]OrangeHyperspectral technology.
PLS-DA and other methods for modeling.		The false positive rate is 0.78%.It also reduces the false positive rate while reducing the dimension of spectral space.The effect of thick skin was not considered.
[57]HoneySPA
FCM
KNN		It can classify grade 3 fruit accurately, but grades 1 and 2 and grades 4 and 5 are easy to misjudge between each other.The samples are non-destructive and can capture internal qualities.It is suitable for grades 1–3, and the recognition accuracy of grades 4 and 5 is low.
[58]WatermelonNear-infrared reflectance spectroscopyR2cv = 0.73, RMSECV = 0.39%, R2p = 0.81, RMSEP = 0.30%.Near-infrared light penetration.Heavily dependent on optical geometry measurements; further instrument optimization is required.
[59]JujubeHyperspectral technology is combined with VISSA-GWO-SVM model.The accuracy rate is 91.67%.The signal-to-noise ratio of the spectrum is improved.
It is fast and lossless.		The recognition accuracy is low.
[60]EggplantHyperspectral continuous-projection method.Rc2 =0.94, Rp2 = 0.90, RMSEC = 0.19, RMSEP = 0.21.
The accuracy rate is 96.82%.		The grading accuracy is improved, and the eggplant damage can be effectively graded and evaluated.The data dependence is strong, and the feature selection will affect its stability.
[61]PotatoPartial least squares regression.
OSC-CARS–PLSR		R2 is 0.9606 and 0.8925.
RMSE is 0.070% and 0.1385%.		The prediction accuracy and stability are improved.It increases the computational complexity and requires the use of more computing resources.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lu, J.; Zhang, M.; Hu, Y.; Ma, W.; Tian, Z.; Liao, H.; Chen, J.; Yang, Y. From Outside to Inside: The Subtle Probing of Globular Fruits and Solanaceous Vegetables Using Machine Vision and Near-Infrared Methods. Agronomy 2024, 14, 2395. https://doi.org/10.3390/agronomy14102395

AMA Style

Lu J, Zhang M, Hu Y, Ma W, Tian Z, Liao H, Chen J, Yang Y. From Outside to Inside: The Subtle Probing of Globular Fruits and Solanaceous Vegetables Using Machine Vision and Near-Infrared Methods. Agronomy. 2024; 14(10):2395. https://doi.org/10.3390/agronomy14102395

Chicago/Turabian Style

Lu, Junhua, Mei Zhang, Yongsong Hu, Wei Ma, Zhiwei Tian, Hongsen Liao, Jiawei Chen, and Yuxin Yang. 2024. "From Outside to Inside: The Subtle Probing of Globular Fruits and Solanaceous Vegetables Using Machine Vision and Near-Infrared Methods" Agronomy 14, no. 10: 2395. https://doi.org/10.3390/agronomy14102395

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop