1. Introduction
The tomato is an annual herb belonging to the Solanaceae genus. It is originally from South America, and today, countries such as the United States, Russia, Italy, and China are the primary producers of tomatoes. It is a good source of carotenoids (namely, lycopene, β-carotene, and lutein), flavonoids, phenols, minerals, vitamin C, vitamin E, dietary fiber, and other essential nutrients and phytochemicals [
1,
2,
3]. Regarded as the most crucial vegetable [
4], it is one of the most widely consumed vegetable crops across the globe [
5,
6]. People love it for both daily consumption and food processing purposes.
In the tomato market, it is crucial to test the quality of tomatoes and classify them appropriately for sale. The main factors that determine the quality of a tomato are its color and appearance [
7]. Traditionally, fruit farmers use manual sorting to identify the quality of tomatoes by visually inspecting them [
8]. However, this process is inefficient, labor-intensive, expensive, and subjective. Therefore, we developed an automatic tomato sorting system. This system can quickly sort a large number of tomatoes, improving efficiency and reducing the workload for fruit farmers. Additionally, since different markets have varying demands for tomato quality, our sorting system can detect three key attributes to obtain more accurate data than manual sorting. It ensures that each tomato meets the quality requirements of different markets, ultimately improving the competitiveness of our tomatoes in the market.
We utilized the technology of machine vision to recognize and process images of tomatoes. Machine vision offers several advantages, such as being non-destructive, accurate, and consistent. This technology is capable of identifying the internal and external characteristics of agricultural products, including external size, maturity, defect area, internal moisture, and nutrient composition [
9]. By using machine vision technology for automatic sorting, we can reduce costs, improve efficiency, and increase the accuracy of sorting. Zhang et al. developed an automatic cherry tomato grading system based on machine vision. The system collected images of each cherry tomato from three different angles and extracted nine feature points from each image. The Principal Component Analysis (PCA) and the Linear Discriminant Analysis (LDA) were used for feature analysis and classification. The system classified cherry tomatoes into three levels: mature, semi-mature, and immature. The accuracy of the system was tested on a total of 414 cherry tomatoes and achieved an overall accuracy rate of 94.9% [
10]. M. Haggag et al. proposed an intelligent hybrid experimental deep-learning algorithm based on a tomato sorting controller. Five deep learning training algorithms were used as variables to analyze the three groups of tomato images, identify the maturity and defect degree of tomatoes, and obtain the optimal algorithm through comparison to improve the reliability and classification accuracy of automatic tomato sorting [
11]. Harvey Jake G. Opena et al. used the artificial neural network trained by ABC to automatically grade tomato maturity [
12]. M. Fojlaley et al. studied tomato classification and sorting in machine vision based on SVM, MLP, and LVQ [
13].
We utilized machine vision analysis technology along with a deep learning algorithm to capture tomato image features, including defect area, size, and maturity. These three indicators were accurately analyzed simultaneously, resulting in more refined sorting and making it more convenient to sell. We categorized tomatoes into three grades based on three characteristic indices: premium grade, first grade, and second grade.
In terms of hardware, O. O. Arjenaki et al. used computers to process CCD camera signals and image capture [
14]. Our hardware mainly used the Raspberry PI, which is an efficient and powerful small computer [
15]. Raspberry Pi offers several advantages over traditional computers, including its low cost, compact size, and user-friendly interface with 40 GPIO (universal input/output) pins, allowing for easy connection of multiple external hardware devices. Moreover, it supports almost all of the peripherals of Arduino. Raspberry Pi also integrates machine learning and deep learning models [
16], and it has a faster processor than Arduino and other microcontroller modules. The latest Raspberry Pi 4B version integrates a 1.6 GHz processor, while the Arduino board integrates the controller. As a result, this board has a faster response time compared to other modules. At this stage, the Raspberry PI embedded system has become an important scientific and engineering application [
17], which helps to improve the objectivity and simplicity of microcomputer and camera fruit physical and chemical properties and quality parameter measurement systems [
18]. The Raspberry PI in this system is equipped with image processing libraries OpenCV and YOLOv8 for image recognition and processing, and the robot arm is controlled by the Arduino CNC shield V3.0 driving board for sorting.
The rest of the paper is arranged as follows: The second part introduces the overall process of the tomato sorting system and grading standard. The third part presents the materials and methods used in the system implementation. The fourth part describes the implementation of the interactive interface of the software. The fifth part explains the experimental method and shows the experimental data. Finally, the sixth part summarizes the conclusion.
4. Implementation of Software Interactive Interface
The tomato grading software was designed with the Raspberry PI driver, communication, control process, and human–computer interaction interface. The development of the Raspberry PI, communication functions, and control software was carried out using RPI.GPIO0.7.1, Socket3.11, and PyQt5.15. It completes the overall software design of fruit sorting equipment. Apart from the automatic control process, the manual control function is also essential. Manual control enables real-time control of hardware such as conveyor belts, mechanical arms, grading mechanisms, and cameras. Additionally, manual control includes operations such as selecting regions of interest in the image and selecting image processing methods.
4.1. Communication Implementation
The Raspberry PI comes with a WiFi Bluetooth module, which can be set as an AP hotspot by modifying the relevant system files. The specific process is shown in
Figure 13.
When we disable the WLAN0 interface, the Raspberry PI may not be able to connect to other hotspots. In addition, the WLAN0 port used as the AP hotspot should have a fixed IP address. To set up the hotspot, we downloaded the Hostap program and modified the hotspot configuration, including the network interface, hotspot name, encryption method, and password. To ensure complete network communication, we allocated an IP address to the new device (PC). The Raspberry PI 4B comes with a WiFi Bluetooth module, which can use the AP hotspot to achieve TCP/IP communication based on Socket mode. The Socket provides an interface for the implementation of the application layer and acts as an intermediate software abstraction layer between the application layer and the transport layer of the TCP/IP protocol family. The communication equipment is divided into client and server, where the PC is the client and the Raspberry PI is the server.
4.2. Setting of User Interaction Interface
The function modules of the control software of the grading system are shown in
Figure 14. The user interface is the embodiment of the integration of the device functions. This software includes the user information module, the device connection module, the image operation and classification control module, the hardware control module, the information display module, and the file operation and database module.
The application form module of the software is designed using PyQT5, an integrated library that is the Python interface of the graphics programming framework Qt5. The application form of the software is designed by the Qt Designer tool, and the controls are arranged in the edit box of the form by dragging. After setting the relevant parameters, such as position, size, and inclusion relationship of the controls, it is saved as a UI file. Pyqt5 is used to compile the UI file into a Python file to form the GUI module of the form. The Pyqt5 signal and slot programming mechanism were used to write the transaction logic of the form application. The main interface of the software is shown in
Figure 15.
5. Experimental Data and Discussion
In this study, the three indicators of defect degree, size, and maturity of tomatoes are used as the basis for evaluating the quality of tomatoes. According to the method mentioned above, the defect degree, size, and maturity of tomatoes are simultaneously detected, and the tomatoes are graded according to the tomato grading standards specified by us. The following table is part of our actual test results, as shown in
Table 6.
We used 90 tomatoes to do experiments, and the accuracy of size was 100%, the accuracy of ripening was 98.37%, and the accuracy of defect area was 99%, so the accuracy of our entire system could be 99.1% on average of these three indicators. We have conducted tests on the tomato sorting system and gathered data from ten tests, as detailed in
Table 7. Based on our findings, we estimate that this tomato sorting system can process 1350 tomatoes per hour, significantly surpassing the manual sorting capacity of 726 tomatoes per hour [
20,
21]. In addition, the efficiency of manual sorting decreases over time, leading to slower sorting and reduced accuracy.
The tomato sorting system studied by Joice A. et al. has an efficiency of 1800 tomatoes per hour, but it only sorts the size and maturity of tomatoes. The accuracy percentage for sorting full-ripened tomatoes is 98% [
22]. In comparison, our system can identify and classify three characteristics of tomatoes: size, ripeness, and defect area, and the accuracy percentage of sorting full-ripened tomatoes is 98.37%. Haggag, M. et al.’s theoretical classification accuracy of tomato ripeness and defect (overripe and rotten) was 90% [
11]. Rajkumar P et al.’s tomato color sorter only recognizes the maturity of tomatoes, with an overall recognition accuracy of 94.6%. Additionally, their system operates at an efficiency of 40 kg of tomatoes per hour, equivalent to an estimated 254 tomatoes per hour based on the average weight of tomatoes [
23]. Most of the current tomato sorting systems rely on computers to identify and classify tomatoes [
10,
11,
14,
23]. However, the Raspberry Pi, as a convenient and cost-effective microcomputer, offers a price advantage over using traditional computers for tomato identification and classification.
6. Conclusions
This paper presents a solution to the problem of tomato sorting by designing and developing a tomato sorting system based on machine vision. The system utilizes the Raspberry Pi 4B as the core control platform and a mechanical arm as the sorting device. It detects the size, maturity, and defect area of tomatoes and integrates the judgment results of these three characteristics to obtain the final grade of the tomatoes. The experimental results demonstrate that the system achieves a total sorting accuracy of over 99.1%, with 100% in size, 98.37% in maturity, and 99% in detecting defect areas. Additionally, the sorting efficiency of the system can reach 1350 tomatoes per hour. Overall, this system has the potential to enhance the competitiveness and profitability of the fresh tomato market, and it holds significant economic value, making it worthy of promotion and widespread application.
The limitations of this study are as follows: First, the system fails to take into account the differences between individual tomato data of different varieties under different environmental conditions, and this system is designed specifically for tomatoes, which has limited adaptability to the sorting of other crops and will decrease the accuracy rate. Meanwhile, the initial setting and calibration of the human–computer interaction interface will become an obstacle for users to use. Therefore, our future research will comprehensively collect and train the data of various varieties of tomatoes under different environmental conditions so that the system has no blind spots in the tomato market. Meanwhile, datasets from other crops will be added and trained to improve the universality of the system. More sensitive and easier-to-adjust sorting equipment will be used on the hardware to adapt to the shapes of different crops: height and size. We will also optimize the human–computer interaction interface, reduce the complexity of user use without affecting the function, make the interface clearer, and add a user operation guide and online question module to help users quickly grasp the basic operations.