Next Article in Journal
Breeding for Biotic Stress Resistance in Pea
Next Article in Special Issue
Study on Vibration Characteristics of Paddy Power Chassis under Different Driving Conditions
Previous Article in Journal
Exploring the Genotype-Dependent Toolbox of Wheat under Drought Stress
Previous Article in Special Issue
Quantification of Biophysical Parameters and Economic Yield in Cotton and Rice Using Drone Technology
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Designing, Optimizing, and Validating a Low-Cost, Multi-Purpose, Automatic System-Based RGB Color Sensor for Sorting Fruits

by
Abdallah E. Elwakeel
1,*,
Yasser S. A. Mazrou
2,
Aml A. Tantawy
3,
Abdelaziz M. Okasha
4,
Adel H. Elmetwalli
5,
Salah Elsayed
6,7 and
Abeer H. Makhlouf
8
1
Agricultural Engineering Department, Faculty of Agriculture and Natural Resources, Aswan University, Aswan 81528, Egypt
2
Applied College, Muhayil Asir, King Khalid University, Abha 62587, Saudi Arabia
3
Food Science and Technology Department, Faculty of Agriculture and Natural Resources, Aswan University, Aswan 81528, Egypt
4
Department of Agricultural Engineering, Faculty of Agriculture, Kafrelsheikh University, Kafr El-Sheikh 33516, Egypt
5
Agricultural Engineering Department, Faculty of Agriculture, Tanta University, Tanta 31527, Egypt
6
Agricultural Engineering, Evaluation of Natural Resources Department, Environmental Studies and Research Institute, University of Sadat City, Sadat City 32897, Egypt
7
New Era and Development in Civil Engineering Research Group, Scientific Research Center, Al-Ayen University, Thi-Qar, Nasiriyah 64001, Iraq
8
Department of Agricultural Botany, Faculty of Agriculture, Minufiya University, Shibin El-Kom 32511, Egypt
*
Author to whom correspondence should be addressed.
Agriculture 2023, 13(9), 1824; https://doi.org/10.3390/agriculture13091824
Submission received: 23 August 2023 / Revised: 5 September 2023 / Accepted: 10 September 2023 / Published: 18 September 2023
(This article belongs to the Special Issue Agricultural Automation in Smart Farming)

Abstract

:
The use of automatic systems in the agriculture sector enhances product quality and the country’s economy. The method used to sort fruits and vegetables has a remarkable impact on the export market and quality assessment. Although manual sorting and grading can be performed easily, it is inconsistent, time-consuming, expensive, and highly influenced by the surrounding environment. In this regard, this study aimed to design and optimize the performance of a low-cost, multi-purpose, automatic RGB color-based sensor for sorting fruits. The proposed automatic color sorting system consists of hardware components including a machine frame, belt and pulleys, conveyor belt, scanning zone, plastic boxes, electric components (stepper motors, RGB color sensors, Arduino Mega, motor drivers), and software components (Arduino IDE version 2.2.1 and C++). Calibration was performed for the light intensity sensor to measure the light intensity inside the scanning zone, the conveyor speed sensor, and the RGB color sensors by testing the RGB color channels. The sensor, the height, conveyor belt color, and light intensity should be carefully adjusted to ensure a high performance of the color-based sorting system. The results showed that the appropriate sensor height ranged from 15 to 30 mm, the optimum color of the conveyor belt was black, and scanning the objects at a light intensity of 25 lux achieved the best output signals. The RGB color sensors achieved an analytical performance similar to that obtained with manual sorting without requiring the use of computers for image processing like other automatic sorting systems do in order to gather RGB data.

1. Introduction

Using automatic sorting and grading systems of fruits in the agricultural sector increases the products’ quality. The export market and quality evaluation are affected by the assortment of fruits. The essential sensory aspect towards fruits that influences their marketable value, the preferences of consumers, and their decision, is appearance [1]. Huge post-harvest losses in handling and processing and the increased need for food products of high quality and safe food necessitate the development of precise, fast, and objective quality determination of food and agricultural products [2]. The initial phase in the post-harvest procedure is the sorting of the product according to color and quality. This process is crucial to maintaining freshness during the harvesting process and distribution to marketing places [3,4]. Manual sorting of fruits is regarded as being quite easy [5], but it is inconsistent, time-consuming, variable, subjective, onerous, expensive, and easily influenced by the surrounding environment. Therefore, the automatic systems are expected to reduce the post-harvest processing time to speed up the marketing process and therefore enhance the shelf-life of the produce [1,4,6].
Picture recognition technology [PRT] has been extensively used across a variety of fields as a result of the growth of information technology (IT) and the appearance of free and open-source solutions [7,8]. For example, many researchers have been working on PRT for many years on text recognition, object identification, etc. [9]. The use of modern technology for agricultural applications has been examined in recent decades in order to manage various practices and make decisions based on ongoing field monitoring using a variety of effective sensors [10], as well as due to the growing demand for fresh fruits and vegetables in the market [11]. For both medium- and large-scale enterprises, the concepts of smart farming have improved the quality of fruit and vegetable production [12]. Sorting, packaging, and delivering the product to the market with the use of efficient logistical techniques has improved market production, which has resulted in commercialization [13]. However, the high cost of these technologies and the need for specialized learning are the two main major challenges that researchers are facing in adopting them in agriculture [14,15,16].
Color is shown to be very essential for detecting and assessing the physical properties of materials [17,18]. Image processing and object detection techniques are achieved in industries using color characteristics, while color sensors can be useful in detecting and sorting materials according to their color [18]. Additionally, environments can also be detected and confirmed using color sensitivity gradations. This approach can immediately and spontaneously detect changes [19], where the three basic colors (RGB; red, green, and blue) and the wavelength associated with each color are determined from the reflected light. Juliano et al. [20] distinguished the physical appearance of strawberry fruit and tomatoes using a color sensor to determine their maturity, and organize and separate them by considering their color. For every ripening stage, RGB has a different value.
Recently, different techniques and algorithms have been used to sort and classify fruits and vegetables using color features. For instance, Pourdarbani et al. [21] evaluated an automatic sorting system for date palm fruits. Ansari et al. [22] employed a two-level discrete wavelet transform to extract fruit image features and support vector machines [SVM] to classify the fruit images based on features of the dataset images. Prasetyo and Bimantaka [23] used the back propagation method based on an Android application; red dragon fruits were classified into several stages of ripeness based on color features. Behera et al. [24] used Convolutional Neural Network [CNN] and SVM to classify and identify various fruit properties, where 40 different fruit datasets were investigated. Risdin et al. [25] used CNN to classify fruits based on their variety. Sari et al. [4] used a procedure based on both image processing and a sensor in a tomato sorting machine to classify tomatoes by color, size, and weight.
In this context, the overall aim of the present research was to design, optimize, and evaluate a multi-purpose automatic sorting system based on an RGB color sensor, and design a smart monitoring unit for sorting apple fruits. To the best of our knowledge, the current study offers two significant new insights. Firstly, the design of a multi-purpose automatic sorting machine based on color features for different fruits. The proposed device offers more precise RGB color detection than traditional inspection techniques by scanning fruits from two opposite directions. Moreover, some important design parameters such as conveyor belt color, light intensity, type of color channels, and height of RGB color sensors, which have a great impact on the performance of automatic sorting and grading systems based on RGB color features, were optimized.

2. Materials and Methods

2.1. Design and Construction of the Proposed Automatic Color Sorting System [ACSS]

To achieve the proposed objectives for the current research, a proposed ACSS prototype was designed and evaluated at the Department of Agricultural Engineering, Faculty of Agriculture and Natural Resources, Aswan University, Aswan, Egypt, 2023. The prototype was designed for sorting a wide range of fruits based on color only. Under laboratory conditions, the operational efficiency of the prototype was evaluated using three varieties of local apples. Figure 1 illustrates the isometric view of the proposed ACSS.

2.1.1. ACSGS Components

The ACSS includes many hardware components as shown in Figure 2, like the machine frame (made from C and Rectangular metallic bars), belt and pulleys (used for motion transmission from stepper motor to conveyor belt), conveyor belt (for transmitting fruits through the scanning zone), scanning zone (for detecting and sorting fruits), plastic boxes (for collecting the sorted product), and sliding plate (for transmitting the detected and sorted fruits to plastic boxes without damage). Detailed views and main dimensions of the purposeful ACSS are shown in Figure 3.

2.1.2. Design of Electronic Circuits

1.
Design of smart monitoring unit and control circuit
Software components are the operating systems that guarantee a system runs smoothly. Four main software components, including an Arduino mega board, two RGB color-based sensors, three stepper motors, and three motor drivers, made up the smart monitoring unit and control circuit, in addition to some axillary parts such as a laptop, jumpers, bread boards, wires, and a USB cable, which are listed and described in Figure 4.
A fritzing program was used to sketch the electric circuit of different components as seen in Figure 5.
  • Color sensors
Photoelectric RGB color-based sensors generate light and pick up reflected light from object [26]. It is a popular sensor from Texas Advanced Optoelectronic Solutions (Plano, TX, USA). For example, during a scan of a red object, each LED is shining on the object, which causes the reflected light from the object to fall on the color sensor, and it is known that a red object tends to reflect all the red light falling upon it while it absorbs the green and blue lights, as seen in Figure 6. In such a situation, the color sensor receives little light intensity from the green and blue LEDs, while receiving maximum light intensity from the red LED [27].
The color sensor is connected to an Arduino mega board, as shown in Figure 7. It has eight connection pins linked to the Arduino mega board.
The proposed approach for the detection and sorting of multiple fruits is depicted in Figure 7 wherein two RGB color sensors were used for color scanning of fruits on the scanning zone and to determine their RGB color characteristics before showing them on the laptop monitor.
The output is a square wave (50% duty cycle) with frequency being directly proportional to the light intensity (irradiance), as shown in Equation (1) [28]:
f o = f D + ( R e E e )
The received signal from the RGB color sensor is read by pulse In command [29] and the frequency f0 is converted into the length of pulses, equivalent to the period T of the signal based on Equation (2).
T = 1 f 0   
The light-to-frequency converter in the RGB color-based sensor read an 8 × 8 array of photodiodes. Four sets of various sixteen photodiodes were used that had different filter colors, including blue filters, green filters, red filters, and clear with no filters. The four types of photodiodes were interdigitated in order to reduce the impact of incident irradiance’s non-uniformity. All photodiodes of the same color were linked in parallel. Pins S2 and S3 were used to determine whether the photodiode group (red, green, blue, and clear) was active or not. Lastly, RGB color-based sensor signals from fruits were recognized, sent to a laptop or PC via USB cable, digitized, and finally saved on the PC in RGB color PW.
  • Stepper motors
Stepper motors are frequently employed in electrical and mechanical equipment and can directly convert electrical pulse signals into angular or linear displacement for the implementing agencies. Controlling the pulse frequency can also affect the stepper motor’s speed and acceleration [30]. Stepper motors were employed to control the conveyor belt movement. Two identical stepper motors were used to operate both push arms.
2.
Measuring electronic circuit [MEC]
The MEC was designed for measuring conveyor belt operational speed and light intensity across the scanning zone, then sending the obtained data directly to a laptop or PC through a USB cable. The delay between the frequency measurements can be changed easily by editing the programming code of Arduino IDE software. Table 1 Illustrates list of components of MEC.
3.
Operating algorithm
Many variables should be adjusted before the sorting process (e.g., fruit or vegetable type and characteristics), including conveyor belt speed (based on stepper motor speed), color grades, push arm delay for each stepper motor, color sensor lens position, and stepper motor speeds for both push arms according to the operational algorithm as depicted in Figure 8.
As described in the operating algorithm, the first step is editing programming codes based on fruit type and operation criteria (i.e., conveyor belt speed, color grades, push arm delay, color sensor lens position, and stepper motors speeds), then initializing both stepper motors for both push arms. After starting, both RGB color sensors perform a color reading when the item (different fruits) is conveyed at a constant speed. The operating system is programmed to activate push arms at two conditions; the first condition being when the color detected is red (push arm No. 1 [ON] and push arm No. 2 [OFF]), and the second condition when the color detected is yellow (push arm No. 1 [OFF] and push arm No. 2 [ON]). However, when the color detected is green or any other color, the conveyor belt (stepper motor) continues and drops the object in the plastic box (push arm No. 1 [OFF] and push arm No. 2 [OFF]).

2.2. Programming Codes of RGB Sensors

Programming of the operating codes is highlighted in the following steps:
Step 1: Define the different pins of the RGB color sensor [S0, S1, S2, and S3] and the three pulse width modulation variables for red, green, and blue.
Step 2: Selection of pins [S0, S1, S2, and S3] as output as this will make the color photodiode high or low and take the out-pin of the RGB color sensor as input.
Step 3: The RGB color sensor pins S2 and S3 are used to take the characteristics of RGB color, as presented in the programming code. The data in Table 2 are available in the RGB color sensor data sheet.
For various frequency counters or microcontrollers, scaling the output frequency enhances the sensor assessments. Generally, the out-pin will provide frequency and frequency scaling should initially be adjusted at 20%. Set the baud rate at 9600 bytes per second to start serial communication between the Arduino mega board and the laptop using a USB cable.
The programming code for input-pins S2 and S3, (LOW and LOW), respectively, was written for activation of the red photodiodes to take the readings for red color frequency and print it on a serial monitor.
The programming code for input-pins S2 and S3, (LOW and HIGH), respectively, were written to activate the blue photodiodes to take the readings for blue color frequency and print it on a serial monitor.
The programming code for input-pins S2 and S3, (HIGH and HIGH), respectively, were written for activation of the green photodiodes to take the readings for green color frequency and print it on serial monitor.
Step 4: The output frequency’s minimum and maximum RGB values are calculated, saved, and applied once a color sorting system starts and sets the color RGB value following the detection of fruit and colors.

2.3. Experimental Set up

2.3.1. Software Calibration

The sorting process for fruits requires an accurate operation for all devices that have an impact on the quality of the sorted items by calibrating over a specified period of time. The sensors used in ACSS were calibrated to assure their accuracy as follows:
  • Conveyor belt speed sensor
A digital LASER tachometer (model: UNI-T UT373, Uni-Trend Technology, Dongguan city, China) was used to calibrate the conveyor belt speed sensor (HC-89). The conveyor belt operating speed was set at different values, and the observed values were compared with the values of the sensor’s readings.
2.
LDR sensor module (light-dependent resistor)
The reading of the sensor was compared with the reading obtained by a digital light meter (LUX meter) (model: UNI-T UT383, China) to calibrate the LDR sensor module [light intensity sensor]. Calibration was performed using an LED light source with changing illumination intensity.
3.
RGB color-based sensor (control system)
For the calibration of the RGB color sensor (model: TCS3200, Ams AG, Unterpremstaetten, Austria) and the detection of the RGB color channels, both RGB color sensors were fixed on the scanning zone in opposite directions at a specified height of 30 mm above the objects, under 25 lux light intensity. During the calibration of the RGB color sensor, six different colored paper sheets (red, green, blue, yellow, white, and black) were used. Then, the three color channels (RGB; red, green, and blue) were recorded and evaluated and both minimum and maximum RGB values for each color channel were taken. Finally, the programming codes of the control system were edited based on minimum and maximum RGB values.

2.3.2. Selection of Some Engineering Factors Affecting the Performance of the RGB Color Sensors

The red, green, and blue channel values of three different apple fruit varieties (red, green, and golden) were recorded at different engineering factors (sensor height, conveyor belt color, and light intensity). The three different apple fruit varieties (red, green, and golden) were put on the conveyor belt speed without movement of the conveyor belt at zero time. Then, the conveyor belt was run, and the output signals changed up and down.
  • Sensor height
To identify the optimal height for installing RGB color-based sensors, the conveyor belt speed was set at 0.1 m/s, and light intensity at 25 lux. Care was taken to avoid surrounding effects; thus, the conveyor belt color was made in white. According to Khanh et al. [31], due to space limitations, the sensor cannot be placed closer than 15 mm. Therefore, the sensor height was tested at 15, 30, and 50 mm. Tests were run on every batch of three different apple fruit varieties (red, green, and golden). Thirty apples were tested 3 times, totaling 90 tests.
2.
Conveyor belt color
Tests were run in every batch of three different apple fruit varieties (red, green, and yellow). Thirty apple were tested three times, for a total of 90 tests. This test was conducted at a conveyor belt speed of 0.1 m/s, as recommended by Khanh et al. [31]. The two RGB color sensors were installed at a specified height of 30 mm and at a light intensity of 25 lux. The test was conducted using different colors of the conveyor belt by covering the conveyor belt with colored paper (blue, black, and white), aiming to determine the appropriate conveyor belt color that gives the best RGB values compared to those of fruits being sorted to raise the automatic system’s efficiency.
3.
Light intensity
The light intensity during the scanning of fruits to be sorted has a direct impact on the accuracy and quality of the process. During this test, the conveyor belt speed was set at 0.1 m/s as recommended by Khanh et al. [31], and the two RGB color sensors were fixed at 30 mm height. Utilizing attached lamps on the RGB color sensor board, the test was conducted using different light intensities of 25, 40, and 50 lux. Tests were run on every batch of the three different apple varieties (red, green, and golden).

2.4. Sample Collection

A total of 30 unsorted apples of three different colors (red, green, and golden; red delicious, Granny Smith, and golden delicious cultivars, respectively) were collected from marketplaces in Aswan, Egypt for the experimental works. Steps were taken to ensure that each batch’s selection of apples from the gardens was random. After the collection of apples, each apple was given a code with a serial number based on its color. They were stored for only 24 h at 4 °C until the start of the measuring process to prevent the color characteristics of the examined samples from changing.

2.5. Statistical Analysis

Some statistical metrics were used, such as Pearson correlation (R), R-squared correlation (R2), root mean square error (RMSE), and index of agreement (IOA), in order to ensure the accuracy of the different sensors used in the measurement processes, such as (RGB color sensor, conveyor belt speed sensor, and light intensity sensor). Comparisons were made between the data obtained from the different sensors and the corresponding (reference) data observed by the standard devices under the same operating conditions. R2 expresses the strength of the correlation between the measured and observed data. Pearson correlation (R-value) can be calculated according to Equation (3).
R = N ( X Y ( X ) ( Y ) ) ( n X 2 X 2 ) ( n Y 2 Y 2 )       
The parameters being described include “ X ”, which represents the actual value determined by the formula; “ Y ”, which is the projected or simulated value; and “ n ”, which refers to the total number of data points.
RMSE compares the difference between measured data by the different sensors and the observed data by the standard devices. RMSE can be calculated according to Equation (4).
R M S E = i = 1 n ( X i Y i ) 2 N
The parameters being described include “ X i ”, which represents the actual value determined by the formula; “ Y i ”, which is the projected or simulated value; and “N”, which refers to the total number of data points.
IOA describes the ratio of the MSE and the potential error. It ranges between 0.0 and 1.0; the agreement value of (0.0) indicates no agreement at all and (1.0) indicates a perfect match. IOA can be calculated according to Equations (5) and (6).
I O A = 1 i = 1 n X i Y i 2 i = 1 n Y i X o ¯ + X i X o ¯ 2
0 d 1  
where X i is the observation value, Y i is the forecast value, X o ¯ is the average observation values, and Y i   is the average forecast values.
The statistical analyses of the obtained data were conducted at a 0.05 significance level by the analysis of variance (ANOVA) using the statistical analysis program of IBM SPSS v25 and Microsoft Office (Excel 365).

3. Results and Discussion

3.1. Calibration of Light Intensity Sensor and Conveyor Belt Speed Sensor

Some environmental and operating factors (light intensity and conveyor belt speed) can affect the accuracy of the measured values for the RGB color sensors utilized in the proposed ACSS. Inaccurate measurements could result from these factors. In order to validate sensor measurements before use, it is vital to apply sensor calibrations that are subject to the majority of the aforementioned factors. In addition, the output signals of the sensors need to be calibrated against the reference standard instrument for producing calibration curves that describe the responses of the sensors to the reference instrument data, ensuring data acquisition quality [32].
Before deploying the sensors for the proposed ACSS measurements, the sensors were calibrated against the suitable reference standard instruments. Table 3 shows the comparison between the measured and the reference values for light intensity and conveyor belt speed using the standard statistical metrics (R, R2, RMSE, and IOA). The results listed in the table demonstrate that the used sensors achieved the required accuracy within the target parameters, which is acceptable for the tested sensors.
The sensors’ calibrations were applied for three different sensors, as shown in Figure 9 and Table 3. The observed values are plotted on the x-axis, and the measured values are plotted on the y-axis for each calibration curve. Figure 9 shows the validation of the light intensity measurements. The measured light intensity using the LDR sensor has been validated with the observed measurements by the light intensity meter with high R2 values of 0.996 (RMSE = 26.9 and IOA = 0.986), indicating a perfect match between the measured light and the observed light. The LDR sensor showed a high performance at the various light intensities and had strong linear regressions (y = 0.992x + 0.235) that nearly overlapped the 1:1 line.
The measured speed using the HC-89 sensor has been validated with the observed measurements by the speed meter (Figure 10). The results showed a strong relationship between the measured and observed speed values with a high R2 of 0.99 (RMSE = 0.143, and IOA = 0.997), indicating a perfect match in the measured conveyor belt speed with the observed speed. The HC-89 sensors showed a perfect performance at the various tested speed values and had remarkable linear regressions that nearly overlapped with the 1:1 line (y = x + 0). A linear regression equation was derived to predict conveyor belt speed as follows:
y = 0.992 x − 0.0103.
Sofu et al. [33] reported that in order to analyze the visual properties of apples, two identical industrial color cameras were set on the roller conveyor. Four images of any apple rolling on the conveyor can be captured and processed using image processing software in 0.52 s. The proposed machine can sorted averagely 15 apples in per second using two channels, in real time. In the experimental studies, the system design was tested using three different conveyor band velocities and three apple cultivars to sort and inspect 183 samples with an average sorting accuracy rate of 73–96% [32].

3.2. Calibration of RGB Color Sensors

Six different colors (white, red, yellow, blue, green, and black) were chosen. These represent the majority of most colors of fruits that exist in nature, in order to calibrate RGB color channels of the RGB color-based sensors to ensure high operating efficiency. In addition, the RGB color-based sensors have been calibrated to obtain maximum and minimum RGB values for various colored papers. The proposed ACSS use these values in the programming of the control circuits.
Figure 11 illustrates the average RGB values obtained from the laboratory tests of the RGB color-based sensors, where the conveyor belt speed was set at zero and the illumination intensity at 25 lux. Table 4 shows the minimum and maximum RGB values obtained from the same laboratory tests.

3.3. Effect of the Sensor Height

At varied sensor heights of 15, 30, and 50 mm, the difference in the graph for the green and blue channels is very clear, as depicted in Figure 12. This is obvious because of the large size of the apple, which accordingly reduces the radiation intensity [34]. The radiation intensity increases when the sensor is placed at a distance higher than 15 mm. In another instance, if the sensor is positioned too closely, the apple’s exposure to radiation is reduced, which lowers the radiation intensity. Green and blue channels are selected as the detection factor when comparing three color channels because they generate strong significant signals.

3.4. Effect of the Conveyor Belt Color

The distinction between the apple and the conveyor belt is clear with the black conveyor belt compared to the white and blue conveyor belts as shown in Figure 13. With the white conveyor belt, it was also possible to distinguish between the status of with and without fruits on with a slight fluctuation, as presented in the Figure 13. In case of the blue conveyor belt, it becomes more challenging to distinguish, as noticed in Figure 13.

3.5. Effect of Light Intensity

As demonstrated in Figure 14, the difference between the green and blue channels can be clearly seen at a light intensity level of 25 lux. This is explained by the fact that increased light intensity increases the reflectance and reduces RGB color pulse width (PW) [35], so based on the information illustrated in Figure 14, the optimum light intensity is 25 lux. The RGB color sensors’ connected lights can offer this level of light intensity without using any external light sources.
When the light intensity is 40 lux for the green and blue channels, there is a moderate difference between the upper and lower points of the graph. In this situation, the device has a medium ability to sort and color-distinguish items. On the other hand, at 50 lux light intensity, the difference between the upper and lower points of the graph is very small, and thus the device’s ability to color-distinguish between objects is nominal. Other studies such as Afrisal et al. [36] found that the test has been conducted for orange fruit in sizes between 25 mm–75 mm and indicates a good result in color classification, sizing, and sorting. The fluctuation of the lighting condition does not affect the precision of the system when the light intensity is enough (150–500 lux). Machine vision systems are affected by the level and quality of illumination. The changes in light level and wrong positioning could have negative effects on the software used in the system [6].

4. Conclusions and Future Work

The present research aimed to design, adjust, and evaluate a low-cost, color-based sensor for sorting fruits. The results obtained from the current study showed the possibility of using the automatic RGB color-based sensor (TCS3200) effectively for sorting and grading apple fruit varieties. The light intensity and the conveyor belt speed sensors used in the measuring circuit have been tested and calibrated, which showed a high performance at the various light intensity and speed values. The calibrations of RGB color-based sensors, which are the backbone of the control circuit, control the operation of the stepper motors and push arms. It is found that the RGB values of the same object vary with light intensity, conveyor belt color, and sensor height. To maintain optimum operation and maximum performance of the purposed ACSS, several design criteria should be taken into consideration as follows.
The color of the conveyor belt should be black since black color does not appear naturally in apples and the signal produced by the black conveyor belt is as large as possible compared to white and blue colors, which makes it easier to distinguish between the signal produced by the apples and the other signals produced by the black conveyor belt. The results further demonstrated that the optimum height of the color sensor should be adjusted between 15 and 30 mm to increase the ability to distinguish the output signals. Moreover, 25 lux was found to be the optimum light intensity, and any increase in light intensity leads to a decrease in the differences of the resulting signal.
It can also be considered that using an RGB color sensor eliminates the need for a computer or a mobile application for image processing, which minimizes ownership and operating costs for that technology. Finally, the cost of this system is low compared to other automatic systems used in fruit sorting and grading.
Further advancements can be made using different types of fruits and different engineering parameters to achieve high speed and high accuracy for sorting and grading fruits.

Author Contributions

Conceptualization, A.E.E.; methodology, A.E.E., S.E., A.H.E. and A.H.M.; software, A.E.E., A.A.T., A.M.O. and Y.S.A.M.; validation, A.E.E., A.A.T., A.M.O., S.E., A.H.E. and Y.S.A.M.; formal analysis, A.E.E., A.A.T., A.M.O., A.H.E., S.E. and Y.S.A.M.; investigation, S.E., A.H.E. and A.H.M.; resources, A.E.E.; Y.S.A.M. and A.A.T., data curation, A.E.E., A.A.T., A.H.E., S.E. and Y.S.A.M.; writing—original draft preparation, A.E.E., A.A.T. and A.M.O.; writing—review and editing, A.H.E., S.E. and A.H.M.; visualization, A.E.E. and A.A.T.; supervision, A.H.E. and A.H.M.; project administration, A.E.E., A.A.T., Y.S.A.M. and A.H.M.; funding acquisition, Y.S.A.M. All authors have read and agreed to the published version of the manuscript.

Funding

King Khalid University for funding the APC for this work through a large group research project under grant number (RGP 2/386/44).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data are presented within the article.

Acknowledgments

The authors would like to acknowledge the Deanship of Scientific Research at King Khalid University for funding the APC for this work through a large group research project under grant number (RGP 2/386/44).

Conflicts of Interest

The authors declare no conflict of interest.

Nomenclature

Nomenclature
f o Output frequency
f D Output frequency for dark condition
R e Device responsivity for a given wavelength of light given in kHz/(mW/cm2)
E e Incident irradiance in mW/cm2
T Period time
HHue angle (°)
C*Chroma
a*Stand for the tones between redness (+) and greenness (−)
b*Stand for the tones between yellowness (+) and blueness (−)
RPearson correlation
NNumber of data points
X,   X i , X o Observed data
Y, Y i , Y o Measured data
Abbreviations
F and VFruits and vegetables
ITInformation technology
PRTPicture recognition technology
SVMSupport vector machines
CNNConvolutional neural network
RGBRed, green, and blue
ACSSAutomatic color sorting system
MECMeasuring electronic circuit
IDEIntegrated development environment
RMSERoot mean square error
IOAIndex of agreement

References

  1. Bhargava, A.; Bansal, A. Fruits and vegetables quality evaluation using computer vision: A review. J. King Saud Univ. Inf. Sci. 2021, 33, 243–257. [Google Scholar]
  2. Narendra, V.G.; Hareesh, K.S. Prospects of computer vision automated grading and sorting systems in agricultural and food products for quality evaluation. Int. J. Comput. Appl. 2010, 1, 1–9. [Google Scholar]
  3. Francis, F.J. Color quality evaluation of horticultural crops. HortScience 1980, 15, 58–59. [Google Scholar] [CrossRef]
  4. Sari, M.I.; Fajar, R.; Gunawan, T.; Handayani, R. The Use of image processing and sensor in tomato sorting machine by color, Size, and weight. JOIV Int. J. Inform. Vis. 2022, 6, 244–249. [Google Scholar] [CrossRef]
  5. Yang, M.; Kumar, P.; Bhola, J.; Shabaz, M. Development of image recognition software based on artificial intelligence algorithm for the efficient sorting of apple fruit. Int. J. Syst. Assur. Eng. Manag. 2022, 13, 322–330. [Google Scholar] [CrossRef]
  6. Brosnan, T.; Sun, D.W. Improving quality inspection of food products by computer vision—A review. J. Food Eng. 2004, 61, 3–16. [Google Scholar] [CrossRef]
  7. Breathnach, P. Globalisation, information technology and the emergence of niche transnational cities: The growth of the call centre sector in Dublin. Geoforum 2000, 31, 477–485. [Google Scholar] [CrossRef]
  8. Prabha Singh, S. The role of technology in the emergence of the information society in India. Electron. Libr. 2005, 23, 678–690. [Google Scholar] [CrossRef]
  9. Li, T.; Jin, Q.; Huang, B.; Li, C.; Huang, M. Cargo pallets real-time 3D positioning method based on computer vision. J. Eng. 2019, 2019, 8551–8555. [Google Scholar] [CrossRef]
  10. Sahitya, G.; Balaji, N.; Naidu, C.D.; Abinaya, S. Designing a wireless sensor network for precision agriculture using zigbee. In Proceedings of the 2017 IEEE 7th International Advance Computing Conference (IACC), Hyderabad, India, 5–7 January 2017; pp. 287–291. [Google Scholar]
  11. Tripathi, M.K.; Maktedar, D.D. A role of computer vision in fruits and vegetables among various horticulture products of agriculture fields: A survey. Inf. Process. Agric. 2020, 2, 183–203. [Google Scholar] [CrossRef]
  12. Yan, B.; Wu, X.; Ye, B.; Zhang, Y. Three-level supply chain coordination of fresh agricultural products in the Internet of Things. Ind. Manag. Data Syst. 2017, 117, 1842–1865. [Google Scholar] [CrossRef]
  13. Nukala, R.; Panduru, K.; Shields, A.; Riordan, D.; Doody, P.; Walsh, J. Internet of Things: A review from ‘Farm to Fork’. In Proceedings of the 2016 27th Irish Signals and Systems Conference (ISSC), Londonderry, UK, 21–22 June 2016; pp. 1–6. [Google Scholar]
  14. Zhou, Y.; Sharma, A.; Masud, M.; Gaba, G.S.; Dhiman, G.; Ghafoor, K.Z.; AlZain, M.A. Urban rain flood ecosystem design planning and feasibility study for the enrichment of smart cities. Sustainability 2021, 13, 5205. [Google Scholar] [CrossRef]
  15. Ren, X.; Li, C.; Ma, X.; Chen, F.; Wang, H.; Sharma, A.; Gaba, G.S.; Masud, M. Design of Multi-Information Fusion Based Intelligent Electrical Fire Detection System for Green Buildings. Sustainability 2021, 13, 3405. [Google Scholar] [CrossRef]
  16. Sodhi, G.K.; Kaur, S.; Gaba, G.S.; Kansal, L.; Sharma, A.; Dhiman, G. COVID-19. Role of robotics, artificial intelligence and machine learning during the pandemic. Curr. Med. Imaging 2022, 18, 124–134. [Google Scholar] [CrossRef] [PubMed]
  17. Li, Q.; Xiong, Y.; Yang, W.; Han, J.; Liang, H. Study on color analyzer based on the multiplexing of TCS3200 color sensor and microcontroller. Int. J. Hybrid Inf. Technol. 2014, 7, 167–174. [Google Scholar] [CrossRef]
  18. Xiong, N.N.; Shen, Y.; Yang, K.; Lee, C.; Wu, C. Color sensors and their applications based on real-time color image segmentation for cyber physical systems. EURASIP J. Image Video Process. 2018, 2018, 23. [Google Scholar] [CrossRef]
  19. Gomaa, M.; Elhifnawy, H. Automated unsupervised change detection technique from RGB color image. IOP Conf. Ser. Mater. Sci. Eng. 2019, 610, 12046. [Google Scholar] [CrossRef]
  20. Juliano, A.; Hendrawan, A.H.; Ritzkal, R. Information system prototyping of strawberry maturity stages using arduino uno and TCS3200. J. Robot. Control 2020, 1, 86–91. [Google Scholar] [CrossRef]
  21. Pourdarbani, R.; Ghassemzadeh, H.R.; Seyedarabi, H.; Nahandi, F.Z.; Vahed, M.M. Study on an automatic sorting system for Date fruits. J. Saudi Soc. Agric. Sci. 2015, 14, 83–90. [Google Scholar] [CrossRef]
  22. Ansari, M.A.; Dixit, M.; Kurchaniya, D.; Johari, P.K. An effective approach to an image retrieval using SVM classifier. Int. J Comput. Sci. Eng. 2017, 5, 62–72. [Google Scholar]
  23. Prasetyo, D.; Bimantaka, R.D.M. Identification of red dragon fruit using backpropagation method based on android. Int. J. Appl. Bus. Inf. Syst. 2018, 2, 40–45. [Google Scholar]
  24. Behera, S.k.; Rath, A.K.; Sethy, P.K. Fruit recognition using support vector machine based on deep features. Karbala Int. J. Mod. Sci. 2020, 6, 16. [Google Scholar] [CrossRef]
  25. Risdin, F.; Mondal, P.K.; Hassan, K.M. Convolutional neural networks (CNN) for detecting fruit information using machine learning techniques. IOSR J. Comput. Eng. 2020, 22, 1–13. [Google Scholar]
  26. Olorunfemi, B.O.; Nwulu, N.I.; Ogbolumani, O.A. Solar panel surface dirt detection and removal based on arduino color recognition. MethodsX 2023, 10, 101967. [Google Scholar] [CrossRef]
  27. Amhani, A.; Iqbal, Z. Automated color sensor system using LDR and RGB leds controlled by Arduino. ARPN J. Eng. Appl. Sci. 2017, 12, 5532–5537. [Google Scholar]
  28. Yurish, S.Y. Intelligent opto sensors’ interfacing based on universal frequency-to-digital converter. Sens. Transducers 2005, 56, 326–334. [Google Scholar]
  29. Arduino. Language Reference. 2023. Available online: https://www.arduino.cc/reference/en (accessed on 10 March 2023).
  30. Qi, F.-Q.; Jing, X.-D.; Zhao, S.-Q. Design of stepping motor control system based on AT89C51 microcontroller. Procedia Eng. 2011, 15, 2276–2280. [Google Scholar]
  31. Huynh, Q.K.; Nguyen, C.N.; Tran, N.P.L.; Vo, N.H.P.; Huynh, T.T.; Nguyen, V.C. Evaluating the optimal working parameters of the color sensor TCS3200 in the fresh chili destemming system. Can Tho Univ. J. Sci. 2022, 14, 35–42. [Google Scholar] [CrossRef]
  32. Mohammed, M.; Riad, K.; Alqahtani, N. Design of a smart IoT-based control system for remotely managing cold storage facilities. Sensors 2022, 22, 4680. [Google Scholar] [CrossRef]
  33. Sofu, M.M.; Er, O.; Kayacan, M.C.; Cetişli, B. Design of an automatic apple sorting system using machine vision. Comput. Electron. Agric. 2016, 127, 395–405. [Google Scholar] [CrossRef]
  34. Lammertyn, J.; Peirs, A.; De Baerdemaeker, J.; Nicolaı, B. Light penetration properties of NIR radiation in fruit with respect to non-destructive quality assessment. Postharvest Biol. Technol. 2000, 18, 121–132. [Google Scholar] [CrossRef]
  35. Ma, R.; Liao, N.; Yan, P.; Shinomori, K. Categorical color constancy under RGB-LED light sources. Color Res. Appl. 2018, 43, 655–674. [Google Scholar] [CrossRef]
  36. Afrisal, H.; Faris, M.; Utomo, G.; Grezelda, L.; Soesanti, I.; Andri, F.M. Portable smart sorting and grading machine for fruits using computer vision. In Proceedings of the 2013 International Conference on Computer Control, Informatics and Its Applications, Jakarta, Indonesia, 19–21 November 2013. [Google Scholar]
Figure 1. Isometric view of the proposed ACSS.
Figure 1. Isometric view of the proposed ACSS.
Agriculture 13 01824 g001
Figure 2. The hardware component of the proposed ACSS.
Figure 2. The hardware component of the proposed ACSS.
Agriculture 13 01824 g002
Figure 3. Detailed views and main dimensions of the purposed ACSS, all dimensions in cm.
Figure 3. Detailed views and main dimensions of the purposed ACSS, all dimensions in cm.
Agriculture 13 01824 g003
Figure 4. The main electronic component of the proposed ACSS.
Figure 4. The main electronic component of the proposed ACSS.
Agriculture 13 01824 g004
Figure 5. A schematic diagram of RGB color-based sensors and stepper motors on a control circuit with successful electrical rule checks for the current meters and power control.
Figure 5. A schematic diagram of RGB color-based sensors and stepper motors on a control circuit with successful electrical rule checks for the current meters and power control.
Agriculture 13 01824 g005
Figure 6. The color sensor received mirrored light reflected by an object.
Figure 6. The color sensor received mirrored light reflected by an object.
Agriculture 13 01824 g006
Figure 7. Proposed approach for detection and grading of multiple F and V.
Figure 7. Proposed approach for detection and grading of multiple F and V.
Agriculture 13 01824 g007
Figure 8. Flowchart of the color detection and sorting process.
Figure 8. Flowchart of the color detection and sorting process.
Agriculture 13 01824 g008
Figure 9. Calibration results for the light intensity sensor.
Figure 9. Calibration results for the light intensity sensor.
Agriculture 13 01824 g009
Figure 10. Calibration results for conveyor belt speed sensor.
Figure 10. Calibration results for conveyor belt speed sensor.
Agriculture 13 01824 g010
Figure 11. Calibration results of RGB color sensors using different colored papers: (a) white, (b) red, (c) yellow, (d) blue, (e) green, and (f) black.
Figure 11. Calibration results of RGB color sensors using different colored papers: (a) white, (b) red, (c) yellow, (d) blue, (e) green, and (f) black.
Agriculture 13 01824 g011
Figure 12. Effects of the sensor height on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Figure 12. Effects of the sensor height on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Agriculture 13 01824 g012
Figure 13. Effects of the conveyor belt color on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Figure 13. Effects of the conveyor belt color on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Agriculture 13 01824 g013
Figure 14. Effects of light intensity on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Figure 14. Effects of light intensity on the RGB channels of three different apple fruit varieties, for the (a) red channel, (b) green channel, and (c) blue channel.
Agriculture 13 01824 g014
Table 1. List of components of MEC.
Table 1. List of components of MEC.
No.Component
1.Arduino Uno (ATmega328P microcontroller, 5 Vdc operating voltage)
2.LDR sensor module (light intensity sensor) (maximum voltage 150 V, peak wavelength 600 nm and dark resistance 0.5 MΩ, light resistance (10 lux) 50–100 KΩ, power dissipation 100 mW, response time 20–30 ms)
3.Speed sensor (single output signal, 5 Vdc operating voltage).
Table 2. The photodiodes connect correspondingly.
Table 2. The photodiodes connect correspondingly.
S2 PinS3 PinPhotodiode TypeOutput Frequency Scaling
LLRedPower down
LHBlue2%
HLClear (no filter)20%
HHGreen100%
Table 3. Statistical metrics values of R, R2, RMSE, and IOA derived from comparing the measured values of the light intensity sensor (LDR kit) and conveyor belt speed sensor (HC-89).
Table 3. Statistical metrics values of R, R2, RMSE, and IOA derived from comparing the measured values of the light intensity sensor (LDR kit) and conveyor belt speed sensor (HC-89).
ParametersStatistical Metrics
RR2RMSEIOA
Conveyor belt speed0.9950.9900.1430.997
Light intensity0.9960.99226.980.986
Table 4. The maximum and minimum RGB values obtained from the calibration of RGB color-based sensors using different colored papers.
Table 4. The maximum and minimum RGB values obtained from the calibration of RGB color-based sensors using different colored papers.
RGB ValueWhiteRedYellowBlueGreenBlack
RMax152318151532323922497384
Mini137316421479303920515843
GMax142826551442206318225540
Mini121622821400199216914276
BMax122320472056166020786062
Mini109518532002159119055115
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Elwakeel, A.E.; Mazrou, Y.S.A.; Tantawy, A.A.; Okasha, A.M.; Elmetwalli, A.H.; Elsayed, S.; Makhlouf, A.H. Designing, Optimizing, and Validating a Low-Cost, Multi-Purpose, Automatic System-Based RGB Color Sensor for Sorting Fruits. Agriculture 2023, 13, 1824. https://doi.org/10.3390/agriculture13091824

AMA Style

Elwakeel AE, Mazrou YSA, Tantawy AA, Okasha AM, Elmetwalli AH, Elsayed S, Makhlouf AH. Designing, Optimizing, and Validating a Low-Cost, Multi-Purpose, Automatic System-Based RGB Color Sensor for Sorting Fruits. Agriculture. 2023; 13(9):1824. https://doi.org/10.3390/agriculture13091824

Chicago/Turabian Style

Elwakeel, Abdallah E., Yasser S. A. Mazrou, Aml A. Tantawy, Abdelaziz M. Okasha, Adel H. Elmetwalli, Salah Elsayed, and Abeer H. Makhlouf. 2023. "Designing, Optimizing, and Validating a Low-Cost, Multi-Purpose, Automatic System-Based RGB Color Sensor for Sorting Fruits" Agriculture 13, no. 9: 1824. https://doi.org/10.3390/agriculture13091824

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop