1. Introduction
Palm oil is the most productive vegetable oil in the world. It accounts for about 30% of the total production of vegetable oils and fats in the world. Palm oil has proven to be useful and is used in various products such as soap, margarine, cosmetics, and surfactants [
1].
Currently, traditional human graders play an important role in distinguishing oil palm ripeness in plantations. During the pre-harvesting stage, human graders evaluate the oil palm fresh fruit bunches (FFBs) based on the number of detached fruits that fell to the ground and the surface color of oil palm FFBs to determine their ripeness stage [
2]. For the post-harvesting stage, a human expert inspects the color of the oil palm FFBs’ surface to reject unripe and over-ripe FFBs so that they do not proceed to the oil extraction process [
3]. According to Siregar [
4], oil extraction rate (OER) decreases by 0.13% if unripe fruits are present, up to 1% during oil extraction. Furthermore, the National Economic Advisory Council (NEAC) in Malaysia reported that if OER rises by 3%, it brings economic profit of RM 8 billion [
5]. In other words, if there is a 0.13% drop in OER due to misjudgment, it will cause the equivalent of RM 340 million in economic loss. Therefore, it is essential to use an automated detection classification method to prevent losses and increase productivity.
Several detection methods have been introduced, and computer vision is one of the main methods that are widely used for this application. Other than red-green-blue (RGB), color evaluation methods are also used such as hue, saturation and intensity (HSI) [
6]. Research conducted by Cherie et al. [
7] under three spectrum regions tested ultraviolet, visible, and near infrared to evaluate the oil palm FFB harvesting decision. Besides that, Hazir et al. [
8] introduced a multi-parameter fluorescence sensor to investigate the potential of flavonoids and anthocyanins parameters as a predictor to classify the degree of oil palm FFB ripeness.
Conventionally, the computer vision evaluation is paired with the automated artificial intelligence system by computer to accurately grade the oil palm fruit ripeness stages. The methods mentioned are based on image processing, which requires complicated procedures and strictly controlled conditions for measurement, such as no environmental light and constant settings of measurement devices. Therefore, they require image color correction calibration, which uses an image processing software and manually picks up pixel values on a color chart that is placed next to a target object in an image.
There are also various microwave sensors that have been tested for the same purpose, as Ahmad et al. [
9] suggested, such as coplanar and microstrip sensors. The measurement system was tested with a microstrip ring resonator that operated between 2.2 and 3 GHz on oil palm FFB seeds and fruits with various maturity stages. However, the fruit sample preparation is very time-consuming, since the oil palm FFB mesocarp need to be separated from its seeds, and the mesocarp is mushed into semi-solid sample.
Furthermore, the new inductive method was also proposed with a different structure: circular, single, dual, and triple coil structure [
10]. This sensor was developed based on the fact that the resonant frequency of the sensor increases as the fruit ripens and inversely, the capacitance value decreases as the fruit ripens [
11]. However, these inductive methods are time-consuming and require specific equipment for measurement and analysis.
For this study, a fruit battery is proposed as new detection method together with computer vision. The fruit battery method measures the load resistance voltage when copper and zinc electrodes are embedded in the oil palm fruit. The value of load resistance voltage is different between unripe and ripe fruit principally due to the presence of ions in the form of fruit moisture content, which serves as the electrolyte. The advantage of the fruit battery method is that it is possible to construct a simple and inexpensive detection device, since ripeness can be determined by simply measuring the load resistance voltage. In a previous study, suitable electrode conditions for the fruit battery method for depth and distance were discovered. It was concluded that 3 mm depth and 2 mm distance between electrodes are the best parameters when evaluating load resistance voltage results and shape of oil palm fruit [
12].
The objective of this study was to determine the best load resistance value that produces the highest sensitivity for the fruit battery method when tested with fruit sample at different maturity stages. Besides that, this study also aimed to evaluate the accuracy score for fruit battery and compare it with the computer vision method’s accuracy score applied to the same fruit sample.
3. Methodology
A total of 52 fruit samples were collected and used for this experiment. Among 52 fruit samples collected, 21 ripe, 15 under ripe, and 16 unripe fruits were identified according to their moisture content from
Table 1. Photographs were taken of all samples using an augmented reality (AR) marker, and they were then tested with the fruit battery method. Lastly, moisture content was determined using the infrared moisture analyzer.
Firstly, load resistance determination was tested using only ripe and unripe fruits. Twenty-one ripe and 16 unripe fruits were tested with load resistance ranging from 10 Ω to 1 MΩ to obtain the differences between ripe and unripe samples and observe the best load resistance to be chosen for moisture content resolution analysis. Moisture content resolution analysis used all 52 fruits collected to identify the best load resistance that produced the highest sensitivity. The best load resistance results were used for accuracy score evaluation together with the computer vision.
For the computer vision method used in this study, we used an automatic and low-cost color correction method using augmented reality (AR) technology [
13] and classified samples with the machine learning algorithm. The proposed method requires no strict calibration and adjustment at the measurement stage and automatically picks up pixel values of a color chart with an AR marker based on the relative position from the marker. For the present study, the accuracy scores obtained were saturated between 80% and 90%. This is due to the FFB surface color that exhibits similar color distinctiveness even though their ripeness stages are not distinct. Therefore, in order to improve its accuracy, an add-on feature of color feature identification was used simultaneously to increase the accuracy of classifying oil palm ripeness stages.
The following subsections explain further the computer vision and fruit battery methods applied in this study.
3.1. Basic Concept of Fruit Battery Method
This paper proposes the fruit battery method to distinguish oil palm fruit ripeness by utilizing the principle of the fruit battery.
Figure 2a shows a schematic diagram illustrating the principle of a fruit battery using an oil palm fruit. Equations (1) and (2) express the chemical equation of the fruit battery. A fruit battery basically generates electromotive force when two electrodes with different standard electrode potential are pierced through the fruit surface.
Figure 2a shows that when a copper and a zinc electrode pierce through the fruit, the zinc atom undergoes oxidation reaction where it loses an electron, since zinc is located higher in electrochemical series than copper. The copper electrode accepts the electron from the zinc electrode, and the electron is combined with a positive hydrogen ion from the fruit, producing a hydrogen molecule as shown in Equations (1) and (2). The movement of an electron generates current flow, thus producing electricity and behaving like a battery, hence the name fruit battery.
Figure 2b shows an equivalent circuit of a fruit battery. The equivalent circuit of the fruit battery can be expressed by an electromotive force
Vi (V) and an internal resistance
Ri (Ω). Thus, the ripe and unripe oil palm fruit internal resistance is high or low depending on the fruit’s moisture content. The differences in internal resistance causes the load resistance voltage
VL (V) differences, as shown in Equation (3). The fruit battery method aims to detect the difference in electrolyte load resistance voltage between unripe and ripe fruit.
3.2. Selecting Load Resistance Value
To determine a suitable load resistance for distinguishing oil palm ripeness stages for the fruit battery method, two experiments were conducted. The first experiment aimed to calculate the differences of the load resistance voltage. The load resistance voltage differences show the behavior of load resistance with oil palm fruit from different maturity stages with different moisture content. The second experiment aimed to derive the resolution of the estimated moisture content for the field test. The resolution of (A/D) converter and the value of slope of regression formula between the load resistance voltage and moisture content were derived to determine the moisture content resolution.
3.3. The Changing Rate of the Load Resistance Voltage
Figure 3 and
Table 2 show the experimental setup schematic and experimental condition, respectively. The load resistance voltage was measured using a digital multimeter Pro’s Kit, 3PK-600T (Prokit’s Industries, New Taipei City, Taiwan) when electrodes were embedded into oil palm fruit, as shown in
Figure 3. Zinc and copper were used as the electrode, and the dimensions of the electrodes were 16 mm long, 6 mm wide, and 0.5 mm thick. As determined from a previous study, the best distance between the electrodes is 2 mm with 3 mm depth [
12]. The total sample used was 37 fruits, where 21 and 16 fruits were identified as ripe and unripe fruit, respectively. In those conditions, the load resistance R
L (Ω) tested varied from 10 Ω to 1 MΩ.
In order to determine the value of load resistance that can effectively differentiate the value between unripe fruit and ripe fruit, the average load resistance voltage of both unripe and ripe fruit was calculated. The average load resistance voltage was calculated in order to determine the value of load resistance voltage that differentiates the value between unripe and ripe sample. Then, load resistance voltage differences percentage of the load resistance voltage between unripe and ripe fruit was calculated using Equation (4):
where
dVL is the load resistance voltage difference between unripe and ripe,
VLRipe is the ripe fruit’s load resistance voltage, and
VLUnripe is the unripe fruit’s load resistance voltage. Large |
dVL| means larger resolution, and this improves the load resistance sensitivity to distinguish the differences between ripe and unripe fruits.
3.4. Resolution of Estimated Moisture Content
To derive the resolution of the estimated moisture content, the slope value of regression formula was determined using least squares method. A total of 52 oil palm fruit samples were used in this experiment. The best value of load resistance that was determined from a previous experiment was selected for this experiment.
Then, a scatter plot between moisture content and the load resistance voltage was plotted, and the regression formula between them was derived. Next, the values obtained were compared to each other and evaluated. The load resistance voltage was measured three times and the results were averaged.
3.5. Computer Vision
For computer vision, a color chart and an oil palm fruit sample were taken together using a camera with a pixel resolution of 3264 × 2448 on a smartphone iPhone 5S (Apple Inc., Cupertino, CA, USA), as shown in
Figure 4a. At this time, the picture was taken under a fluorescent light, and the color correction was conducted to decrease the influence of variation of photographing condition by using the color chart [
14]. The color chart with AR marker was used to automatically correct the color condition of images.
Figure 4b shows the AR marker based on color chart that has 16 color chips for color correction. The color chips are located around the marker, and each position of a color chip was automatically identified when the marker was detected. The proposed method requires a reference color chart image that is taken in the ideal lighting condition. Target images for color correction are images that include a target object, such as oil palm fruit and the AR color chart.
In the correction process, a transformation matrix (regression coefficient matrix) in Equation (5) was calculated by linear multiple regression analysis as the pixel values of the color chips were closest when using 16 RGB values from the color chips on the reference image and the target image. The color of the target image was changed by multiplying the transformation matrix
by the all pixel matrix
as shown in Equation (5). The advantages of the proposed color correction method are a reduction in the influence of the environmental lighting and a less complicated data acquisition procedure.
: A corrected pixel value;
: An original pixel value;
: A transformation matrix.
After taking the picture, the average
RGB value of oil palm fruit was extracted, as shown in
Figure 5. To extract the average
RGB value, the picture taken was imported, and the background was changed to black. In the background removal, a pixel value was set to 0 when the condition (
R-value > 90,
G-value > 90 and
B-value > 90) is satisfied. Then, the average
RGB value of the oil palm was calculated using Equation (6). The color feature R
ave/G
ave extraction was done by using Numpy, a numerical calculation library in Python.
We used the ratio R/G because we set the reference wavelength to increase the stability. In the field of remote sensing and plant physiology, the ratio of light intensity of wavelengths is often used to make vegetation indices such as normalized difference vegetation index (NDVI) [
15] and green normalized difference vegetation index (GNDVI) [
16].
3.6. Accuracy Scores
The accuracy score for three stages of ripeness was derived by support vector machine (SVM). Accuracy score was used as a metric to evaluate classification model in this study.
Table 3 shows the experimental condition setup for SVM. From
Table 3, the classification by SVM was performed with three features, that is: Color, R
ave/G
ave; fruit battery,
VL; and combined computer vision with fruit battery method,
VL-R
ave/G
ave.
The hyperparameters for the grid search are C = 1, 10, 100; Gamma = 1, 0.1, 0.01; Kernel = Linear, rbf; and the number of divisions of k-fold cross validation are 8. Hence, the combination produces 18 sets of accuracy scores where each feature is compared for the best score.