Next Article in Journal
Climate Change, Biased Technological Advances and Agricultural TFP: Empirical Evidence from China
Previous Article in Journal
Effect of Land Use Type on Soil Moisture Dynamics in the Sloping Lands of the Black Soil (Mollisols) Region of Northeast China
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Smartphone Contact Imaging and 1-D CNN for Leaf Chlorophyll Estimation in Agriculture

1
Faculty of Computer Technology, Assam down town University, Guwahati 781026, India
2
Department of Electrical Engineering, University of North Florida, Jacksonville, FL 32224, USA
*
Author to whom correspondence should be addressed.
Agriculture 2024, 14(8), 1262; https://doi.org/10.3390/agriculture14081262
Submission received: 25 June 2024 / Revised: 16 July 2024 / Accepted: 30 July 2024 / Published: 31 July 2024
(This article belongs to the Section Digital Agriculture)

Abstract

:
Traditional leaf chlorophyll estimation using Soil Plant Analysis Development (SPAD) devices and spectrophotometers is a high-cost mechanism in agriculture. Recently, research on chlorophyll estimation using leaf camera images and machine learning has been seen. However, these techniques use self-defined image color combinations where the system performance varies, and the potential utility has not been well explored. This paper proposes a new method that combines an improved contact imaging technique, the images’ original color parameters, and a 1-D Convolutional Neural Network (CNN) specifically for tea leaves’ chlorophyll estimation. This method utilizes a smartphone and flashlight to capture tea leaf contact images at multiple locations on the front and backside of the leaves. It extracts 12 different original color features, such as the mean of RGB, the standard deviation of RGB and HSV, kurtosis, skewness, and variance from images for 1-D CNN input. We captured 15,000 contact images of tea leaves, collected from different tea gardens across Assam, India to create a dataset. SPAD chlorophyll measurements of the leaves are included as true values. Other models based on Linear Regression (LR), Artificial Neural Networks (ANN), Support Vector Regression (SVR), and K-Nearest Neighbor (KNN) were also trained, evaluated, and tested. The 1-D CNN outperformed them with a Mean Absolute Error (MAE) of 2.96, Mean Square Error (MSE) of 15.4, Root Mean Square Error (RMSE) of 3.92, and Coefficient of Regression ( R 2 ) of 0.82. These results show that the method is a digital replication of the traditional method, while also being non-destructive, affordable, less prone to performance variations, and simple to utilize for sustainable agriculture.

1. Introduction

Assam is known for its tea farming. The state produces the highest amount of tea in India. Nowadays, tea farmers use different computer-oriented modern technologies such as sensors, the Internet of Things (IoT), and machine learning to improve tea production. But, in the case of the chlorophyll estimation of tea leaves, farmers use many traditional and mechanical methods, and some digital mechanisms such as a spectrophotometer, Soil Plant Analysis Development (SPAD) meter, and AtLeaf meter. The spectrophotometer is already reported as a destructive method [1], which takes more time and is costly for the entire process of leaf chlorophyll estimation. SPAD and AtLeaf are non-destructive methods that take less time than spectrophotometer-based methods. However, researchers noted these devices and their processes as being high-cost mechanisms [1,2]. To reduce the cost and time, researchers are focusing on developing low-cost techniques to estimate the value of leaf chlorophyll.
Previous research on chlorophyll estimation using computer vision-based techniques showed the relation between chlorophyll and the actual color of the leaf [1,2,3,4]. In most of the previous research, the researchers extracted the mean value of Red, Green, and Blue (RGB), and their different color combinations, such as G − R, G + R, (G − R)/(G + R), R − B, R + B, etc., from the leaf images [4,5,6]. Different image-capturing devices such as a smartphone [1,3], digital cameras [5,7,8], and remote sensing devices [9] were employed to capture leaf images. In 2020, Sonobe et al. [10] used hyperspectral imaging to estimate tea leaf chlorophyll value. In recent advancements [11], the AgriQ, a low-cost and open-source unmanned aerial system, has also demonstrated the effectiveness of using dual-spectrum cameras to compute vegetation indices from visible and near-infrared data, providing detailed information for farmers and achieving outstanding results compared to commercial systems. Again, Agriculture drones offer clear advantages over other monitoring methods, but for large-scale areas, deploying a fleet of drones can significantly reduce the total required mission time and enhance operational efficiency [12]. Among all, image capturing using a smartphone has already been recognized as a low-cost image-capturing device [1], so in this study, the use of smartphone is explored and reported.
Researchers in the past focused on the extraction of RGB or Hue, Saturation, and Value (HSV) colors from the leaf images and formed the different Color Indices (CI) such as H − S, G − R, G + R, G/B, G/R, VI = (G − R)/(G + R), (R − B)/(R + B), (R − B)/(R + B), R + G + B, R − B, R + B, R + G, R/(R + G + B), G/(R + G + B), B/(R + G + B), R + G, G + B, H + S, S, H + S + V from the RGB and HSV [1,5,6,7,8]. The researchers correlated the values of the CI with the values of the leaf chlorophyll using ANN, Linear Regression (LR), etc., for the chlorophyll estimation. Barman and Choudhury introduced smartphone-based citrus leaf chlorophyll estimation using ANN and LR [1]. They used contact imaging in their research and achieved a good result with different color indexes in the different categories of leaves. The use of contact imaging techniques for the chlorophyll estimation of corn leaves with LR and ANN was reported by Vesali et al. [3]. They also achieved a good result in the hue color index. The LR technique was also used to estimate the chlorophyll of tomato [5], lettuce [5], broccoli [5], pan betel [6], quinoa [7], amaranth [7], and potato [13], whereas the ANN was used to estimate the chlorophyll of potato [14], citrus [1], etc. The relationship of potato leaves’ chlorophyll with three different color combinations such as G/R, luminosity, and B/L was presented by S.D. Gupta and A.K. Pattanayak [14]. Barman and Choudhury [1] also reported different color combinations for citrus chlorophyll predictions.
Apart from ANN and LR, researchers also used CNN to estimate chlorophyll concentration, although its application was limited. Syariz et al. [15] reported a CNN-based chlorophyll concentration retrieval system using remote sensing images. Prilianti et al. [16] used leaf reflectance spectrum and 1-D CNN to estimate the chlorophyll, carotenoid, and anthocyanin of Jasminum, S. Oleana, G. Pictum, and P. Betel. Sonobe et al. [17] reported that a 1-D CNN can estimate tea leaf chlorophyll based on hyperspectral reflectance.
From the above literature, it is noted that researchers used different types of self-made color combinations to estimate the chlorophyll of the leaves using ANN and LR. These color combinations are user-centric and may vary from research to research. It was found that a few of the previous techniques for chlorophyll estimation using ANN and LR were cost-effective due to the use of low-cost image-capturing devices, such as a smartphone, but the use of the CNN technique was missing. Though a few of the researchers used the CNN technique, they were not cost-effective. In this paper, the first-ever 1-D CNN and modified contact imaging enabling a low-cost, non-destructive technique is put forward to estimate the chlorophyll of the tea leaf. The system was also developed to not rely on any self-made color combinations. The paper forwards the following contributions toward the estimation of chlorophyll content in tea leaves.
  • A non-destructive chlorophyll estimation method using smartphone-based modified contact imaging and 1-D CNN is forwarded.
  • The method does not need to rely on any self-derived color combinations techniques.
  • The results of the 1-D CNN model are compared with the other traditional machine learning models, including the ANN, SVR, LR, and KNN.
  • The 1-D CNN model uses 12 original color combinations of the tea leaf images to estimate the chlorophyll values.
  • The investigated method can be considered a low-cost solution, and it can be considered a digital SPAD in tea leaf chlorophyll estimation.
Again, our approach leverages standardized image-based color features, which are less influenced by leaf thickness and structure variations. The use of controlled imaging conditions (contact imaging) minimizes the impact of external factors like ambient light and temperature, ensuring more reliable data. Furthermore, the consistent sampling process eliminates operator-induced variability. The 1-D CNN model effectively learns spatial relationships between color features, enhancing prediction accuracy. These aspects collectively contribute to the method’s lower susceptibility to variations, making it a robust alternative to SPAD meters for accurate chlorophyll content prediction in tea leaves.

2. Materials and Methods

In this section, tea leaf image acquisition, SPAD estimation, feature extraction, and chlorophyll estimation based on contact imaging are discussed. The flowchart of the overall system is presented in Figure 1.

2.1. Tea Leaf Collection, Image Acquisition, and Dataset Preparation

In this work, we focused on tea plants and tea leaves. A total of 1500 tea leaves were collected from different tea gardens in Assam, India. This included tea estates from different districts like Kokrajhar, Golaghat, Jorhat, Dibrugarh, etc. For the leaf collection, the first 3 or 4 tea leaves (immature) of a branch were identified and plucked as shown in Figure 2. These leaves produce a high-quality tea [18,19]. After plucking, the contact imaging [1] technique was used to snap tea leaf images. The technique introduces a smartphone and a flashlight as special devices in the image-capturing process as shown in Figure 3. A relatively low-cost Android smartphone (Xiaomi Redmi) with a 64-megapixel camera and default image-capturing settings such as Focal-stop = f/2, exposure time of the camera = 1/60 s, ISO sensitivity of the camera = 125, and the focal length of the camera lens = 4 mm was placed above the tea leaf. U. Barman and R.D. Choudhury [1] used an 11 lm flashlight with 0.2 watts as a constant light support in the contact imaging technique. In this work, the constant light of an Eveready digiLED DL46 COMPET with 0.75 watt, 3.2 V, and 120 lm was applied at the bottom of the leaves to obtain brighter tea leaf images compared to the work presented by U. Barman and R.D. Choudhury [1]. The higher luminosity of the light means brighter images and it reduces the blur, inconsistency, and interference from the tea leaf image background. In this method, the natural environmental light condition does not affect the images, as there is no gap between the tea leaf and the smartphone camera (Figure 3). The entire process of image acquisition is also low-cost and simple.
With the help of the contact imaging technique, for a single leaf, images were captured by snapping at five different locations on each side of the leaf (Figure 3), which resulted in a total of ten snaps per leaf, as shown in Figure 4. Thus, a total of 1500 × 10 = 15,000 tea leaf images with a resolution of 4000 × 3000 were snapped during the process and we labeled the images accordingly as shown in Figure 4. Previous researchers considered only one location in their contact imaging [1,3]. In this work, the improved contact imaging technique helps to extract the most accurate color distribution features of the entire tea leaf. Concurrently, the SPAD estimation of tea leaves was also carried out on the front and backside of all the tea leaves to obtain the average distribution of chlorophyll.

2.2. SPAD Chlorophyll Estimation

SPAD is a non-destructive chlorophyll estimation device. The chlorophyll of a leaf can be measured using a spectrophotometer or SPAD. The spectrophotometer is a destructive method, where the leaf is crushed before the chlorophyll estimation. In this research, a SPAD 502 plus (Konica Minolta, Tokyo, Japan) (Figure 5a) was used to estimate tea leaves’ chlorophyll value. Jiang et al. [20] reported the correlation of tomato leaves’ chlorophyll with the SPAD readings. Before measuring the tea leaf chlorophyll, the SPAD 502 plus meter was calibrated using the reading checker, Item No 20011126 (Figure 5b), of the SPAD 502. The calibrated value was 68.3, as shown in Figure 5c. After calibration, the average value of the tea leaf SPAD was recorded to obtain the most accurate SPAD estimation. This was carried out by taking five readings on each side of the tea leaf similar to the image-capturing process shown in Figure 3. Thus, we had a total of 1500 SPAD-averaged readings for the 1500 tea leaves. All the SPAD readings of tea leaves were labeled and saved in a .csv file for further processing. Figure 6 shows the SPAD estimations of the first ten tea leaves. The average chlorophyll value of all the tea leaves was 35.22.

2.3. Tea Leaf Image Pre-Processing and Feature Extraction

Image pre-processing is a technique applied to images before feature extraction. Pre-processing steps are generally applied to remove blur and noise from the image, cropping, and resizing of images, etc. Few researchers resized the images obtained by contact imaging before the feature extraction step. In this work, the goal was not to introduce any image preprocessing to reduce variability and to make the system simple. This was achieved as, here, no image processing techniques were required before feature extraction, since there was no blur and noise in the tea leaf images due to the contact imaging technique. The images were also not cropped or resized, because this may reduce the important features of the leaf image.
As no pre-processing steps were applied to the tea leaf images, images are kept ready for feature extraction. In previous research, investigators specially focused on extracting the mean or standard deviation of RGB and HSV color space values. They combined those color indices to obtain the other color indices. For example, G/R, and B/L were extracted for potato leaf chlorophyll estimation [14]. U. Barman and R.D. Choudhury [1] showed the relation of different derived color indices, such as NRI, VI, R + G + B, etc., with the value of the chlorophyll of various categories of citrus leaves. However, these color indexes are self-derived and may vary from research to research. To overcome the issue of the derived color indexes, 12 different original color features were extracted from the tea leaf images, such as the mean of R, mean of G, mean of B, Standard Deviation (SD) of R, SD of G, SD of B, SD of H, SD of S, SD of V, kurtosis, skewness, and variance using the Equations (1)–(5) and as explained below.
  • The mean RGB color spaces gave the mean value of R, G, and B for all pixels of the tea leaf images.
  • The standard deviation (SD) of RGB and SD of HSV was used to calculate the variation of a pixel of the tea leaf image from the mean of RGB and HSV values.
  • The kurtosis was used to find whether the pixel value was peaked or flat to the normal distribution.
  • The skewness was used to find the color symmetry of an image using the Equation (4). Skewness was used to measure the degree of distortion of the pixel values to judge the image surface.
  • The variance was used to find the spread of an image pixel from its mean.
M e a n : i , j = 0 n 1 i · m i , j
S t a n d a r d D e v i a t i o n : i , j = 0 n 1 ( i m e a n ) 2 m i , j 1 / 2
K u r t o s i s : i , j = 0 n 1 ( i m e a n ) 4 m i , j
S k e w n e s s : i , j = 0 n 1 ( i m e a n ) 3 m i , j
V a r i a n c e : i , j = 0 n 1 m i , j ( i σ ) 2
In Section 2.1, it was mentioned that ten different images were captured from a single leaf collected. After the image capture, the final aforesaid features of a single leaf were extracted by calculating the average of the features from the ten different images of the respective leaf. For example, the final R is the average of the mean of R of the ten different images of a single tea leaf. This process was carried out for the entirety of the 1500 tea leaves, and a dataset of size 1500 × 12 was created for further analysis. This process helps to correlate the average SPAD estimations with the average color features for chlorophyll estimation. In the previous research, the investigators considered only one area of a leaf for image acquisition during contact imaging, and they extracted color features from that leaf area only [1,3]. The different color features along with the chlorophyll value of the tea leaves are shown in Figure 7, Figure 8 and Figure 9. Table 1 presents the average of the twelve color features and the SPAD chlorophyll value in the dataset.

3. Experimental Results

In this section, models used for tea leaf chlorophyll estimation and experimental results are explained. In this work, the 1-D CNN was considered as a primary model to estimate the chlorophyll of the tea leaves. Finally, the results from the 1-D CNN were compared with the results from the other models, such as ANN, LR, SVR, and KNN regression. The size of the dataset is 1500 × 12 . This means that 12 different original color features were there in the dataset for all 1500 tea leaves, along with the SPAD chlorophyll value for the respective tea leaves. The dataset was divided into training, testing, and validation sets, as shown in Table 2. In the dataset, the range of variance was larger than the range of other features. Again, the range of skewness was very low compared to the other ranges of features. To make all the features in standard form, all the color features were standardized using the StandardScaler function of sklearn library in Python. For training and testing purposes, the hyperparameters of the models, such as learning rate, batch size, and epoch, were considered as 0.01, 10, and 10, respectively, as described in the next Section 3.1. The chosen batch size of 10 was selected to balance computational efficiency and the accuracy of error gradients, facilitating faster convergence during training.

3.1. Tea Leaf Chlorophyll Estimation Using 1-D CNN

The 1-D CNN model was implemented in the Python programming platform using the Keras library, and its architecture is explained below in Figure 10.
  • The first 1-D convolution layer was applied to the input size (12, 1) with a 64-layer size and ReLU activation functions. The kernel size was 2.
  • The second layer of the model was also a 1-D CNN convolution layer with a 128-layer size, kernel size of 2, and ReLU activation function.
  • After step 2, the model was flattened, and applied the first hidden layer with 128 hidden neurons with ReLU activation function.
  • The second hidden layer with 64 hidden neurons was introduced in the model with the ReLU activation function.
  • The third hidden layer, followed by the final hidden layer with 32 and 1 hidden neurons, respectively, was applied in the model with ReLU and linear activation function.
As the training process of the 1-D CNN model was very fast, the model was optimized with the ADAM optimizer because it combines two Stochastic Gradient Descent (SGD) algorithms, such as SGD with momentum and RMSProp. The SGD with momentum considers the exponentially weighted average of the SGD, and it helps to converge faster towards minima. Again, the RMSProp of ADAM considers the exponential moving average. D.P. Kingma and J. Ba [21] suggested the default values of ADAM for training purposes depending on the model complexity. Due to the low complexity of the 1-D CNN model and to reduce the overshooting and local minima issue, we considered the hyperparameters mentioned in Table 3 to train our 1-D CNN model. In the training process, a total of 223,809 trainable parameters were evaluated. The overall model summary is presented in Table 4. As mentioned in Table 3, the model was compiled only for 10 epochs, with a batch size of 10. The MSE of the model (loss) and the MAE of the model (metrics) were calculated in all three of the training, validation, and testing setsto evaluate the per-epoch performance of the model (Table 5).
Overall, the performance (Table 5) of the 1-D CNN model was evaluated by calculating the MAE, MSE, RMSE, and R 2 of the model using Equations (6)–(9), where P is the predicted or estimated chlorophyll value and T is the original chlorophyll value. The graphical representations of the loss (MSE), MAE, and R 2 of the 1-D CNN model are presented in Figure 11 and Table 6.
M A E : 1 n i = 1 n ( p i t i )
M S E : 1 n i = 1 n ( p i t i ) 2
R M S E : 1 n i = 1 n ( p i t i ) 2
R 2 : 1 i = 1 n ( y i y ) 2 i = 1 n ( y i y ¯ ) 2

3.2. Tea Leaf Chlorophyll Estimation Using ANN

ANN was already recognized as one of the best models for chlorophyll estimation [1,3,20]. Researchers have already used this technique in different plants. Here, ANN was also used as one of the primary models along with 1-D CNN, and the results are compared to 1-D CNN, LR, SVR, and KNN.
The ANN model used had four hidden layers and one dropout layer. The first hidden layer, followed by a dropout, was added to the model with 512 hidden neurons with a ReLU activation function (Figure 12). The learning process of ANN was very fast due to its simple architecture. To reduce the model for overlearning and the possibility of co-adaption of the 512 hidden neurons in the ANN model, a dropout layer was added in the model with a 20% dropout. Otherwise, many neurons of the first hidden layer may extract the same features from the 12 different input parameters of the tea leaf image, and this can lead to overfitting in the model. In the next step, another two hidden layers were added to the ANN model with 256 and 64 hidden neurons, respectively. The ReLU activation function was selected as an activation function for these two layers. Finally, the output layer was added to the model with a linear activation function and one hidden neuron (Figure 12).
For the ANN model’s training, validation, and testing, the same number of samples was considered, as given in Table 2. The model was optimized with the ADAM optimizer by considering all hyperparameters, as given in Table 3. The model was evaluated using MAE and MSE at each epoch of training, validation, and testing, and a total of 154,497 trainable parameters were evaluated during the training process (Table 7).
As stated in Table 3, the ANN model was also compiled for ten epochs with a batch size of ten. The MAE and the MSE of the ANN model were presented epoch-wise in Table 8 for the training, validation, and testing set. The graphical presentations of the loss (MSE), MAE, and R 2 of the ANN model are presented in Figure 13. Using the Equations (6)–(9), the MAE, MSE, RMSE, and R 2 of the ANN model were calculated, and they are presented in Table 9.

3.3. Tea Leaf Chlorophyll Estimation Using LR, SVR, and KNN

Researchers already recognized the LR as one of the efficient methods to estimate the chlorophyll value of different plants such as tomatoes [20], citrus [1], lettuce [22], quinoa [7], and rice [4]. In this work, regression was performed between the 12 original color values of the tea leaf and the actual chlorophyll of the tea leaf. The performance of the LR model was checked using the training regression score (75.16%) and testing regression score (73.57%) (Table 10). As stated in the Section 3.1 and Section 3.2, the LR was also evaluated by calculating the value of MAE, MSE, RMSE, and R 2 using the Equations (6)–(9) and they are presented in Table 11. The plot of the coefficient of linear regression ( R 2 ) is presented in Figure 14a.
The Support Vector Machine (SVM) is used for classification. Researchers have already reported the application of SVM in agri-informatics, such as soil texture classification [23] and plant disease detection [24]. In this paper, Support Vector Regression (SVR) was used to estimate tea leaf chlorophyll. The SVR was implemented using the sklearn library on Python by considering the hyperparameters as shown in Table 12. The cost of the misclassification parameters (C) for the model was considered as 1 to avoid the overfitting of the model. As shown in Table 12, the SVR model was trained with the linear kernel because only the C parameter was required to optimize in the case of the linear kernel. The high value of C leads to low bias and vice versa. In both cases, it may lead to either overfitting or underfitting the model. So here, the value of C was kept as 1, which was neither high nor low. The training regression score of the SVR model was 73.13%, with a testing regression score of 70.45% (Table 10). Using the different evaluation parameters as shown in Table 11, the performance of the model is reported with an MAE value of 3.82, MSE value of 25.82, RMSE value of 5.08, and R 2 value of 0.70 (Figure 15).
Along with LR and SVR, we also experimented with KNN regression with K = 5. The KNN model is usually used for classification problems, but here we used it for regression analysis. M. Sood and P.K. Singh [25] reported using KNN regression for tea leaf chlorophyll estimation, where the leaf images were captured by a digital camera. Here, the KNN regression was compiled by considering the Euclidian distance and uniform weights between the neighborhood points. As shown in Table 10, the training regression score of the KNN model was 85.78%, with a testing regression score of 80.99%. The MAE of the KNN was 3.01 with an MSE of 16.63, RMSE of 4.07, and R 2 of 0.80 (Table 11 and Figure 13).

4. Discussions

This paper presents a 1-D CNN and contact imaging technique to estimate chlorophyll in tea leaves. In the previous contact imaging research [3,25], researchers captured the leaf images only from one side of the leaf. But here, we captured ten different images from ten different locations of the leaf, where five locations belong to the front and five backside of the leaves. Different self-derived color indexes were used by previous researchers [1,3,20], but we used original colors to estimate the leaf chlorophyll value. From the results, it is found that the performance of the 1-D CNN model was better than the other models for tea leaf chlorophyll estimation. The MAE of the 1-D CNN was 2.96, whereas the MAE values of ANN, LR, SVR, and KNN were 3.04, 3.7, 3.82, and 3.01, respectively. Like our 1-D CNN model, Syariz et al. [15] also used 1-D CNN with different architectural parameters to estimate the chlorophyll of the algae using the remote sensing images with an RMSE value of 2.365 in a single-stage training program. Y. Guo et al. [26] reported an MAE of 2.947 for ANN, 2.460 for SVR, and 2.389 for Random Forest (RF) in the case of RGB features.
Table 6, Table 9 and Table 11 indicate that the 1-D CNN model outperforms ANN, LR, SVR, and KNN. With an RMSE value of 3.92, the 1-D CNN reports 82.3% accuracy, whereas the accuracies of the ANN, LR, SVR, and KNN are 81.8%, 73%, 70%, and 80.9%, respectively. The accuracy of ANN is almost similar to CNN because the working architecture of both models is the same, i.e., both models have the same number of hidden layers but have different hidden neurons for estimation. The convolution layer of the 1-D CNN model is one-dimensional, i.e., linear, whereas the ANN model is also linear. Vesali et al. [3] reported more accuracy in ANN than in the LR. U. Barman and R.D. Choudhury also reported more accuracy than LR in all three categories of the citrus leaf. Also, in this research, the ANN reported a greater accuracy (81.8%) (Table 7) than the accuracies of LR, SVR, and KNN, but it was less than the accuracy of the 1-D CNN (82.3%). This was due to the introduction of a 1-D convolution layer with the normal linear dense model. The epoch-wise performance of the 1-D CNN model using parameter loss and MAE (Table 5 and Figure 6) indicates that the model was neither overfitted nor underfitted due to very little difference among the values of training MAE, testing MAE, and validation MAE of the model. The loss differences were also very less among the sets (Figure 15).
Figure 15 shows that the MSE of the LR was very high (23.82) as compared to the MSE obtained by Barman et al. [25] in the tea leaf, where the images were captured using a digital camera. They reported 80% accuracy in multiple linear regression (MLR), which was more than the accuracy of the ANN (50%), SVR (53.8%), and KNN (70%). The other light conditions might have affected their results as they captured the tea leaf images in natural environmental light conditions using a digital camera. Here, we introduce an improved contact imaging technique where the chances of environmental light effects are significantly low. So, we can claim that the 1-D CNN model is the best model to estimate leaf chlorophyll in the case of contact imaging technique, and the results will not be changed with the change of smartphone due to the use of the original color combination and adequate contact imaging technique. U. Barman and R.D. Choudhury [1] already reported that the default camera properties of most smartphones are the same, and the Android version of the smartphone should not affect the image properties. This research methodology can be applied to any geographical location as original color combinations and contact imaging techniques are used. The change in the original color combination values in a captured image relates to the change in the leaf’s chlorophyll value since the leaf’s color directly relates to the chlorophyll value [1,3]. The comparative analysis of the different methods of contact imaging is presented below in Table 13. Additionally, SPAD (Soil Plant Analysis Development) chlorophyll meters are widely used tools for estimating chlorophyll content in leaves by measuring the transmission of light through the leaf at specific wavelengths. Variations in leaf thickness and internal structure can affect light transmission readings, leading to inconsistencies in SPAD measurements. But our approach leverages standardized image-based color features, which are less influenced by leaf thickness and structure variations. The use of controlled imaging conditions (contact imaging) minimizes the impact of external factors like ambient light and temperature, ensuring more reliable data.
Though this paper focused on the potential use smartphone, the integration of multispectral and hyperspectral imaging, including the Red Edge band, and the use of thermal cameras may enhance the results. The Red Edge band is highly sensitive to chlorophyll content [27], which may improve the precision of our chlorophyll estimation models. Additionally, multispectral and hyperspectral imaging enable the calculation of various vegetation indices and provide detailed spectral information, allowing for the detection of subtle variations in leaf properties. Integrating thermal imaging can also identify plant stress factors such as water deficiency or disease [28], enhancing the robustness and accuracy of our predictions. This combined approach will lead to a more comprehensive and reliable system for monitoring plant health and can be considered for future study.
For a more comprehensive view of our model, the ANN and CNN models were again compiled up to 50 epochs. The extended training of 1-D CNN for 50 epochs shows that the model generally improves over time, with added variability to mimic real-world training behavior. The comparison between epochs 1–10 and epochs 11–50 highlights the importance of monitoring validation and testing metrics to ensure the model maintains good generalization without overfitting. The model parameters provided in below Figure 16 are effective in creating a robust 1-D CNN model for the given data.
The ANN model, extended to 50 epochs, shows a consistent decrease in training, validation, and testing loss/MAE, indicating effective learning and good generalization without significant overfitting Figure 17. The model’s performance improves over time, demonstrating robustness in capturing complex patterns. The regular monitoring of metrics ensures continued model accuracy and prevents overfitting.

5. Conclusions

The proposed system introduces a new mechanism to estimate chlorophyll values in tea leaves. A few original color combinations derived from the tea leaf image, captured by a smartphone using modified contact imaging, are also reported in the paper. The method of image acquisition using modified contact imaging is acceptable and very low cost. The models’ regression scores prove that the original color combination can be used as a color index for chlorophyll estimation. The results of the 1-D CNN model make the claim for it being a good model compared to ANN, LR, SVR, and KNN in chlorophyll estimation. Overall, the model estimation is accurate and acceptable. This method can be an effective solution to overcome the cost issue of SPAD and other traditional chlorophyll estimation methods, such as a spectrophotometer. Considering the 1-D CNN model as the base model and contact imaging technique presented here, this work also further encourages researchers to explore the 2-D and 3-D CNN models for leaf chlorophyll estimation.

Author Contributions

Conceptualization, U.B. and M.J.S.; methodology, U.B.; software, U.B.; validation, M.J.S.; formal analysis, U.B. and M.J.S.; investigation, U.B. and M.J.S.; resources, U.B. and M.J.S.; data curation, U.B.; writing—original draft preparation, U.B.; writing—review and editing, M.J.S.; visualization, M.J.S.; supervision, U.B. and M.J.S.; project administration, M.J.S. All authors have read and agreed to the published version of the manuscript.

Funding

The authors acknowledge the support of anonymous tea farmers from Assam, Northeast India in data collection.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The required data are presented in the manuscript. The full data will be shared upon request.

Acknowledgments

The authors thank the Biomedical Sensors & Systems Lab, University of North Florida, Jacksonville, FL 32224, USA for supporting this research and the article processing charges.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Barman, U.; Choudhury, R. Smartphone image based digital chlorophyll meter to estimate the value of citrus leaves chlorophyll using Linear Regression, LMBP-ANN and SCGBP-ANN. J. King Saud Univ.-Comput. Inf. Sci. 2022, 34, 2938–2950. [Google Scholar] [CrossRef]
  2. Agarwal, A.; Gupta, S. Assessment of spinach seedling health status and chlorophyll content by multivariate data analysis and multiple linear regression of leaf image features. Comput. Electron. Agric. 2018, 152, 281–289. [Google Scholar] [CrossRef]
  3. Vesali, F.; Omid, M.; Kaleita, A.; Mobli, H. Development of an android app to estimate chlorophyll content of corn leaves based on contact imaging. Comput. Electron. Agric. 2015, 116, 211–220. [Google Scholar] [CrossRef]
  4. Mohan, P.; Gupta, S. Intelligent image analysis for retrieval of leaf chlorophyll content of rice from digital images of smartphone under natural light. Photosynthetica 2019, 57, 388–398. [Google Scholar] [CrossRef]
  5. Ali, M.; Al-Ani, A.; Eamus, D.; Tan, D.K.Y. A new image processing based technique to determine chlorophyll in plants. Am.-Eurasian J. Agric. Environ. Sci. 2012, 12, 1323–1328. [Google Scholar]
  6. Dey, A.; Sharma, M.; Meshram, M. An analysis of leaf chlorophyll measurement method using chlorophyll meter and image processing technique. Procedia Comput. Sci. 2016, 85, 286–292. [Google Scholar] [CrossRef]
  7. Riccardi, M.; Mele, G.; Pulvento, C.; Lavini, A.; D’Andria, R.; Jacobsen, S. Non-destructive evaluation of chlorophyll content in quinoa and amaranth leaves by simple and multiple regression analysis of RGB image components. Photosynth. Res. 2014, 120, 263–272. [Google Scholar] [CrossRef]
  8. Suo, X.; Jiang, Y.; Mei, Y.; Li, S.; WANG, K.; Wang, C. Artificial neural network to predict leaf population chlorophyll content from cotton plant images. Agric. Sci. China 2010, 9, 38–45. [Google Scholar] [CrossRef]
  9. Ma, X.; Feng, J.; Guan, H.; Liu, G. Prediction of chlorophyll content in different light areas of apple tree canopies based on the color characteristics of 3D reconstruction. Remote Sens. 2018, 10, 429. [Google Scholar] [CrossRef]
  10. Sonobe, R.; Hirono, Y.; Oi, A. Non-destructive detection of tea leaf chlorophyll content using hyperspectral reflectance and machine learning algorithms. Plants 2020, 9, 368. [Google Scholar] [CrossRef] [PubMed]
  11. Oca, A.; Flores, G. The AgriQ: A low-cost unmanned aerial system for precision agriculture. Expert Syst. Appl. 2021, 182, 115163. [Google Scholar]
  12. Liang, M.; Delahaye, D. Drone fleet deployment strategy for large scale agriculture and forestry surveying. In Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 27–30 October 2019; pp. 4495–4500. [Google Scholar]
  13. Yadav, S.; Ibaraki, Y.; Dutta Gupta, S. Estimation of the chlorophyll content of micropropagated potato plants using RGB based image analysis. Plant Cell Tissue Organ Cult. (PCTOC) 2010, 100, 183–188. [Google Scholar] [CrossRef]
  14. Dutta Gupta, S.; Pattanayak, A. Intelligent image analysis (IIA) using artificial neural network (ANN) for non-invasive estimation of chlorophyll content in micropropagated plants of potato. Vitr. Cell. Dev. Biol.-Plant 2017, 53, 520–526. [Google Scholar] [CrossRef]
  15. Syariz, M.; Lin, C.; Nguyen, M.; Jaelani, L.; Blanco, A. WaterNet: A convolutional neural network for chlorophyll-a concentration retrieval. Remote Sens. 2020, 12, 1966. [Google Scholar] [CrossRef]
  16. Prilianti, K.; Setiyono, E.; Kelana, O.; Brotosudarmo, T. Deep chemometrics for nondestructive photosynthetic pigments prediction using leaf reflectance spectra. Inf. Process. Agric. 2021, 8, 194–204. [Google Scholar] [CrossRef]
  17. Sonobe, R.; Hirono, Y.; Oi, A. Quantifying chlorophyll-a and b content in tea leaves using hyperspectral reflectance and deep learning. Remote Sens. Lett. 2020, 11, 933–942. [Google Scholar] [CrossRef]
  18. Baruah, P. Tea: An Important Constituent of Biodiversity of Assam, Northeast India. Int. J. Agric. Sci. 2019, 11, 8986–8989. [Google Scholar]
  19. Chen, Y.; Chen, S. Localizing plucking points of tea leaves using deep convolutional neural networks. Comput. Electron. Agric. 2020, 171, 105298. [Google Scholar] [CrossRef]
  20. Jiang, C.; Johkan, M.; Hohjo, M.; Tsukagoshi, S.; Maruo, T. A correlation analysis on chlorophyll content and SPAD value in tomato leaves. HortResearch 2017, 71, 37–42. [Google Scholar]
  21. Kingma, D.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  22. Ibrahim, N.; Abd Aziz, S.; Jamaludin, D.; Harith, H. Development of smartphone-based imaging techniques for the estimation of chlorophyll content in lettuce leaves. Food Res. 2021, 5, 33–38. [Google Scholar] [CrossRef]
  23. Barman, U.; Choudhury, R. Soil texture classification using multi class support vector machine. Inf. Process. Agric. 2020, 7, 318–332. [Google Scholar] [CrossRef]
  24. Sood, M.; Singh, P.K. Hybrid system for detection and classification of plant disease using qualitative texture features analysis. Procedia Comput. Sci. 2020, 167, 1056–1065. [Google Scholar]
  25. Barman, U.; Sarmah, A.; Sahu, D.; Barman, G. Estimation of Tea Leaf Chlorophyll Using MLR, ANN, SVR, and KNN in Natural Light Condition. In Proceedings of the International Conference On Computing And Communication Systems: I3CS 2020, NEHU, Shillong, India, 28–30 April 2020; Springer: Berlin/Heidelberg, Germany, 2021; pp. 287–295. [Google Scholar]
  26. Guo, Y.; Yin, G.; Sun, H.; Wang, H.; Chen, S.; Senthilnath, J.; Wang, J.; Fu, Y. Scaling effects on chlorophyll content estimations with RGB camera mounted on a UAV platform using machine-learning methods. Sensors 2020, 20, 5130. [Google Scholar] [CrossRef] [PubMed]
  27. Chang-Hua, J.; Yong-Chao, T.; Xia, Y.; Wei-Xing, C.; Yan, Z.; Hannaway, D. Estimating leaf chlorophyll content using red edge parameters. Pedosphere 2010, 20, 633–644. [Google Scholar]
  28. Pineda, M.; Barón, M.; Pérez-Bueno, M. Thermal imaging for plant stress detection and phenotyping. Remote Sens. 2020, 13, 68. [Google Scholar] [CrossRef]
Figure 1. Flowchart of the proposed method for tea leaf chlorophyll estimation.
Figure 1. Flowchart of the proposed method for tea leaf chlorophyll estimation.
Agriculture 14 01262 g001
Figure 2. Region of consideration for the first (a,b) three or (c,d) four tea leaves for contact imaging.
Figure 2. Region of consideration for the first (a,b) three or (c,d) four tea leaves for contact imaging.
Agriculture 14 01262 g002
Figure 3. Image acquisition process. Five different points on the front and back side of a tea leaf for contact imaging using a smartphone and flashlight.
Figure 3. Image acquisition process. Five different points on the front and back side of a tea leaf for contact imaging using a smartphone and flashlight.
Agriculture 14 01262 g003
Figure 4. Ten different snaps of tea leaf no. 1.
Figure 4. Ten different snaps of tea leaf no. 1.
Agriculture 14 01262 g004
Figure 5. Calibration of (a) SPAD meter using a (b) reading checker, and (c) the calibrated value, 68.3.
Figure 5. Calibration of (a) SPAD meter using a (b) reading checker, and (c) the calibrated value, 68.3.
Agriculture 14 01262 g005
Figure 6. SPAD estimation of first ten tea leaves.
Figure 6. SPAD estimation of first ten tea leaves.
Agriculture 14 01262 g006
Figure 7. Plot of ten color features with the chlorophyll value.
Figure 7. Plot of ten color features with the chlorophyll value.
Agriculture 14 01262 g007
Figure 8. Subplots of the individual color features including the chlorophyll plots.
Figure 8. Subplots of the individual color features including the chlorophyll plots.
Agriculture 14 01262 g008
Figure 9. Additional plots of the color features including chlorophyll distribution.
Figure 9. Additional plots of the color features including chlorophyll distribution.
Agriculture 14 01262 g009
Figure 10. The 1-D CNN model for tea chlorophyll (Chl.) estimation using 12 input features.
Figure 10. The 1-D CNN model for tea chlorophyll (Chl.) estimation using 12 input features.
Agriculture 14 01262 g010
Figure 11. Error plot: (a) MSE and (b) MAE, and (c) R 2 of the 1-D CNN.
Figure 11. Error plot: (a) MSE and (b) MAE, and (c) R 2 of the 1-D CNN.
Agriculture 14 01262 g011
Figure 12. ANN model for tea leaf chlorophyll estimation.
Figure 12. ANN model for tea leaf chlorophyll estimation.
Agriculture 14 01262 g012
Figure 13. Error plot: (a) MSE and (b) MAE, and (c) R 2 of the ANN.
Figure 13. Error plot: (a) MSE and (b) MAE, and (c) R 2 of the ANN.
Agriculture 14 01262 g013
Figure 14. R 2 plot of (a) LR, (b) SVR, and (c) KNN.
Figure 14. R 2 plot of (a) LR, (b) SVR, and (c) KNN.
Agriculture 14 01262 g014
Figure 15. Performance comparison in terms of MAE, MSE, RMSE, and R 2 of 1-D CNN, ANN, LR, SVR, and KNN.
Figure 15. Performance comparison in terms of MAE, MSE, RMSE, and R 2 of 1-D CNN, ANN, LR, SVR, and KNN.
Agriculture 14 01262 g015
Figure 16. Loss and MAE of 1-D CNN over 50 epochs.
Figure 16. Loss and MAE of 1-D CNN over 50 epochs.
Agriculture 14 01262 g016
Figure 17. Loss and MAE of ANN over 50 epochs.
Figure 17. Loss and MAE of ANN over 50 epochs.
Agriculture 14 01262 g017
Table 1. Mean of the 12 color features and mean value of chlorophyll in the dataset.
Table 1. Mean of the 12 color features and mean value of chlorophyll in the dataset.
ParametersValue
Mean_R163.17
Mean_G167.12
Mean_B157.06
SD_R43.77
SD_G35.68
SD_B57.17
Variance2364.98
SD_H48.11
SD_S69.68
SD_V36.48
Kurtosis7.11
Skewness−2.18
Chl.35.22
Table 2. Division of dataset.
Table 2. Division of dataset.
Total SizeTraining SetValidation SetTesting Set
1500960240300
Table 3. Hyperparameters of 1-D CNN.
Table 3. Hyperparameters of 1-D CNN.
ParametersPurposeValue
α Learning rate or step size0.01
β 1 Decay rate of an average of gradients0.9
β 2 Decay rate of an average of gradients0.999
ϵ Positive constant to ignore the ‘division by 0’ error 10 8
EpochNumber of times the learning occurs10
Batch sizeTo control the accuracy of the error gradient10
Table 4. Model parameters of 1-D CNN.
Table 4. Model parameters of 1-D CNN.
LayerActivation FunctionNo. Hidden NeuronsOutputParameter Evaluated
Conv1DReLU-(None, 12, 64)192
Conv1DReLU-(None, 12, 128)16,512
FlattenReLU-(None, 1536)0
Dense 1ReLU128(None, 128)196,736
Dense 2ReLU64(None, 64)8256
Dense 3ReLU32(None, 32)2080
Final DenseLinear1(None, 1)33
Table 5. Loss and MAE of the 1-D CNN.
Table 5. Loss and MAE of the 1-D CNN.
EpochTraining LossValidation LossTesting LossTraining MAEValidation MAETesting MAE
121.5419.1321.033.493.373.38
218.2316.7818.823.183.133.35
316.9716.2321.793.093.023.46
416.6616.4319.053.053.083.33
516.4116.2617.583.023.033.18
616.4016.5017.013.042.993.18
716.4115.6618.143.042.973.13
815.8415.9417.752.973.063.25
915.6515.9719.932.953.003.46
1015.4815.4316.302.942.933.11
Table 6. MAE, MSE, RMSE, and R 2 of the 1-D CNN.
Table 6. MAE, MSE, RMSE, and R 2 of the 1-D CNN.
ParametersValue
MAE2.96
MSE15.40
RMSE3.92
R 2 0.823
Table 7. Model parameters of ANN.
Table 7. Model parameters of ANN.
LayerActivation FunctionNo. of Hidden Neurons/PropertiesOutputParameter Evaluated
DenseReLU512(None, 512)6656
DropoutWith a 0.2 dropout rate(None, 512)
Dense 1ReLU256(None, 256)131,328
Dense 2ReLU64(None, 64)16,448
Final DenseLinear1(None, 1)65
Table 8. Loss and MAE of ANN.
Table 8. Loss and MAE of ANN.
EpochTraining LossValidation LossTesting LossTraining MAEValidation MAETesting MAE
122.0518.3823.333.593.203.72
219.0916.8220.843.323.123.48
319.0516.4223.963.323.053.80
418.6716.0223.653.253.063.76
519.2016.8821.973.373.213.61
618.2716.2522.093.233.123.58
718.9815.8420.843.353.013.45
818.2115.9824.693.273.023.80
917.7116.2419.733.183.093.37
1018.4315.9917.183.253.043.16
Table 9. MAE, MSE, RMSE, and R 2 of the ANN.
Table 9. MAE, MSE, RMSE, and R 2 of the ANN.
ParametersValue
MAE3.04
MSE15.87
RMSE3.98
R 2 0.818
Table 10. Training and testing scores of LR, SVR, and KNN.
Table 10. Training and testing scores of LR, SVR, and KNN.
ModelTrainingTesting
LR75.1673.57
SVR73.1370.45
KNN (K = 5)85.7880.99
Table 11. MAE, MSE, RMSE, and R 2 of the LR, SVR, and KNN.
Table 11. MAE, MSE, RMSE, and R 2 of the LR, SVR, and KNN.
ModelLRSVRKNN
MAE3.73.823.01
MSE23.1225.8216.63
RMSE4.805.084.07
R 2 0.730.700.80
Table 12. Hyperparameters of SVR.
Table 12. Hyperparameters of SVR.
ParametersValue
C1.0
Cache Size200
KernelLinear
Degree1
Epsilon0.1
GammaAuto
Table 13. Comparison of results of various contact imaging techniques.
Table 13. Comparison of results of various contact imaging techniques.
Sl. NoPaperPlantExternal Light SourceAccuracyModel
1Vesali et al. [3]Corn leafNo74%LR
2Vesali et al. [3]Corn leafNo82%ANN
3Vesali F et al. [3]Corn leafNo97%LR
4Barman and Choudhury [1]Tender CitrusYes82%LR
5Barman and Choudhury [1]Immature CitrusYes75%LR
6Barman and Choudhury [1]Mature CitrusYes80%LR
7Barman and Choudhury [1]Tender CitrusYes94%ANN (LM)
8Barman and Choudhury [1]Immature CitrusYes99%ANN (LM)
9Barman and Choudhury [1]Mature CitrusYes96%ANN (LM)
10Barman and Choudhury [1]Tender CitrusYes94%ANN (SCG)
11Barman and Choudhury [1]Immature CitrusYes93%ANN (SCG)
12Barman and ChoudhuryMature CitrusYes96%ANN (SCG)
13Proposed SystemTea leafYes82%1-D CNN
14Proposed SystemTea leafYes81%ANN
15Proposed SystemTea leafYes80%KNN
16Proposed SystemTea LeafYes73%LR
17Proposed SystemTea LeafYes70%SVR
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Barman, U.; Saikia, M.J. Smartphone Contact Imaging and 1-D CNN for Leaf Chlorophyll Estimation in Agriculture. Agriculture 2024, 14, 1262. https://doi.org/10.3390/agriculture14081262

AMA Style

Barman U, Saikia MJ. Smartphone Contact Imaging and 1-D CNN for Leaf Chlorophyll Estimation in Agriculture. Agriculture. 2024; 14(8):1262. https://doi.org/10.3390/agriculture14081262

Chicago/Turabian Style

Barman, Utpal, and Manob Jyoti Saikia. 2024. "Smartphone Contact Imaging and 1-D CNN for Leaf Chlorophyll Estimation in Agriculture" Agriculture 14, no. 8: 1262. https://doi.org/10.3390/agriculture14081262

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop