**Non-Invasive Tools to Detect Smoke Contamination in Grapevine Canopies, Berries and Wine: A Remote Sensing and Machine Learning Modeling Approach**

**Sigfredo Fuentes 1,\*, Eden Jane Tongson 1, Roberta De Bei 2, Claudia Gonzalez Viejo 1, Renata Ristic 2, Stephen Tyerman <sup>2</sup> and Kerry Wilkinson <sup>2</sup>**


Received: 16 July 2019; Accepted: 28 July 2019; Published: 30 July 2019

**Abstract:** Bushfires are becoming more frequent and intensive due to changing climate. Those that occur close to vineyards can cause smoke contamination of grapevines and grapes, which can affect wines, producing smoke-taint. At present, there are no available practical in-field tools available for detection of smoke contamination or taint in berries. This research proposes a non-invasive/in-field detection system for smoke contamination in grapevine canopies based on predictable changes in stomatal conductance patterns based on infrared thermal image analysis and machine learning modeling based on pattern recognition. A second model was also proposed to quantify levels of smoke-taint related compounds as targets in berries and wines using near-infrared spectroscopy (NIR) as inputs for machine learning fitting modeling. Results showed that the pattern recognition model to detect smoke contamination from canopies had 96% accuracy. The second model to predict smoke taint compounds in berries and wine fit the NIR data with a correlation coefficient (R) of 0.97 and with no indication of overfitting. These methods can offer grape growers quick, affordable, accurate, non-destructive in-field screening tools to assist in vineyard management practices to minimize smoke taint in wines with in-field applications using smartphones and unmanned aerial systems (UAS).

**Keywords:** bushfires; infrared thermography; near-infrared spectroscopy; smoke taint; artificial intelligence

### **1. Introduction**

A recent report from the Victorian government of Australia concluded that bushfires have increased in number and severity since the 1970s across the east and south of the country [1]. The main contributing factor to this environmental disaster is climate change, specifically the increased frequency of recurrent heat waves (i.e., prolonged periods of hotter weather) and drought conditions, which have increased the window of risk for bushfires, as well as their number, and severity. Recently, Chile (central region), USA (California), Greece, South Africa (Stellenbosch) and Australia (various states) have suffered some of the worst bushfires experienced in each country's history. These countries are major producers of wines, and their grape growers and winemakers are similarly affected by global warming with detrimental effects in drought, vine phenological changes, shifting of suitable grapevine growing regions towards the north and south, and increased bush fire events near wine growing regions [2–4].

When bushfires occur in close proximity to vineyards, smoke can contaminate leaves and fruit. One of the main physiological effects of bush fire smoke in grapevines is the reduction of stomatal conductance (gs) [5]. Decreased gs may be explained by the combination of the main smoke components

carbon dioxide (CO2) and carbon monoxide (CO), with water vapor (100% RH in the substomatal cavity) producing carbonic acid (H2CO3) that reduces pH in the stomata, thereby causing partial or complete stomatal closure [5]. In berries, smoke contamination results in adsorption of smoke-derived volatile phenols (which accumulate in glycoconjugate forms), that are extracted into the final wine during the winemaking process [6]. Several mitigating measures have been evaluated to minimize smoke taint in berries or to remove volatile phenols (and their glycoconjugates) from wine, including defoliation [7] or foliar application of kaolin [8] in the vineyard, reverse osmosis treatment [9] and the addition of fining agents in wines [10]. However, implementation of defoliation or kaolin applications are often indiscriminate and broadly applied irrespective of the degree of grapevine exposure to smoke. Furthermore, the removal of smoke taint from wine is not selective and may inadvertently remove important wine compounds, thereby affecting the desirable organoleptic characteristics of wine.

Physiological assessment of control (non-smoked) and smoke affected grapevine cultivars have shown that some cultivars are more susceptible than others in terms of photosynthesis and stomatal conductance, in particular, Merlot and Cabernet Sauvignon. In contrast, Sauvignon Blanc was not significantly affected by smoke contamination from a physiological perspective. However, berries exposed to smoke resulted in wines with significantly higher concentrations of volatile phenols and guaiacol glycoconjugates compared to wines made from uncontaminated fruit [6].

The need to assess smoke contaminated fruit and wine has led to the implementation of new laboratory-based analytical methods [11], including liquid chromatography-tandem mass spectrometry for quantification of volatile phenol glycoconjugates [12,13]. However, these techniques require expensive laboratory instrumentation, and specialized technical skills to prepare samples (i.e., to extract the analytes of interest), operate the instruments and data analysis. Spectral methods, both mid-infrared (MIR) reflectance spectroscopy and chemometric techniques have been evaluated for rapid detection of smoke-taint in grapes [8] and bottled wines [14], but also with limitations. Spectral reflectance measurements of berries were affected by fruit maturity, while MRI-based classification of wines was influenced by cultivar, oak maturation and the level of smoke taint. Thus, reliable and rapid in-field techniques available to determine whether vines and fruits have been contaminated with smoke from bushfires are not yet available. This paper presents pattern recognition and regression models based on machine learning algorithms developed for the identification of smoke contaminated grapevine canopies and fruit. The first machine learning model generated used infrared thermography data from grapevine canopies as inputs to predict smoke contamination, as a target for four grapevine cultivars. Near infrared (NIR) spectroscopy readings from berries were used as inputs for regression machine learning algorithms to assess specific smoke-related compounds in berries and final wines from seven cultivars. These models combined with affordable geo-referenced NIR spectroscopy measurements of berries could allow growers to map contaminated areas of a vineyard to facilitate decision making at harvest. Finally, potential applications of these models using proximal and mid-range remote sensing using unmanned aerial systems (UAS) are discussed.

### **2. Materials and Methods**

### *2.1. Experimental Site and Application of Smoke to Grapevines*

Grapevine smoke exposure experiments were conducted in the 2009/10 season using seven different cultivars grown at two locations: (i) Sauvignon Blanc, Pinot Gris, Chardonnay and Pinot Noir grown in a commercial vineyard in Adelaide Hills region, South Australia, Australia (35◦00 S, 138◦49 E) and (ii) Shiraz, Cabernet Sauvignon and Merlot vines grown in a vineyard located at the University of Adelaide's Waite campus in Adelaide, South Australia (34◦58 S, 138◦38 E). Grapevines (three per replicate) were exposed to smoke (for 1 h) using a purpose-built smoke tent and experimental conditions described previously (Figure 1) [15]. Smoke was applied to vines at a phenological stage corresponding to approximately seven days post-veraison; when total soluble solids (TSS) concentrations were approximately 15 Brix, determined using a digital handheld refractometer (PAL-1, Atago, Tokyo, Japan).

Experiment 1 consisted of the physiological assessment of smoke contamination at the canopy level using porometry, infrared thermography, and pattern recognition machine learning using four cultivars two hours after smoke exposure: Chardonnay, Merlot, Sauvignon Blanc, and Shiraz. Experiment 2 assessed smoke taint in berries and wine at harvest for all seven cultivars. In this experiment, control (unsmoked) and smoke-affected berry samples (two berries taken from the mid-section of two bunches from two replicates per treatment, per cultivar; *n* = 112 berries) were collected at harvest. Morphometry of berries was measured using a caliper to obtain diameter (equatorial length in cm); length (cm) and calculated radius (cm), area (cm2), and perimeter (cm).

**Figure 1.** Grapevines were enclosed in a tent, and smoke from combustion of straw was blown into the tent using a fan. Pre-installation of the tent (**A**), and installed and operational tent (**B**). Photos obtained from the 2009/10 trial in Adelaide, Australia.

### *2.2. Experiment 1*

### 2.2.1. Physiological Measurements Using Leaf Porometry

Leaf conductance to water vapor was measured as stomatal conductance (gs) using a porometer (AP4, Delta-T Devices, Cambridge, UK). Porometer readings used were obtained from the cultivars Shiraz, Sauvignon Blanc, Chardonnay, and Merlot. Measurements were performed two hours after smoke treatments using nine mature, fully expanded sunlit leaves, for each of the two middle vines of two replicates per treatment per cultivar (*n* = 72) under natural leaf orientation with natural light intensity. Leaves were chosen to ensure measurements were performed on three leaves from the top, middle, and bottom parts of the canopies from each vine in a 3 × 3 matrix arrangement.

### 2.2.2. Infrared Thermal Imagery of Canopies

Thermal images were acquired from grapevine canopies using an infrared thermal camera FLIR® T-series (Model B360) (FLIR Systems, Portland, OR, USA), with a resolution of 320 <sup>×</sup> <sup>240</sup> pixels. The camera measures temperature in the range of −20 to +1200 ◦C. The thermal sensitivity of the camera is <0.08 ◦C @ +30 ◦C/80 mK with a spatial resolution of 1.36 milliradians. Each pixel is considered an effective temperature reading in degrees Celsius (◦C). Infrared thermal images were acquired from the same side and in parallel to porometer measurements (shaded side of the canopy to reduce variability) in the estimation of the infrared index (Ig), which is proportional to gs [16]. One thermal image from the canopy of each of the middle vines of two replicates per treatment per cultivar was obtained from a constant distance of 2.5 m perpendicular to the row direction (distance between rows being 3 m; Figure 2A). The calculated infrared thermal index (Ig) was compared with porometry measurements acquired immediately after obtaining each thermal image from corresponding vines. All thermal images were acquired on a clear day. The smoke treatments were applied with minimal wind; a requirement for

undertaking the field trials implemented to avoid the risk of fire spreading from accidental burning of interrow dry plant material and to secure representativeness of thermal images [16,17].

**Figure 2.** Examples of a radiometric thermal image (**A**) processing for data extraction of Tdry (A, solid circle) and Twet (A, dotted circle) by painting leaves with petroleum jelly and water, respectively. Binary image obtained by thresholding Tdry and Twet (**B**); masked radiometric image extracting non-leaf material, such as overheated elements and sky (**C**); and subdivision of thermal image to extract information from sections of the canopy in a 5 × 5 sub-division (**D**).

### 2.2.3. Algorithms Used to Calculate Crop Water Stress Indices (CWSI) and Infrared Index (Ig)

Crop water stress index (CWSI) was calculated using the following equation, after determining *Tdry* and *Twet* [18]:

$$\text{CWSI} = \frac{T\_{\text{canopy}} - T\_{\text{wet}}}{T\_{\text{dry}} - T\_{\text{wet}}} \tag{1}$$

where *Tcanopy* is the actual canopy temperature extracted from the thermal image at determined positions, and *Tdry* and *Twet* are the reference temperatures (in ◦C), obtained using the method of painting both sides of reference leaves with petroleum jelly and water, respectively [16].

An infrared index (*Ig*), proportional to leaf conductance to water vapor transfer (*gs*), can be obtained using the relationship as follows [19]:

$$I\_{\mathcal{S}} = \frac{T\_{\text{canopy}} - T\_{\text{wet}}}{T\_{\text{dry}} - T\_{\text{wet}}} = \ g\_{\text{s}} \left( r\_{\text{aw}} + \left( \frac{\text{s}}{\mathcal{Y}} \right)\_{r\_{\text{HR}}} \right) \tag{2}$$

where *raw* = boundary layer resistance to water vapor, *rRH* = the parallel resistance to heat and radiative transfer, Υ = psychrometric constant and *s* = slope of the curve relating saturation vapor pressure to temperature [17,19].

### 2.2.4. Infrared Thermography Data Extraction

The Tdry and Twet values were obtained on a per image basis using a customized code written in Matlab® R2019a (Mathworks Inc. Natick, MA, USA) to crop the radiometric data from the areas within the respective painted leaves with water (Twet) and petroleum jelly (Tdry) (Figure 2A). To filter non-leaf material from the radiometric image using the determined threshold, a second customized code was written in Matlab®to binarize a masked image (Figure 2B) and to extract these values from the original image (Figure 2C). For automatic extraction of data within a canopy, a pre-defined subdivision of 3 × 3 = 9; 5 × 5 = 25; 7 × 7 = 49 and 10 × 10 = 100 was automatically implemented (Figure 2D; for the case of 5 × 5). From these subdivisions, data were extracted for Tcanopy per image (Figure 2D), Ig Equation (2) and CWSI Equation (1).

The image sub-divisions (Figure 2D) represent the matrix (A) with *m* × *n* (*m* = rows and *n* = columns) extraction points represented as per the following matrix:

$$A(m,n) = \begin{pmatrix} T\_{1,1} & \cdots & T\_{1,n} \\ \vdots & \ddots & \vdots \\ T\_{m,1} & \cdots & T\_{m,n} \end{pmatrix} \tag{3}$$

Since *m*, *n* represent the pre-determined subdivision for automatic cropping sections from the infrared thermal image (A), every sub-image is processed for automatic canopy extraction by filtering non-leaf temperatures using the Tdry and Twet values extracted (Figure 2B) as minimum and maximum possible temperatures for the canopy. The calculated T value then corresponds to the averaged Tcanopy for each sub-division.

2.2.5. Pattern Recognition of Infrared Thermal Imagery using Machine Learning for Smoke Contamination Prediction

Pattern recognition models were developed using a customized Matlab®code, which is able to test 17 different training algorithms, two from Backpropagation with Jacobian derivatives, 11 from Backpropagation with gradient derivatives and four from Supervised weight and bias training functions, in loop to select the best model. This model was constructed using the infrared thermal image output values as inputs to classify the samples into smoked and non-smoked (control). The infrared thermal images were analyzed with the methodology described in Figure 2 to obtain Tcanopy, Ig, and CWSI data obtained using Equations (1) and (2) with sub-divisions of 3 × 3 (*n* = 27 per image); 5 × 5 (*n* = 75, per image; 7 × 7 (*n* = 147, per image) and 10 × 10 (*n* = 300 per image). All algorithms tested used a random data division. However, for the algorithms such as scaled conjugate gradient, which consist of three stages—training, validation and testing, the data was divided as 60% (*n* = 28 images) for training, 20% (*n* = 10 images) for validation with a cross-entropy performance algorithm, and 20% (*n* = 10 images) for testing with a default derivative function. For the algorithms such as sequential order weights and bias, which only consist of training and testing stages, the data was divided as 70% (*n* = 34) for training and 30% (*n* = 14) for testing with a cross-entropy performance algorithm. A trimming exercise was conducted using 3, 7 and 10 neurons to select the best model with no signs of overfitting (Figure 3).

**Figure 3.** Diagram of the two-layer feedforward network with a tan-sigmoid function in the hidden layer and a Softmax transfer function in the output layer. For hidden and output layers, *w* = weights and *b* = biases. Input volume and neuron trimming exercises are included in the diagram.

### *2.3. Experiment 2*

### 2.3.1. Berry Near Infrared (NIR) Spectroscopy Measurements

Full berries were scanned using a spectrophotometer (ASD FieldSpec®3, Analytical Spectral Devices, Boulder, CO, USA) equipped with the ASD contact probe, built for contact measurements, attached by fiber optic cable to the instrument. A total of 112 berries collected at harvest from seven cultivars (16 berries per cultivar) were scanned by putting the probe's lens in contact with the berries and a total of 401 spectra were recorded for each berry. The instrument records spectra with a resolution of 1.4 nm for the region 350–1000 nm and 2 nm for the region 1000–1850 nm. The instrument was used in reflectance mode and data was then transformed into absorbance values (absorbance = log (1/reflectance)). A reference tile (Spectralon®, Analytical Spectral Devices, Boulder, CO, USA) was used as a white reference, for scatter correction. A new reference was taken every ten spectra acquisitions.

### 2.3.2. Winemaking and Chemical Analysis of Berries and Wine

Small scale winemaking of control and smoke-affected fruit from this trial has been described previously in detail by Ristic et al. [6]. Guaiacol glycoconjugates were measured in fruit and wine by HPLC–MS/MS using a stable isotope dilution analysis (SIDA) method developed by Dungey et al. (2011) [12]. Volatile phenols, including guaiacol, were determined in berries and wine by the Australian Wine Research Institute's (AWRI) Commercial Services Laboratory (Adelaide, Australia). Volatile phenols were measured by GC–MS according to SIDA methods reported previously [13].

2.3.3. Fitting Modeling of Near-Infrared (NIR) Spectroscopy of Berries Using Machine Learning Modeling to Predict Smoke Taint in Berries and Wine

A regression model was developed using a customized Matlab® code, which is able to test 17 different training algorithms, two from Backpropagation with Jacobian derivatives, 11 from Backpropagation with gradient derivatives and four from Supervised weight and bias training functions, in loop to select the best model. NIR absorbance values corresponding to the range of wavelengths within 700 and 1100 nm with a second derivative transformation, which were used as inputs in the machine learning algorithms, since that range corresponds to alcohol and alcohol-based compounds to predict (i) guaiacol glycoconjugates in berries (μg Kg<sup>−</sup>1), (ii) guaiacol glycoconjugates in wines (μg L<sup>−</sup>1) and iii) guaiacol in wine (μg L<sup>−</sup>1). All algorithms tested used a random data division. However, for the algorithms, which consist of three stages—training, validation and testing, the data was divided as 60% (*n* = 28) for training, 20% (*n* = 10) for validation with a means squared error (MSE) performance algorithm and 20% (*n* = 10) for testing with a default derivative function (data not shown). For the algorithms such as sequential order weights and bias, which only consist of training and testing stages, data was divided as 70% (*n* = 34) for training and 30% (*n* = 14) for testing with a means squared

error performance algorithm. A trimming exercise was conducted using 3, 7, and 10 neurons to select the best model with no signs of overfitting (Figure 4).

**Figure 4.** Diagram of the two-layer feedforward network with a tan-sigmoid function in the hidden layer and a linear transfer function in the output layer. For hidden and output layers, *w* = weights and *b* = biases. Neuron trimming exercise is included in the diagram.

### *2.4. Statistical Analysis*

Data from chemometry and morphometry of berries, wine compounds, and Ig and gs were analyzed through ANOVA using SAS® 9.4 software (SAS Institute Inc., Cary, NC, USA) with Tukey's studentized range test (HSD; *p* < 0.05) as post-hoc analysis for multiple comparisons to assess significant differences. Statistical data such as means and standard deviation (SD) were obtained from the replicates of each cultivar and treatment.

### **3. Results**

### *3.1. Experiment 1*

### 3.1.1. Grapevine Physiological Data Relationships between Porometry and Infrared Thermal Imagery

Table 1 shows the mean values for gs and Ig with respective standard deviations (SD) for the four cultivars from Experiment 1. The general trend for the mean values of the control treatments follows a positive linear relationship (R<sup>2</sup> = 0.99; Ig = 0.0027 gs). On the contrary, the trend for the mean values of the smoke treatments have lower linearity and relationship, but still showed a positive linear pattern (R2 = 0.23; Ig = 0.0023 gs; data not shown). In the control samples, the mean Ig values per cultivar did not show significant change, as reflected by the SD values, but Merlot showed a significantly higher Ig (*p* < 0.05). This trend was similar for the mean gs values showing Merlot with the highest mean (*p* < 0.05). The mean Ig values for smoked samples were more variable, while gs showed higher mean values, except Sauvignon Blanc, and more variability as reflected by the higher SD values compared to control. The Ig mean values for both treatments were not very sensitive, as seen in Table 1.

Figure 5 shows the relationships between gs and Ig for different sections of canopies (top, middle, and bottom) of the grapevines monitored for both non-smoked (control) and smoked treatments. The graph (Figure 5A) shows a strong and significant linear relationship between gs and Ig (R<sup>2</sup> = 0.85; Ig = 0.0026 gs). However, there was no relationship observed for smoke treatments, with the data presenting high variability, which is consistent with results shown in Table 1. Figure 5A shows that regardless of the measurement position within the canopy for Ig, there is a broader distribution of values between top, middle, and bottom of the canopy along the linear relationship found. On the contrary, Figure 5B shows that the bottom readings for gs are more concentrated towards the lower values (<200 mmol m<sup>2</sup> s<sup>−</sup>1). Furthermore, the Ig values become less sensitive (spread between 0 and 1). The same pattern can be seen for most of the top readings with the middle readings having a wider spread distribution.


**Table 1.** Means and standard deviation (SD) per variety and treatment for the infrared index (Ig, unitless) and stomatal conductance (gs in mmol m2 s<sup>−</sup>1) calculated for all the images without sub-divisions.

Means followed by different superscript letters are statistically significant between treatments based on Tukey's studentized range test (HSD, *p* < 0.05).

**Figure 5.** Relationship between gs and Ig for Experiment 1 in the four cultivars with data separated between canopy sections: top (Top), middle (Mid) and bottom (Bot) measurements for control treatments (**A**) and smoked treatments (**B**).

3.1.2. Pattern Recognition Using Machine Learning Modeling of Physiological and Infrared Thermal Data

Table 2 shows the results of the pattern recognition modeling for the data extracted from infrared thermal images from the canopies of four different cultivars combined for Experiment 1. The best performing algorithm for the 3 × 3 sub-division and extraction of Tcanopy, Ig, and CWSI used as inputs and classification of smoked and non-smoked as target was the scaled conjugate gradient algorithm. The training, validation, and testing procedures (using 10 neurons) resulted in an overall model with 94% accuracy. In the case of the data extracted using a 5 × 5 sub-division, the overall best model (sequential order weight and bias) resulted in an accuracy of 88% (using 10 neurons) in the classification of smoked and non-smoked canopies. For the 7 × 7 sub-division, the best algorithm (also the sequential order weight and bias) resulted in an accuracy of 94% (using 7 neurons) in the classification. Finally, the 10 × 10 was the best performing algorithm overall (sequential order weight and bias) resulted in an accuracy of 96% (using 3 neurons). Furthermore, the performance of training was lower than the one for testing, and testing accuracy was close to that from the training stage, which are evidence of no overfitting [20,21].

**Table 2.** Best pattern recognition model developed for each set of inputs showing the best training algorithm and number of neurons to predict whether canopies are smoked or non-smoked (control). Inputs corresponds to data extracted from infrared thermal images for Tcanopy, Ig and crop water stress index (CWSI) in matrix arrangement of 3 × 3 (*n* = 27), 5 × 5 (*n* = 75), 7 × 7 (*n* = 147) and 10 × 10 (*n* = 300) data points per thermography. Performance reported is based on cross-entropy.


Figure 6 shows the Receiver Operating Characteristic (ROC) for the best performing model found to predict smoke contamination in grapevine canopies (10 × 10 sub-division; Table 2). The figure shows that results for both smoke and control pattern recognition using infrared thermography data as inputs are projected in a similar trend to the True Positive Rate prediction axis of the graph.

**Figure 6.** Receiver Operating Characteristic (ROC) showing the false positive rate (*x*-axis) and true positive (*y*-axis) for control and smoked treatments for the best performing classification model found in Table 3.


**Table 3.** Morphometric data obtain from berries for all seven cultivars consisting in Perimeter (P in cm), Equatorial Diameter (D in cm), calculated Area (A in cm2) and Radius (D/2 in cm). For chemometry, Total Soluble Solids (TSS) represented by Brix and Near Infrared (NIR) absorbance at 982 nm corresponding to H–O–H and O–H chemical bonds.

Abbreviations: C = Control, S = Smoke, PinGr = Pinot Gris, PinNoir = Pinot Noir, Char = Chardonnay, CabSauv = Cabernet Sauvignon, SauvBl = Sauvignon Blanc. Different superscript letters are statistically significant between treatments based on Tukey's studentized range test (HSD, *p* < 0.05).

### *3.2. Experiment 2*

### 3.2.1. Berry Morphology and NIR Peak within the 700–1100 nm

Table 3 shows the average data of morphometric and chemometric measurements obtained from berry samples for all the seven cultivars for Experiment 2. Even though there are some significant differences between morphometric measurements of berries for the different cultivars comparing smoke and non-smoked (Control) treatments, they do not affect results and models developed.

### 3.2.2. Smoke-Related Compounds Found in Berries and Wines

Data for smoke-related compounds have been previously reported by Ristic et al. (2016) [6], and comprised of volatiles with statistical differences between control (non-smoked) and smoked treatments. Specifically, for purposes of modeling, guaiacol glycoconjugates found in berries (μg Kg<sup>−</sup>1), guaiacol glycoconjugates found in wines (μg L<sup>−</sup>1) and guaiacol found in wines (μg L<sup>−</sup>1) were used since these are the primary compounds identified by the industry to contribute to smoke taint. In berries, the guaiacol glycoconjugates average concentration ranged for control between 37 and 602 μg kg−<sup>1</sup> and from 253 to 2452 μg kg−<sup>1</sup> for smoke-affected treatments. The guaiacol glycoconjugates concentrations in wines ranged from 8 to 334 μg L−<sup>1</sup> for control and from 111 to 1480 μg L−<sup>1</sup> for smoke-affected treatments. In the case of guaiacol concentration in wines, values ranged from 0 (not detected) to 9 μg L−<sup>1</sup> for control and from 0 (not detected) to 26 μg L−<sup>1</sup> [6].

### 3.2.3. Near-Infrared (NIR) Spectroscopy from Berries and Smoke Taint Compounds Found

Figure 7 shows the main average spectra for berries from smoke and non-smoked (control) treatments for red (Figure 7A) and white cultivars (Figure 7B). There were no significant differences in the averaged spectra between smoked and non-smoked berries for red cultivars. On the contrary, there appears to be a consistent difference for white cultivars of around 0.05 in absorbance, especially from 820 to 1100 nm for the range considered for this study. Smoke-related compounds for this trial and used for the machine learning model reported here have been previously reported by Ristic et al. [6]. In this study, statistically significant differences in the main smoke taint compounds were reported for all the seven cultivars included in Experiment 2.

**Figure 7.** Average spectra for control (solid line) and smoke-affected berries (dashed line) from red cultivars (**A**): Merlot, Shiraz, Pinot Noir, Cabernet Sauvignon and white cultivars (**B**): Pinot Gris, Chardonnay and Sauvignon Blanc. The grey rectangles represent the wavelengths used for machine learning fitting modeling (700–1100 nm), with the main peak at 982 nm.

3.2.4. Machine Learning Modeling Based on NIR Spectra to Estimate Smoke Taint Compounds in Berries and Wine

Table 4 shows the best machine learning regression model obtained for the NIR data from berries (700–1100 nm using the second derivative transformation; Sequential Order Weights and Bias) as inputs and smoke taint compounds measured in berries and wine. The correlation between the estimated and observed values was R = 0.97 and slope b = 0.93 (close to unity). The same correlations and similar slopes were found for the training and the test stages. The overall model can also be seen in Figure 8, in which most of the point cloud data fits in the 1:1 line representing the accuracy of predicted versus observed data. Based on the 95% confidence bounds, the overall model had 3.6% of outliers. The performance of training was lower than the one for testing, and testing accuracy was the same as that from the training stage, which are evidence of no overfitting [20,21].

**Table 4.** Regression model using machine learning (Sequential Order Weights and Bias) for NIR data from berries of seven grapevine cultivars showing the correlation coefficient (R) and performance based on mean squared error (MSE) for each stage.


**Figure 8.** Overall fitting model using machine learning (Sequential Order Weights and Bias) using NIR spectra (700–1100 nm; second derivative transformation) of berries from seven grapevine cultivars as inputs and main smoke taint compounds found in berries and wine as targets.

### **4. Discussion**

### *4.1. Physiological Changes within Grapevine Canopies Due to Smoke Contamination*

The relationship between the Ig thermal index and gs is linear, as shown in Table 1 and Figure 5A for non-smoked vines. These results are consistent with other studies showing the same relationships for grapevines [16,17], coffee plants [22] and olive trees [23], which are tree-like or bushy canopies. However, this relationship was not observed for smoked canopies of the four cultivars from Experiment 1 (Figure 5B). Smoke contamination is an external signal to the plant which is composed mainly of CO, CO2 and other gases, which cause acidification of the sub-stomatal cavity due to the production of carbonic acid (H2CO3) when combined with water, with the resulting pH reduction causing partial or complete stomata closure [5]. This effect could explain the increased variability within gs data amongst individual leaves that was detected in porometry data (Table 1 and Figure 5B). The reported Ig data from the whole infrared thermal images (Table 1) did not have significant differences in the variability of the data, which can be explained by the unrepresentativeness of means when using this type of high-resolution information.

It is important to note that the comparison between gs and Ig for Figure 5 was made in this case using the methodology proposed in Figure 2 and with a sub-division of 3 × 3 for comparison purposes. Since every image was taken from 2.5 m distance, the field of view from infrared thermal images was around 140 <sup>×</sup> 110 cm of the canopy, which divided by nine gives a sub-area of <sup>47</sup> <sup>×</sup> 37 cm (area <sup>=</sup> 1739 cm2). Considering that the area of an average leaf (data not shown) is of around 50–80 cm<sup>2</sup> [24], the Ig values represent the average of an area of approximately 25-fold of single leaves, in which porometry was conducted. This difference may explain the lower sensitivity of Ig to gs, especially for smoked canopies with higher gs variability expected even at the leaf level (patchy stomata behavior).

The extraction of Ig values from infrared thermal images require a Tdry and Twet reference temperatures. In this study, the painted leaves method was implemented for more accuracy in the determination of reference temperature thresholds to separate leaf from non-leaf material in the analysis. However, this method is manual and hinders the possibility of automation. Alternatively, the leaf energy balance method could be implemented using micrometeorological weather data such

as temperature, relative humidity, and solar radiation to calculate Tdry and Twet on-the-go, while obtaining the infrared thermal images. It is common nowadays to access cheap sensor technology to measure these micrometeorological variables and dataloggers or access to the Internet of Things (IoT) for data transmission and processing. Previous research has shown that these reference temperatures can be calculated with high accuracy (R2 = 0.95; RMSE = 0.85; *p* < 0.001) [16]. Furthermore, there is the requirement for infrared thermal images to be explored and assessed more in-depth at higher subdivisions and using machine learning modeling to assess the pattern variability and use it as a predictor of smoke contamination levels.

### *4.2. Pattern Recognition of Smoke Contamination Using Machine Learning Modeling*

Considering the sub-division of infrared thermography data, the field of view of canopies and size of single leaves for this study, it is not surprising that the best pattern recognition model (96% accuracy) using machine learning (Sequential order weight and bias) was obtained with the 10 × 10 subdivision. This sub-division will render comparison areas within the canopy of 154 cm2, which is only 2.2-fold compared to a single leaf area (70 cm2). Furthermore, from the neuron trimming analysis, a highly accurate model was obtained for the classification of smoked and non-smoked canopies with three neurons, which makes the model more efficient and less susceptible to overfitting. The latter is also supported by the performance value obtained by this model. Results shown in this paper from pattern recognition modeling using machine learning to asses smoke contamination of canopies have excellent potential for the use in short and mid-range remote sensing based on Unmanned Aerial Vehicles (UAVs) platforms. From Figure 5B, it can be seen that the main variability within gs values is in the bottom and top parts of the canopies, which validates obtaining infrared thermal imagery using UAVs at 0◦ Nadir angle. Furthermore, models developed in this study should be tested using UAV with infrared cameras that could render a 15 × 15-pixel resolution, which corresponds to an area of 225 cm2, which is close to the 154 cm2 area used for machine learning modeling here.

This kind of remote sensing tool can render spatial distribution maps of contaminated areas within vineyards that could aid growers to apply differential management strategies discussed before to mitigate smoke contamination of the fruit. Spatial maps of smoke contamination can also help to achieve differential harvests to avoid mixing fruit with smoke-tainted fruit. Hence, a system is proposed using these methods, which is depicted in Figure 9 for proximal and mid-distance remote sensing using infrared cameras and UAV platforms. For proximal remote sensing, the algorithms developed in this study can be implemented in smartphone devices as computer applications (Apps) connected to portable and affordable infrared thermal cameras (i.e., FLIR One®, FLIR Systems, Portland, OR, USA) and near-infrared spectroscopy devices (i.e., Lighting Passport®, AsenseTek, Taipei, Taiwan).

**Figure 9.** Diagram showing the implementation of machine learning modeling strategies proposed in this paper for proximal (using smartphones and portable infrared thermal cameras and NIR devices) and mid-distance remote sensing using unmanned aerial system (UAS) platforms.

### *4.3. Near-Infrared (NIR) Spectroscopy of Berries*

Since NIR spectroscopy was obtained from full berries, the tool proposed in this paper is non-destructive. Furthermore, it has been shown that a higher concentration of smoke-related compounds after contamination can be found in the skin of berries, which is higher than in the pulp and higher than the seeds [12]. Furthermore, the range of 700–1100 nm was chosen since most of the available NIR instrumentation in this range can be affordable for growers compared to the instrument used in this study which can cost around 45 times more. The 982 nm overtone is associated with the OH overtone band and 1100 for the CH bands, which corresponds to alcohol and phenolic compounds [25].

The model reported using machine learning fitting algorithms can be of great assistance to growers and winemakers to obtain chemometry data in real time using the proposed methodology shown in Figure 9. Currently, growers do not have sophisticated tools to assess potential smoke contamination of berries bunches and wines. The only option available is collection of samples within a vineyard for compositional analysis by an accredited laboratory using GC-MS or HPLC-MS/MS. This process is destructive, expensive, and takes a long time, which makes it less ideal for the implementation of mitigation strategies and/or decision making before harvest. Furthermore, it may minimize smoke taint by the information provided through a spatial assessment of the contamination either through canopies or berries for informed decision making regarding palliative measures (as presented in this paper) or differential harvest.

The models developed in this study were able to predict smoke contamination in canopies, berries and wines, regardless of the cultivar. Hence, the models could be applied as a universal methodology. Further studies and data acquired could be added to models to include more cultivars. However, the seven cultivars included in this study were some of the most commercially important in Australia. Finally, it is important to note that the levels of smoke-taint compounds present in wine are in part related to the winemaking process (i.e., duration of skin contact time during fermentation), hence this model will need to be adjusted for different winemaking techniques, which can influence the extraction of smoke-related compounds from the berry.

### **5. Conclusions**

This paper showed two main advancements for tools to detect smoke contamination in grapevine canopies and smoke-related compounds in berries and wine using remote sensing techniques. This study is the first to apply machine learning modeling techniques to assist growers confronted with vineyard exposure to smoke from bushfires, an issue which has been exacerbated in prominent wine regions around the world due to climate change. Furthermore, this paper has proposed an affordable method to implement these novel techniques using smartphones, portable thermal imagery and NIR spectroscopy devices. More research is required to assess the usage of these affordable devices in the future using the models proposed.

**Author Contributions:** S.F. conceived the machine learning modeling idea and practical applications; S.F., E.J.T. and C.G.V. analyzed the data and created the machine learning models; K.W., S.T. and S.F. were awarded funding for the study; K.W. and R.R. performed field trials, laboratory analysis and winemaking; S.F. and R.D.B. acquired the physiological and NIR data. All authors contributed to the writing of the paper.

**Funding:** This research received no external funding.

**Acknowledgments:** This research was supported under the Australian Research Council's Linkage Projects funding scheme (LP0989138); the financial contributions of industry partners are also gratefully acknowledged. The machine learning modeling research was supported by the Digital Viticulture program funded by the University of Melbourne's Networked Society Institute, Australia.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **Vibration Monitoring of the Mechanical Harvesting of Citrus to Improve Fruit Detachment E**ffi**ciency**

### **Sergio Castro-Garcia 1,\*, Fernando Aragon-Rodriguez 1, Rafael R. Sola-Guirado 1, Antonio J. Serrano 2, Emilio Soria-Olivas <sup>2</sup> and Jesús A. Gil-Ribes <sup>1</sup>**


Received: 13 March 2019; Accepted: 9 April 2019; Published: 12 April 2019

**Abstract:** The introduction of a mechanical harvesting process for oranges can contribute to enhancing farm profitability and reducing labour dependency. The objective of this work is to determine the spread of the vibration in citrus tree canopies to establish recommendations to reach high values of fruit detachment efficiency and eliminate the need for subsequent hand-harvesting processes. Field tests were carried out with a lateral tractor-drawn canopy shaker on four commercial plots of sweet oranges. Canopy vibration during the harvesting process was measured with a set of triaxial accelerometer sensors with a datalogger placed on 90 bearing branches. Monitoring of the vibration process, fruit production, and branch properties were analysed. The improvement of fruit detachment efficiency was possible if both the hedge tree and the machinery were mutually adjusted. The hedge should be trained to facilitate access of the rods and to encourage external fructification since the internal canopy branches showed 43% of the acceleration vibration level of the external branches. The machine should be adjusted to vibrate the branches at a vibration time of at least 5.8 s, after the interaction of the rod with the branch, together with a root mean square acceleration value of 23.9 m/s2 to a complete process of fruit detachment.

**Keywords:** *Citrus sinensis* L. Osbeck; mechanical harvesting; acceleration sensor; vibration time; logistic regression

### **1. Introduction**

Citrus fruits, whether for fresh consumption or industrial processing, are mainly harvested by hand. Worldwide, 147 million tonnes of citrus were produced in 2017, including orange, grapefruit, lemon, mandarin, and other citrus fruits [1]. Spain is the sixth largest producer of citrus fruit in the world, with an approximate production in 2017–2018 of more than seven million tonnes. In Spain, the predominant citrus orchards are trained for manual harvesting, with an orientation towards the fresh market. Manually harvested orchards experience problems due to the availability of labour and the high cost of operation.

Within the citrus production process, harvesting is a phase of enormous economic importance due to its high impact on the final cost of production. The manual harvesting in Southern Spain requires an average of 95 days' work per hectare and represents between 25% and 35% of the final cost of production [2]. Roka and Hyman [3] stated that, under Florida conditions, the application of mechanical harvesting for industrial processing could provide a 50% cost reduction, while increasing labour productivity by ten per cent. With the current approach to citrus production, the high costs of manual harvesting could compromise the profitability of the activity and the future of plantations in the long term [4].

Since the 1970s, the development of mechanical citrus harvesting systems for the juice industry has mainly taken place in Florida. However, none of the mechanised systems have been able to match the flexibility and fruit selection capabilities of manual harvesting [5]. The foremost mechanical harvesting systems are the trunk and canopy shaker systems that were applied to and developed for citrus fruits, and which reach high values of harvesting efficiency ranging between 84–95% and 55–95%, respectively [6]. However, the detachment of immature fruitlets has been identified in both systems; this occurs particularly in the late varieties that are of special interest to the juice industry and, in addition to a reduction in the working capacity (ha/h), it could represent an obstacle to the adoption of these mechanical harvesting systems by farmers. Roka et al. [7] showed yield reduction values for the use of these machines compared to manual harvesting of 20–50% according to their use and regulation. In parallel, the development and testing of abscission agents have permitted an increase in the working capacity and an improvement in the detachment of mature fruit with these harvesting technologies [8]. It was shown that a moderate reduction in fruit detachment force, through the application of an abscission agent, was enough to significantly increase the harvesting efficiency [9]. Subsequently, it was demonstrated that the use of an abscission agent together with an adjustment of the vibration parameters, fundamentally time and frequency of vibration, allowed high percentages of mature fruit detachment to be achieved without significantly affecting the following season [10]. This result was confirmed based on the different frequency responses of mature fruit and immature fruitlets to mechanical harvesting [11].

Canopy shaker systems allow a continuous vibration of the tree row; the rods penetrate the canopy and achieve a high value of fruit detached in areas where there is direct contact of rods with branches. The use of a canopy shaker can generate a greater fall of leaves, shoots, and branches than manual harvesting. The fall of these organs is considered as tree damage, which could have negative implications in the yield, the productive life, and the cost of transport from the orchard to the industry [12]. For this reason, the improvement of these machines has been based on a dual objective; to increase the efficiency of mature fruit removal and to reduce the damage caused to trees. In order to improve the mechanical harvesting process, Savary et al. [13] developed a canopy shaker simulation based on finite element methods to predict and evaluate the interaction between the tree and the machine. Then, Savary et al. [14] evaluated the effect of vibration on the tree canopy according to the distribution of forces and accelerations in branches and fruits. Subsequently, machine improvement proposals were based on mathematical models and prototype tests with a combination of machine operating parameters such as frequency and amplitude of vibration [15,16], and the configuration and material of the rods [17]. The rods have been shown to play an essential role in the process of tree shaking, both in terms of fruit detachment and the possible generation of tree and fruit damage. Liu et al. [17] indicated that rod material affected the vibratory response of trees with respect to the acceleration peaks in branches. They recommended that the rods have high stiffness values, but that their surface should be smooth to reduce damage. The shape of the rods revealed that arc-shaped flexible rods had better performance with respect to fruit detachment efficiency and a lower rate of damage to trees than the free end rods [18]. In a further attempt to adapt the vibration process to tree requirements, Pu et al. [18] designed and tested a two-section independent shaker system in order to minimise tree damage and maximise harvesting efficiency. The adaptation of the machine to the variability of the tree canopy made it possible to reduce tree damage compared to other canopy shaker systems.

During the harvesting process, the machine rods penetrate the tree but do not usually reach all parts of the canopy, so vibration must be transmitted via the branches in order to detach the inner fruit. Whitney et al. [19] indicated that a greater fruit detachment efficiency value was obtained by harvesting smaller rather than larger canopy trees. Canopy shaker systems generally have a limited capacity to detach internal fruits from the tree and yet can remove almost all external fruits. The harvesting process of fruit inside the canopy may cause damage to the outermost parts due to a reduction in

machine ground speed or an increment in vibration frequency value and may also require additional manual harvesting.

This study works on the hypothesis that mechanised harvesting processes can be carried out with a high value of harvesting efficiency, where subsequent hand-harvesting is not necessary, and that the possible, but moderate tree damage does not affect the productive life of the orchard. The objective of this work is to analyse the interaction of the canopy shaker rods with the tree branches from the data collected continuously during the mechanical harvesting process of citrus trees. First, the fruit-bearing branches with and without direct contact with the machine rods were characterised in the position and production. Then, the canopy vibration process during mechanical harvesting was monitored. Finally, the vibration parameters necessary to detach the fruit were determined, and recommendations were given on the harvest parameters necessary to increase the fruit removal efficiency.

### **2. Materials and Methods**

Mechanical harvest tests were carried out in Cordoba (Spain) in 2017 during the sweet orange (*Citrus sinensis* L. Osbeck cv. Valencia) harvest season for juice production, during four weeks from the end of flowering to before the natural fall of immature fruitlets in June. Table 1 shows the main characteristics of the four, mechanically harvested citrus orchards. Trees had been planted in wide hedges over 0.4 m ridges and had wide row distances to allow machine manoeuvrability and the use of canopy shaker harvesters (Figure 1).

**Table 1.** Characteristics of citrus orchards mechanically harvested with the canopy shaker system.


**Figure 1.** Example of a tree row trained in width hedge for mechanised harvesting.

Mechanical harvesting was carried out with a lateral tractor-drawn continuous canopy shaker system (Oxbo 3210, Byron, New York, NY, USA), working under regular conditions, with a ground speed range between 1 and 1.5 km/h (0.28–0.42 m/s), and a vibration frequency close to 4.5 Hz, which caused the fruit to fall to the ground. Harvesting tests were carried out to ensure close contact of the shaker system with the tree canopy (Figure 2). The machine harvested both sides of the hedge in independent passes, with an approximate working capacity of 0.4–0.5 ha/h. Subsequently, the fruit remaining in the canopy was hand-picked and collected together with the fruit from the ground and loaded into a container.

**Figure 2.** Lateral tractor-drawn continuous canopy shaker system (Oxbo, 3210) used in citrus harvesting tests. (**a**) front view before the harvesting process; (**b**) rear view after the harvesting process.

Interaction of the harvesting system with the tree canopy was studied for each side of the tree hedges (Figure 3). Because the tree hedges differed between orchards, a representative cross-section of the tree canopy was selected. The cross-section ranged from 4–5 m2, and was composed of a distance from the line of the trunks to the outside of the canopy of up to 2.0–2.5 m, and of a height ranging from 0.5–2.5 m. This cross-section was selected because it differentiated an external area of the canopy with direct contact of the machine rods with the branches, for a rod length of 1.4 m, and an internal area of the canopy without direct contact with the rods. Furthermore, the cross-section was representative of the tree canopy, with high yield, avoiding the effect of the lower pendulous branches and allowing analysis of the vibration process in a homogeneous canopy area between the tested plots. The cross-section was divided into 16–20 sectors according to the width of the hedge, at intervals of 0.5 m both horizontally and vertically. In each sector, the values of vibration, the properties of the branches, and the number of fruits were recorded. The canopy area that had direct contact with the rods was sampled with 10 sectors, while the canopy area without direct contact was sampled with 6 or 10 sectors.

**Figure 3.** Cross-section of the tree hedge and canopy areas with and without direct contact with the canopy shaker system rods.

Before the mechanical harvesting process, a total of 90 fruit-bearing branches were selected that had mature fruit and were distributed in different sectors. The statistical design established a stratified random sampling, each cluster was a tested plot, and in each plot, 18–24 fruit-bearing branches were randomly selected. The sample guaranteed at least three measurements in each sector. Each branch was assigned a position value in the cross-section at a point close to the fruit that was able to support an acceleration sensor, but which had a diameter less than 10 mm. The fruit detachment ratio was determined by the number of fruits removed from each branch before and after the harvesting process.

Branch vibration measurements were recorded with a triaxial MEMS accelerometer sensor (Gulf Coast Data Concepts LLC X200-4, Waveland, MS, USA) with a measurement range of <sup>±</sup>2000 m/s2, a 16-bit resolution, a sensitivity of 0.06 m/s2, and a sampling frequency of 400 Hz. Figure 4 shows the placement of the sensor on the fruit-bearing branch.

**Figure 4.** Location of the acceleration sensor on a fruit-bearing branch on the outermost part of the canopy.

Analysis of the acceleration signals in the time domain and statistical analysis was performed using the R open software (R Core Team, 2016) and in the frequency domain using the NV Gate v8.0 software, with a fast Fourier transformation with 401 lines in a frequency range of 0–156.2 Hz. In the time domain, the resultant acceleration value (Ar) was determined as the module of the vector sum of the three measurement axes in each sensor. Figure 5 shows a sample of Ar in a fruit-bearing branch.

**Figure 5.** Example of the resultant acceleration (Ar) in the time domain measured in a fruit-bearing branch. Tvib\_300—time elapsing between the first and the last event with an Ar value of 300 m/s2; Apk\_n— n maximum peak value of Ar.

In the time domain, the vibration variables studied were:


In the frequency domain, the vibration variables studied were:


The statistical analysis focused on predicting the fruit detachment ratio according to the vibration variables and the branch measurements as the predicted parameters. Logistic regression was used with a K = 2-fold cross-validation method.

### **3. Results and Discussion**

Most of the fruit (72.7%) was located in the canopy area that had direct contact with the rods, in the height range of 1–2 m from the ground, and in a range of 0.5–2 m from the trunk. Gupta et al. (2015) stated that the area with the highest fructification is in the primary branches of the intermediate zone of the canopy at a height of 1.14–2.29 m, and at a distance of 0.78–0.83 m from the canopy exterior. However, the distribution of fruit in the canopy also depends on planting distances and tree height. A reduced distance between trees can generate a greater percentage of fruit in the upper canopy parts but can reduce the number of fruits inside [19].

The diameter of the branch at the vibration measurement point was 7.9 ± 2.4 mm (mean ± sd), with a variation that ranged from 10.2 mm for the branches closest to the trunk and the ground to 5.53 mm for the outermost and tallest branches of the canopy. Each tested branch carried a mean value of 3.7 ± 1.8 fruits. The results showed a high variability in the distribution of fruit in the canopy and in the morphology of the branches. This variability is important for the outcome of mechanical harvesting systems and was considered by Gupta et al. [15,16] for modelling the tree and simulating the harvesting process in order to improve the canopy shaker system. However, the current harvesting systems based on canopy shakers do not contemplate the variability of branches and fruit within the tree canopy. In an attempt to improve the machine adaptation to the tree, Pu et al. [18] designed and tested a canopy shaker system capable of applying different vibration parameters to the upper and lower parts of the tree. These authors showed the need to use different harvesting parameters and were able to achieve a high fruit detachment ratio (82.6%) with low tree damage.

Canopy shaker systems continuously harvest fruit as they move along the row of trees. Before the machine comes into contact with a branch, the branch may vibrate due to contact with other branches or due to the transmission of vibration from the trunk. Then, the branch vibrates due to contact with the machine rods, and finally, the branch vibrates freely when the machine has passed. Table 2 shows the results of this vibration process measured in branches, both in the canopy area with and without direct contact with the machine rods. In order to define the beginning and end of the vibration process, the acceleration values produced only by natural sources, mainly by wind and gravity, and without machine interaction were recorded. The vibration time where the branch was excited by the machine was defined as the time elapsing between resultant acceleration values greater than 18 m/s2 (Tvib\_18). The average vibration time (Tvib\_18) of the branches was 14.3 ± 2.8 s. No significant differences were found (Student's *t*, *p* > 0.05) between the vibration time of the branches located in the canopy area with or without direct contact with rods. This indicated that all branches vibrated at the same time, but not all at the same level of acceleration. The length of the machine's shaking system, i.e., the length of the rods and drums, together with the machine ground speed, determined the vibration time during which the rods had direct contact with the branches, whereas tree-training and canopy density could define the vibration time during which the branch vibrated without direct contact with the rods. For the field

test conditions, i.e., a rod length of 1.4 m and ground speed ranging from 1–1.5 km/h (0.28–0.42 m/s), the canopy shaker systems were able to maintain direct contact with the branches within the range of 7.7–11.5 s. This indicated that between 20% and 46% of the vibration time may correspond to the transmission of vibration before and after the passage of the machine.

**Table 2.** Vibration parameters measured on branches with and without direct contact with the rods during mechanical harvesting with the canopy shaker systems.


Values shown are mean ± standard deviation, n = 90. The same superscript letters in the same row are not significantly different (Student's *t*, *p* < 0.05; Wilcoxon–Mann–Whitney test, *p* < 0.05).

During the harvesting process, the branches in contact with the rods experienced a process of forced vibration. The rods, powered with an alternating and rotating movement, penetrated the canopy, producing an impulse excitation in the branches of the fruit of the outermost part of the canopy [20]. The branches showed a mean vibration frequency value of 4.1 ± 0.5 Hz. The value of the vibration frequency did not correlate with the position of the branch in the canopy (Pearson = 0.135, *p* > 0.05). The frequency value used was within the range recommended for citrus detachment with canopy shakers. Liu et al. [17] established that the value of 5 Hz was appropriate for the detachment of fruit without increasing the damage caused to the tree. Similarly, it was demonstrated that a frequency value of 4.8 Hz was adequate to produce a high acceleration in the branches when using rigid rods [18]. The frequency of the vibration in combination with the design of the rods used and the state of the immature fruits [10] play an important role in the damage caused to the trees. Although tree damage is an important element in the process of improving mechanised harvesting, it has not been considered in this study because the damage that occurred was minor and similar to that of previous years, mechanically harvested plantations had not shown any problems in tree development or yield compared to previous seasons.

The values of ARMS measured in the branches was positively related to the fruit detachment ratio. The branch vibration was characterised by an ARMS value of 26.5 <sup>±</sup> 13.6 m/s2 for the machine vibration frequency. However, there was a significant variation in the mean ARMS values in branches depending on their position in the canopy (Figure 6). Branches with direct contact with rods showed a significantly higher mean ARMS value (29.6 <sup>±</sup> 10.2 m/s2) (Student *t*, *p* < 0.05; Wilcoxon–Mann–Whitney, *p* < 0.05) than branches without direct contact (12.8 <sup>±</sup> 6.4 m/s2). Pu et al. (2018) showed that the highest values of acceleration in the branches (31.4 m/s2) were provided by contact with the machine rods and these branches reached the highest values of fruit removal efficiency.

During the harvesting process, the acceleration produced in the branch was transmitted to the fruits for their detachment. The internal branches, without direct contact with the rods, showed 43% of the ARMS vibration level of the external branches. A similar result was achieved by Liu et al. [17], whose results showed a reduction of the acceleration in the inner branches of the canopy of 42% with respect to the outer branches. In field trials with measurements inside the fruit before its detachment with canopy shaker systems, Castro–Garcia et al. [20] recorded mean ARMS values between 38.8 and 51.3 m/s2. These values indicated that there was an amplification of the acceleration values from the branch to the fruit. Savary et al. [13] evaluated the acceleration produced in the branches and indicated that the resultant acceleration values were higher at the ends of the branches, especially on the thinnest and outermost branches. These same authors pointed out that the trunk region of unbranched trees had very low acceleration values, whereas the acceleration values began to be noticeably higher from the first branch of the trunk region.

**Figure 6.** (**a**) Distribution of the ARMS values (m/s2); and (**b**) fruit detachment rate (%) produced with a canopy shaker system and measured in fruit-bearing branches in the cross-section of the tree canopy.

The interaction of the rods with the branches was characterizsed by a succession of impacts with a high acceleration value in accordance with the vibration frequency of the machine. These impacts presented a mean Apk value of 495.1 <sup>±</sup> 270.9 m/s2. Similarly to Figure 6, the Apk values in branches with direct contact with rods (616.7 <sup>±</sup> 283.3 m/s2) were higher (Student's *<sup>t</sup>*, *<sup>p</sup>* <sup>&</sup>lt; 0.05) than branches without direct contact (268.1 <sup>±</sup> 164.6 m/s2). The Apk and ARMS values showed a positive linear correlation (Pearson = 0.70, *p* < 0.05) in the tree canopy. In both cases, the direct contact of the rod represented an increment of 2.3 times the acceleration values reached in the branch.

The canopy shaking system achieved a mean fruit detachment value of 69.1 ± 40.7%. However, this variable presented high variability within the cross-section of the tree hedge (Figure 6). As expected, this variable reached its highest values in the branches with direct contact with rods. The fruit detachment ratio was reduced from an average value of 84.7 ± 30.5% in branches with direct contact with rods to 25.1 ± 22.2% for branches without direct contact. The fruit detachment value of 100% was reached in all branches located between 2 and 2.5 m in height. A similar result was reported by Whitney et al. [6] who found that working with small canopies that were accessible to rods achieved a fruit detachment of 96%, while wider canopies achieved reduced values of 55%. Savary et al. [14] reached a fruit detachment value of 88% on the outside of the canopy, while on the inside this figure was reduced to 57%. However, in order to improve the harvesting efficiency, it is not only necessary for the rod to penetrate the canopy, but also for it to interact with the branch. Liu et al. [17], analysing fruit detachment according to the point of contact of the rod with the branch, determined that the operation was more effective when the rod impacted at 30% of the distance to the free end of the branch.

The results obtained from the cross-section of the tree canopy have shown a high variability, both in vibration and in fruit values. Reducing this variability and improving the mechanised harvesting process requires knowledge of the requirements to detach fruit from the tree. The measurements in the canopy showed a high linear correlation between variables, which indicated that fruit detachment prediction could not have a single solution. Analysis of the data focused on the discretisation of quantitative variables that could discriminate whether there was a fruit detachment with a value of 100% and reasonable success. Due to its simplicity and efficiency, logistic regression was used. ARMS and time elapsed between an acceleration greater than 300 m/s2 (Tvib\_300) were significant variables to discriminate the events of 100% fruit detachment. The result was defined as a straight line that separates the conditions at which a fruit detachment of 100% was obtained with a precision measured as the area under the ROC curve of 0.95 in the validation set. Equation (1) shows the values obtained and Figure 7 is the graphical representation.

$$\ln\left(\frac{\text{Prob of complete fruit detected}}{1-\text{Prob of complete fruit detected}}\right) = 7.13417 - 0.52754 \, T\_{\text{vib}\_{30}} - 0.17206 \, A\_{\text{RMS}} \tag{1}$$

**Figure 7.** Distribution of the fruit detachment rate values according to ARMS (m/s2) and Tvib\_300 (s).

Currently, canopy shaker systems for citrus harvesting can employ various types of vibration systems, with variations in the frequency or amplitude of movement, different machine ground speeds or rods with different designs, or mechanical properties. In all cases, the machine produces a forced vibration of the branches with the aim of detaching fruit and avoiding major damage to the tree or fruit. Monitoring of the forced vibration process of the tree canopy showed that it was possible to achieve a 100% fruit detachment ratio based on a combination of acceleration levels and vibration times in branches (Figure 7). Under the conditions of the field tests performed, we propose a combination of a vibration time of at least 5.8 s, after the interaction of the rod with the branch (Tvib\_300), together with an ARMS value of 23.9 m/s2. With these harvest parameters, a complete process of fruit detachment was achieved in 88.9% of the branches tested. Although these values could be modified if another type of tree formation, citrus variety, or harvesting machine was considered, the fruit detachment process can be estimated by vibration time and branch acceleration. Both parameters are of great importance, not only for training trees to facilitate the rod penetration in the canopy but also for the design of new canopy shaker systems.

### **4. Conclusions**

The monitoring of the vibration process in the tree during mechanised harvesting with the canopy shaker showed a great variability in results depending on different parts of the canopy. Branches that had direct contact with the machine rods showed a higher mean value of fruit detachment ratio (84.7%) than non-contact branches (25.1%). Vibration transmission from the external branches to the internal branches in the canopy was not effective to remove internal canopy fruit. During the harvesting process, values of 100% fruit detachment ratio could be achieved with a combination of harvest parameters in the branch. Achieving a complete process of fruit detachment is possible if both the tree canopy and machinery are mutually adjusted to facilitate the contact of the shaking system and the necessary vibration time.

**Author Contributions:** All the authors made significant contributions to the manuscript. S.C.G., F.A.-R., and R.R.S.-G. designed the experiment and wrote the manuscript; S.C.G. and F.A.-R. performed the data acquisition; F.A.-R. and R.R.S.-G. performed the field component; E.S.-O. and A.J.S. performed the statistical analysis; J.A.G.-R. and A.J.S. scientifically supported and reviewed the paper; and S.C.G and E.S.-O. managed the project.

**Funding:** This research was funded by National Institute for Agricultural and Food Research and Technology (INIA, Spain) financed by FEDER funds, from research projects RTA2014-00025-C05-03 and RTA2014-00025-C05-05.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
