A Soft Gripper Design for Apple Harvesting with Force Feedback and Fruit Slip Detection

Abstract

This research presents a soft gripper for apple harvesting to provide constant-pressure clamping and avoid fruit damage during slippage, to reduce the potential danger of damage to the apple pericarp during robotic harvesting. First, a three-finger gripper based on the Fin Ray structure is developed, and the influence of varied structure parameters during gripping is discussed accordingly. Second, we develop a mechanical model of the suggested servo-driven soft gripper based on the mappings of gripping force, pulling force, and servo torque. Third, a real-time control strategy for the servo is proposed, to monitor the relative position relationship between the gripper and the fruit by an ultrasonic sensor to avoid damage from the slip between the fruit and fingers. The experimental results show that the proposed soft gripper can non-destructively grasp and separate apples. In outdoor orchard experiments, the damage rate for the grasping experiments of the gripper with the force feedback system turned on was 0%; while the force feedback system was turned off, the damage rate was 20%, averaged for slight and severe damage. The three cases of rigid fingers and soft fingers with or without slip detection under the gripper structure of this study were tested by picking 25 apple samples for each set of experiments. The picking success rate for the rigid fingers was 100% but with a damage rate of 16%; the picking success rate for soft fingers with slip detection was 80%, with no fruit skin damage; in contrast, the picking success rate for soft fingers with slip detection off increased to 96%, and the damage rate was up to 8%. The experimental results demonstrated the effectiveness of the proposed control method.

1. Introduction

Harvesting is an important element of orchard production since it has a brief window period, high labor intensity, and high labor volume. The high labor cost in the harvesting stage limits the fruit industry’s development. With this backdrop, fruit-picking robots have become a hotspot for study in related fields [1,2]. Researchers have completed several projects and made significant progress in important technologies such as robot perception and positioning [3,4], system integration [5], and efficient harvesting end effector design.

As a critical step in robotic harvesting, grasping determines the picking effect directly. During harvesting, the traditional robotic rigid clamping mechanism has issues: high requirements for fruit positioning [6] and easy damage to the apple pericarp [7,8]. In practical applications, it not only required the grippers to be dexterous, light, stable, and reliable to grasp but also to ensure that the appearance of the fruits is not damaged, to prevent harming commerciality. As a result, research on non-destructive harvesting end grippers for safe, reliable, and stable gripping is an important topic for harvesting robots with a promising application.

To lower the fruit damage rate, the soft gripper technology is attracting more and more researchers’ attention. Some researchers [9,10,11] used soft materials on the surface of the fingers to increase the gripper flexibility and, hence, prevent damage to grabbed objects. However, due to the rigid support of the fingers’ main body, it is also easy to cause different degrees of damage to the fruit pericarp. Furthermore, the structure is more complex, and the grasping stability is insufficient.

The soft structure gripper has a high adaptability, wide range of variability, and excellent working ability for gripping objects that are susceptible to damage [12,13].

Shepherd et al. [14] proposed the PneuNet (pneumatic mesh) structure, a bending multi-cavity pneumatic soft actuator. The soft gripper [15,16,17] designed by Whiteside’s group has the characteristics of minimal pressure bearing, large deformation, and flexible movement. However, the end contact force is limited, and the stability is insufficient when grasping objects. A vision-equipped six-finger soft harvesting gripper [18] can identify the type and maturity of fruits and vegetables, and it can softly grab fruits and vegetables based on their shape but only for tiny fruits. Muscato et al. [19] created a soft citrus harvesting gripper out of spirally organized rubber sheets that had a strong wrapping capacity for gripping things but that lacked rigidity.

German bionics researcher Leif Kniese accidentally discovered the “Fin Ray effect” in 1997 [20], which was later widely employed in the study of robotic soft grippers [21,22]. Fin Ray soft fingers are highly compliant and can take greater loads than other soft constructions. Thanks to its superior grabbing stability, the Fin-Ray-effect-inspired grippers have received extensive attention from researchers.

However, the basic finger structure is not optimal for soft grippers, and studies have recently increased the gripping force by improving the finger structure [23,24,25,26]. Crooks et al. [23] proposed a multi-material structure gripper with a higher grabbing weight, but the fabrication method for this multi-material structure is quite tricky. Basson et al. [24] varied the slope and curve of the cross beams in a Fin Ray finger and analyzed the stress and displacement on the improved finger through simulation. However, the effects of other variables have not been fully tested. Shin et al. [25] analyzed the changes in stress and displacement when the finger touched an object by varying the number of cross beams, the front beam slope, and the slope of the cross beams. Elgeneidy et al. [26] developed a soft finger that could handle fragile objects by varying the angle and number of cross beams. Nevertheless, whatever structure maximizes the Fin-Ray finger gripping force while causing no damage to the object has yet to be determined.

Although it can greatly avoid fruit damage due to grasping by using the soft fingers, it is not sufficient to rely solely on the soft structure to ensure the gripper’s lossless grasping. The gripper’s lack of a force feedback system makes it unable to collect the contact state information between fingers and gripping items, which may cause damage due to excessive gripping force or slippage owing to insufficient gripping force.

Some researchers added force sensors to the fingers of soft grippers [27,28,29,30,31]. The sensing system is simple, but the sensor deforms with soft fingers, which has a great influence on the accuracy. When directly embedding force sensors through the manufacturing process but the cost is large and the universality is low due to its sophisticated driving scheme and manufacturing method [32,33]. Some researchers [34,35,36] estimated the contact force by substituting the force perception model from finger deformation by vision. Belzile et al. [37] used the quasi-static analysis method to calculate the contact force generated by the gripper, which realizes the internal force perception without the use of additional force sensors, but the solution process and control algorithm are complex.

In addition to preventing fruit damage due to excessive gripping force, slip detection is also an important factor due to the rough surface of the fingers [38,39]. Some studies use multi-axis or more force sensors to monitor the static friction coefficient between the finger surface and the object [40,41] or to detect vibration caused by sliding between the two contact surfaces using piezoelectric phenomenon [42], time–frequency conversion technique [43], or filtering [44] to accomplish slip detection. However, the sensors are dependent on the working environment, and utilizing more sensors to gather more tactile information would not only dramatically raise the cost but will also place a significant load on the gripper structure and control system. Some recent studies employ tactile data for training, and neural networks can predict item sliding [45,46], as well as physical parameters such as temperature, electromagnetism, light intensity, and acceleration to predict slippage [47]. Liu et al. [48] introduced a novel design of the GelSight Fin Ray gripper, which used a vision-based tactile sensor for tactile reconstruction, orientation estimation, and slip detection. But it is difficult to grasp heavier objects due to the design of its hollowed-out finger. Nonetheless, these technologies are rarely used on harvesting grippers.

To solve the above problems, this work proposes a novel soft harvesting gripper with flexible adaptive envelope, force feedback, slip detection, and other features. To design the Fin-Ray finger structure in such a way that the gripping force is high enough to ensure it is sufficient to successfully separate fruits from stems, the influence of various parameters of the Fin Ray structure on the gripping force and deformation of the finger was investigated through simulations, as the basis for the design of the soft gripper structure. The following are the main contributions:

(1)

A new three-finger force feedback soft gripper for the apple harvesting robot is proposed. The relationship between the gripping force, the pulling force, and the servo torque was established to achieve the constant-pressure flexible clamping of fruits. Then the sensing system of the soft gripper was implemented by using the servo’s feedback information instead of adding additional sensors, making the structure of the gripper simpler and less costly.

(2)

A force feedback gripper dynamic control approach with slip detection is presented. The relative location of the fruit and the gripper is detected in this manner by incorporating a distance sensor, which makes the gripper structure and calculation simple. When the fruit slippage occurs, the servo output torque is adjusted in real time to reduce fruit harm using the feedback information.

In addition, to provide a theoretical basis for the design of the gripper, some mechanical properties of apples are given in the experiments.

Remark 1.

It should be clarified that the force feedback system and slip detection are two main contributions in this paper. To provide a stable mechanical design of the gripper as a study basis for these two points, we also analyze the structural parameters of Fin Ray fingers by the finite element analysis method.

2. Structural Design of a Soft Gripper with Three Fingers

2.1. Finite-Element Analysis of Finger Structure with the Fin Ray Effect

The harvesting gripper’s finger mechanism uses a triangular Fin-Ray soft finger component, which has a passive compliance quality and can implement an envelope while clamping spherical items. The general construction of the finger consists of the front and rear beams, cross beams, and base, as shown in Figure 1a. The front beam comes into contact with the fruits, and the front and rear beams are linked by cross beams. These cross-beam support rods are the foundation of Fin Ray fingers. Because of the presence of these crossbeam support rods, the Fin Ray structure can withstand greater loads than conventional flexible constructions.

2.1.1. Pre-Preparation of the Simulation Experiment

The finger gripping force must be sufficient to improve the grasping stability. Furthermore, the pressure per unit area of the pericarp should be small enough to guarantee that the fruit pericarp remains intact. As a result, the finger gripping force and the bending degree are two critical criteria. The finger gripping force can ensure clamping stability, while the finger bending degree can assure clamping stability and safety by increasing the contact area between the fingers and the fruits. The stress of the Fin Ray finger during deformation is complicated by making the mathematical modeling difficult. As a result, using the simulated tests, this research investigates the effect of the front and rear beam thickness, the finger width, and the number of cross beams on the finger gripping force and bending degree, as shown in Figure 1a. In the simulation experiment, the contact stress between the finger and apple is used to characterize the gripping force, and the displacement of the fingertip is used to characterize the bending degree, as shown in Figure 1b.

A single finger adopts a symmetrical structure; the total length of the finger is 120 mm, and the front beam and the rear beam are each at an angle of 80° to the base. The cross beams are parallel to the base; the distance is equal, and the thickness of the cross beams is 1.40 mm. The little bulges are designed on the cross beams to increase the rigidity and strengthen the load capacity, as shown in Figure 1c.

The TPU 95A [49] was chosen as the finger material. The TPU soft material is a hyperelastic nonlinear material with isotropic properties throughout the stress process. Furthermore, because the bending deformation of the soft finger is a nonlinear large deformation, the Yeoh model can better represent its material properties [50]. The strain energy density function W can be written as follows:

$$W={\displaystyle \sum _{i=1}^N{C}_{i0}}{\left(I_1-3\right)}^i+{\displaystyle \sum _{k=1}^N\frac{1}{D_k}}{\left(J-1\right)}^{2k},$$

(1)

where N is the order of the model; I₁ is the deformation tensor; C_i0 and D_k are the material constants; J is the volume ratio. When TPU is regarded as the incompressible material, J = 1.

The strain energy density function in the form of the binomial parameters is usually used [51], and the typical binomial parameter form of the Yeoh model is

$$W=C_{10}\left(I_1-3\right)+C_{20}{\left(I_1-3\right)}^2.$$

(2)

The fitting curve of the stress and strain of the TPU 95A was obtained through the uniaxial tensile test, as shown in Figure 2. The material parameters obtained after processing and analysis are shown in

Table 1. Mechanical property parameters of materials.

Materials	Density (kg/m3)	Young’s Modulus (MPa)	Poisson’s Ratio	C10 (MPa)	C20 (MPa)
PA12	1010	1900	0.4	—	—
Apple [52]	840	5	0.35	—	—
TPU 95A	1200	—	—	3.7358	−11.88

.

Because the contact stress between the three fingers and the fruit is the same, the contact between a single finger and the fruit can be considered to reduce the quantity of simulation calculation, to simplify the analysis.

During the simulation, the center of the bottom plate of the gripper is kept aligned with the center of the fruit at a distance of 65 mm [49].

2.1.2. Influence of Geometric Parameters on Contact Stress and Fingertip Displacement

Each geometric parameter has a varied effect on the contact stress and fingertip displacement. All other parameters were held constant to compare their changes when the given parameters were altered, and the influence of the given single parameter on them was gradually optimized.

First, the influence of the thickness of the front and rear beams was analyzed. The stress increases dramatically as the thickness increases, while the fingertip displacement decreases, as shown in Figure 3a.

When the thickness of the front and rear beams is 2 mm, the stiffness of the finger after contact with the apple cannot be guaranteed, resulting in a small gripping force and easy fruit slip; when the thickness is 4.5 mm, the stress of the material itself will greatly limit its bending deformation and reduce the contact area between the fingers and fruit. At the same time, because excessive stress might cause fruit damage, the thickness of the front and back beams should not be too tiny or too large. When the thickness is 3.5 mm, the downward trend of the fingertip displacement becomes stronger as the thickness increases, while the upward trend of stress tends to be soft. As a result, selecting a thickness of 3.5 mm for the front and rear beams not only meets the requirement of the increasing gripping force but also allows fingers to make good contact with the fruits.

The effect of the finger width was then investigated. With the increase of the width, the fingertip displacement diminishes. However, the stress does not follow a constant pattern, as shown in Figure 3b. When the width is 10 mm, the stress and fingertip displacement is the greatest. This is because the finger width is excessively narrow, resulting in a limited contact area between the finger and the apple and high contact stress acting on the apple surface, which is easily damaged. Although the degree of the finger bend is greater when the finger is thin, it also results in insufficient grasping stiffness and fruit slide. When the width is 25 mm, the contact area between fingers and fruit increases, but its structure affects its bending, and it is not suitable for collecting fruits in the complex growing environment. When the width is 16 mm, as the width continues to increase, the fingertip displacement decreases dramatically and the stress tends to be flat. As a result, the best finger width is set to 16 mm in this study.

Finally, the number of beams was taken into account. Because the cross beams are the primary components that influence the stiffness of fingers, the number of cross beams has a substantial impact on the Young’s modulus of the fingers [25]; hence, the distribution of the cross beams may have a major impact on the gripper performance. In distribution, there are several combinations of the cross beams. For the sake of simplicity, just the simplest equidistant parallel arrangement of the cross beams was considered in this study. Change the thickness of the front and rear beams to 3.5 mm, the width of the fingers to 16 mm, and change the number of cross beams. As the number of cross beams grows, so does the stress, and the fingertip displacement declines first and subsequently increases. When the number of beams is 9, the fingertip displacement reaches the maximum and then decreases again, as shown in Figure 3c. As a result, one selects nine as the optimal number of beams.

According to the results of the aforementioned analysis, the thickness of the front and rear beams has the greatest influence on the contact stress and fingertip displacement among the three geometric parameters. It is mostly because the finger surface is in direct contact with the fruit, and the thickness of the front and rear beams has a direct impact on the stiffness of the fingers. The structural parameters of the Fin Ray fingers are extremely complex, and this study just considers the most basic scenario. As a result, the best structural parameters are as follows: the thickness of the front and rear beams is 3.5 mm, the width of the fingers is 16 mm, and the number of beams is 9.

2.2. Overall Design of the Soft Gripper

The overall structure of the three-finger soft gripper for apple harvesting built with optimized Fin Ray fingers is shown in Figure 4a. It can be divided into three parts: the driving and sensing part, the transmission part, and the grasping part for clamping objects. The driving part is performed by a servo with torque and position feedback. To measure the relative distance between the gripper and the fruit, a distance sensor is mounted on the servo installation side of the gripper bottom plate. The transmission part is primarily accomplished by a slider, and the rocker mechanism was composed of a rocker, a connecting rod, a moving plate, and guide rods, as shown in Figure 4b. The servo rotates to drive the moving plate to move up and down. Because the fingers and their connectors are connected with the moving plate through the support rods, the fingers will move with the moving plate moving up and down, as shown in Figure 4c.

In the grasping part, three Fin-Ray finger units are evenly distributed around the bottom plate of the gripper disc, connected with the transmission mechanism by the finger connectors to drive the Fin Ray fingers. A silicone pad is attached to the surface of each finger to increase the contact friction between the finger and the fruit, which ensures the clamping stability, as shown in Figure 4d.

At the initial position, the finger connectors are inclined outward at a certain angle relative to the bottom plate. Because the bottom of the fingers is connected in parallel with the bottom of their connectors, and the finger has a triangular symmetrical structure, the clamping range of the gripper is expanded.

3. Kinematic Mechanics Analysis of a Soft Gripper

The driving force begins with the servo, travels through the slider and rocker mechanism, multi-link mechanism, and Fin Ray soft structure, and eventually acts on the gripped fruit. In conclusion, the static analysis of the rigid multi-link mechanism and the soft finger structure was performed to acquire the gripping force on the fruit surface. Simultaneously, the relationship between the gripper pulling force and the gripping force was investigated in connection with the pulling harvesting method. Because the three fingers are symmetrically arranged relative to the bottom plate of the gripper, and the structure is the same. Furthermore, the servo output torque operates on the center of the moving plate, and the movement process and stress situation are comparable. As a result, the stress analysis of the direct contact between fingers and fruit begins with a single finger, making the analysis procedure simpler.

3.1. Force Analysis of Rigid Multi-Link

The basic structure and motion principle of the soft gripper is shown in Figure 5a,b. The force acting on the fruits of the Fin Ray structure can be equivalent to a single concentrated force in the analysis of the rigid multi-link (the analysis of the soft Fin Ray structure will be discussed below). The servo drives the rocker to rotate counterclockwise when grabbing, the moving plate to travel down along the guide rod, and the support rod to move. Following that, the support rod drives the finger connector to rotate around the joint FF, resulting in the envelope-gripping movement of the finger.

In the figure: M_d is the servo output torque; θ_FR is the angle between the rocker and the horizontal direction; θ_FF is the angle between the finger connector and the horizontal direction; α is the angle between the front and rear beams of fingers and the base.

Because of the light weight of each moving part of the rigid multi-link, the gravity and inertia force during the movement of the gripper can be ignored.

The mechanical analysis of the multi-link mechanism is performed under static equilibrium conditions. The connecting rod is vertical to the moving plate at the time of initial contact. Their angle does not alter much when the rocker is rotated. To make the calculation easier, the difference is negligible. Among the multi-link, the connecting rod is a two-force member, and the moving plate is employed to assess the force, as shown in Figure 6. Therefore, one has

$$F_{CM}=F_{MS},$$

(3)

For which, F_CM and F_MS are in the opposite direction. F_XY is the force of member X applying to member Y. To simplify the analysis, the sliding friction between the moving plate and the guide rod is negligible.

Thus, the support rod is a two-force member. Figure 7a shows the force analysis of the finger and its connector. The closing force triangle shown in Figure 7b can be obtained according to the geometric conditions for the equilibrium of the plane-intersecting force systems.

To maintain the force balance of the finger and its connector, one obtains

$$F_c\cos \gamma =F_{FF}\cos \beta ,$$

(4)

$$F_c\sin \gamma +F_{FF}\sin \beta =F_{SF},$$

(5)

where F_c is the contact reaction between the finger and fruit, that is the finger gripping force; γ is the angle between F_c and the horizontal direction; β is the angle between F_SF and the horizontal direction.

According to Equations (4) and (5),

$$F_{\mathrm{c}}=\frac{1}{\sin \gamma +\cos \gamma \tan \beta }{F}_{SF},$$

(6)

where

$$\gamma =\frac{\pi }{2}-\alpha +{\theta }_{FF},$$

(7)

$$\tan \beta =\frac{h}{L_{Fc}\cos {\theta }_{FF}{\sin }^2\left(\alpha -{\theta }_{FF}\right)}-\frac{1}{\tan \left(\alpha -{\theta }_{FF}\right)},$$

(8)

where h is the distance from the center of fruit to the bottom plate of the gripper; L_X is the length of component X, that is L_Fc is the length of the Finger connector, and L_R is the length of the rocker.

To obtain the relationship between the servo torque M_d and the gripping force F_c, the rocker is taken as the forced object, and the force situation is shown in Figure 8. The moment balance at joint FR is

$$F_{CR}{L}_R\cos {\theta }_{FR}=M_d.$$

(9)

According to the force characteristics of the two-force members,

$$F_{CR}=F_{MS}=F_{FS}=F_{SF},$$

(10)

where F_FS and F_SF are in opposite directions.

From Equations (6), (9), and (10), one can obtain

$$F_{\mathrm{c}}=\frac{1}{\sin \gamma +\cos \gamma \tan \beta }\cdot \frac{1}{L_R\cos {\theta }_{FR}}\cdot M_d.$$

(11)

3.2. Contact Force Analysis between Soft Finger and Fruit

When the finger comes into contact with the fruit, it creates an adaptable envelope, and the contact area expands. The flexible deformation of the Fin Ray structure makes the mechanical analysis difficult. Therefore, to facilitate the calculation, the fruit is simplified as a regular sphere. Aiming at the picking method for pulling fruits, a simplified single-finger plane stress model is given in Figure 9.

The contact between the finger and the fruit is divided into two areas with angles of δ and σ, with the y-axis as the limit. The positive touching pressure of the fruit is simplified as a uniform load; the size is n; the unit is N/m, and the directions all point to the center of the circle, whose angle with the y-axis is φ_i (i = 1, 2, …, m). F_s is the static friction force generated by the positive pressure of the finger on the fruit. When pulling the fruit, the positive pressure on the fruit and the component force of the static friction force generated along the x-axis direction are the main forces to ensure the stability of grasping. Specify that the direction of the force is positive along the positive x-axis. To obtain the resultant force in the x-axis direction F, one has

$$\begin{array}{c}F={\displaystyle {\int }_{-\delta }^{\sigma }\left(F_s\cos \phi +n\sin \phi \right)}\cdot rd\phi \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=r\cdot \left[F_s\left(\sin \delta +\sin \sigma \right)+n\left(\cos \delta -\cos \sigma \right)\right],\end{array}$$

(12)

where

$$F_s=\mu \cdot n.$$

(13)

In the Equation, μ is the maximum static friction coefficient between the finger and the pericarp of the fruit; r is the radius of the fruit.

Therefore, from Equations (11) and (12), the relationship between the resultant force F and the positive touching pressure on the fruit can be obtained,

$$F=rn\cdot \left[\mu \left(\sin \sigma +\sin \delta \right)+\left(\cos \delta -\cos \sigma \right)\right].$$

(14)

The relationship between the equivalent single concentrated force F_c and the uniform load n in the rigid multi-link force analysis above is

$$F_{\mathrm{c}}={\displaystyle {\int }_{-\sigma }^{\delta }n\cdot rd\psi =nr\cdot \left(\delta +\sigma \right)}.$$

(15)

According to Equations (11), (14), and (15), the relationship between the servo torque M_d and the resultant force F can be obtained as

$$F=\frac{\mu \cdot \left(\sin \sigma +\sin \delta \right)+\left(\cos \delta -\cos \sigma \right)}{L_R\cdot \left(\sin \gamma +\cos \gamma \tan \beta \right)\cdot \cos {\theta }_{FR}\cdot \left(\delta +\sigma \right)}\cdot M_d.$$

(16)

4. Soft Gripper Control Method for Slip Detection and Constant-Pressure Feedback

During the actual grasping, the gripping force is f_c, which is the same magnitude as the force F_c but in the opposite direction, and the pulling force is the resultant force in the x-axis direction F. From Equations (11) and (16), the relationship between the gripping force f_c of the gripper, the pulling force F, and the servo torque M_d can be obtained, as shown in Figure 10a. Therefore, when the fruit detachment force F_d is determined, the driving torque required for fruit detachment can be calculated according to the diameter of the fruit, thereby setting the servo output torque M_d. Simultaneously, it is possible to conclude that the gripping force f_c on the fruit surface at this time. To ensure constant pressure acting on the surface of the fruit, f_c should not be greater than the maximum pressure F_m that the pericarp of the fruit can withstand.

In addition to the fruit damage caused by the excessive gripping force of the fingers, which also includes bruises and scratches caused by the relative sliding between the fruit and the fingers, as shown in Figure 11. Therefore, to avoid the slippage between the fruit and the finger during harvesting, this paper detects the relative position between the fruit and the gripper by integrating a distance sensor to assess the fruit slippage and minimizing the damage caused by fruit slippage during harvesting, as shown in Figure 10b.

Combined with the constant-pressure feedback state, the specific implementation steps of the soft gripper control method for slippage detection are as follows:

4.1. Control Method of Constant-Pressure Feedback

The required servo output torque M_d can be obtained by identifying the diameter of the target fruit. To ensure that f_c is less than F_m at this time, the output torque must be adjusted further. When the f_c obtained at this time is greater than F_m, it should be ensured that the maximum torque can be output while the fruit is safely held. From Equation (11), let f_c equal F_m at this point to obtain the critical torque M_m of safe clamping, which is set as the servo’s output torque. The gripper control method of the constant-pressure feedback is shown in Figure 12.

During the no-load closing motion of the gripper, the servo output torque is stable at an initial torque. When the feedback torque of the servo is greater than the initial torque, the finger and the fruit seem to be in contact. To avoid fruit damage due to the impact of the dynamic load, close the gripper quickly to reach the contact position before touching the fruit, and slowly close the gripper after the finger is in contact with the fruit. When the feedback torque reaches the preset value, it is assumed that the fruit has been grasped, and the servo stops rotating.

In contrast to the sensor system embedded in the finger, the servo with feedback information is used as the driver to ensure constant-pressure contact between the finger and the fruit, simplifying the structure of the soft harvesting gripper and facilitating the fruit harvesting in complex growth environments.

4.2. Control Method of Slip Detection

Fin Ray soft fingers have great advantages in dealing with the problem of fruit unilaterally damaged by extrusion. The cross beams act as rigid support rods to ensure the stiffness of the fingers while also allowing the fingers to adaptively wrap the entire fruit, preventing fruit damage due to the stress concentration.

However, it is difficult to ensure that relative slippage between the fingers and the fruit does not occur during the fruit detachment process. Because of the rough silicone pads attached to the surface of the fingers, the sliding friction force between the fingers and the fruit is relatively great when there is relative slippage between them, and it is easy to cause bruises and scratches on the fruit pericarp. As a result, effectively avoiding relative slippage is essential to ensure that the fruit is not damaged. The condition of the relative slippage, which causes the fruit damage, is complicated and will not be discussed in this paper.

A slip detection method is proposed for the designed soft gripper, which obtains the fruit position in real-time through the distance sensor. One believes that when the relative slip distance between the fruit and the fingers ΔL reaches L_s, the fruit tends to slip off, as shown in Figure 13. At this time, the output torque can be increased on the premise of ensuring that the maximum gripping force F_m is not exceeded, and the fruit can be clamped to prevent further sliding; if the relative slip distance ΔL can still reach L_s after increasing the output torque, clamping and pulling the fruit will increase the risk of damage, such as bruises and scratches. It means that the fruit is difficult to harvest at this point, and it is considered a harvesting failure, and the soft gripper is released. Controlling the gripper to perform the aforementioned operations n times, if harvesting fails all n times, give up picking this fruit. The slip detection control method is shown in Figure 14.

Although the risk of harvesting failure is increased by the method proposed above, it does not cause damage to the fruit, and the fruit after harvesting failure can still be harvested manually without affecting its economic value or reducing economic losses.

5. Test and Analysis

5.1. Test Analysis of the Mechanical Properties of Apple

The Model E43 of MTS Exceed^® Electromechanical Test Systems was used to conduct the relevant tests to obtain the relevant mechanical properties of the apples as the basis for the design of the gripper in this study. The range is 100 N, and it has a force and displacement sensor. Yantai Fushi apples were chosen as the test samples during the experiments.

In our study, a silicone pad is attached to the surface of the finger to improve the grasping performance by increasing the friction of the fruit’s surface. To measure the maximum static friction coefficient μ between the silicone pad and the fruit, the pressure F_n was applied to the fruit through Model E43, and a silicone pad was pasted on the upper indenter and lower support, respectively. The tensile force of horizontally pulling the fruit was measured with a tension meter, as shown in Figure 15, and the horizontal pulling force F_p was measured from the beginning of the fruit slippage.

Ignoring the apple’s weight, it can be obtained from the static balance of the apple,

$$F_p=2\mu \times F_n.$$

(17)

The test results and fitting function are shown in Figure 16, R² = 0.92. Therefore, μ = 0.8 can be obtained.

To obtain the detachment force F_d required for fruit detachment, the apple was fixed on the support and kept still; then one end of the branch was fixed with the collection of the Model E43 and pulled axially. When the fruit branch was broken through the force sensor, the maximum pulling force was recorded. The experimental results are shown in Figure 17. The experiment used twenty apple samples with diameters ranging from 65 mm to 95 mm.

The results show that F_d is distributed between 8.88 N and 39.6 N. F_d generally increased as the apple diameter increased, but a small portion showed an irregular distribution. This could be because fruits with larger stem diameters have more connection force between branches and apples, necessitating more detachment force. At the same time, in the report of Bu [53] et al., the detachment force is much greater when the natural growth angle of the fruit is obtuse than when it is acute, as shown in Figure 18. In this experiment, we did not pay too much attention to the relation of detachment force to stem diameter and fruit growth angle. The test results were consistent with those of Bu [53] et al.

To avoid damaging the apple pericarp due to excessive gripping force, the maximum pressure F_m that the fruit pericarp can withstand must be known. We make a rectangular apple sample block of 10 mm × 10 mm × 20 mm near the apple’s pericarp, place it on the middle of the support of the Model E43, and apply a load to the apple sample until it is destroyed. The force–displacement relationship during the apple-sample compression experiment was recorded, and the results are shown in Figure 19.

It can be seen that, once the force reaches 15.35 N, it remains almost unchanged with a one-stage displacement increase. This demonstrates that, when the force reaches 15.35 N, the apple begins to tend to plastically deform. According to the energy principle of the apple damage proposed by Schoorl [54], the damage volume of the apple is proportional to the energy it absorbs. To reduce the amount of energy transmitted to the apples during harvesting, set the maximum pressure F_m that apples can withstand to 15.35 N. The test results were consistent with those of Grotte [55] et al.

5.2. Gripping Force Verification Experiment

The rated torque that the servo can provide in this soft gripper is 12 kg∙cm (1.2 N∙m), assuming that the maximum torque that a single finger can provide is 0.4 N∙m. L_Fc is 28 mm; L_R is 12 mm; h is 65 mm; α is 80°, and μ is 0.8. According to the structural design of the gripper, θ_FF is between −12°~15°, and θ_FR is between −30°~53°. Given that the fruit radius r varies, σ is customarily between 0° and 40°, and δ is traditionally between 0° and 25°. In the test, the diameter of the apple sample is about 90 mm. At initial contact, the finger and the fruit can be regarded as point contact. θ_FF is generally around 10°, and θ_FR is generally around 30°, as shown in Figure 20a. According to Equation (11), the maximum initial gripping force f_c of a single finger is approximately 15.34 N. The output torque M_d is little as the first contact, so the contact force between the finger and the fruit is far less than the maximum initial gripping force. When the gripper continues to close, σ and δ become larger, γ becomes smaller, and β becomes larger, so the finger gripping force f_c becomes larger, as does the pulling force F. At full contact, θ_FF is typically around −12°, and θ_FR is typically around −30°, as shown in Figure 20b. Therefore, the maximum gripping force f_c of a single finger is about 16.21 N. On the basis of Equation (16), the maximum pulling force F of a single finger is about 14.18 N, resulting in the maximum pulling force of the entire gripper being approximately 42.55 N.

According to the above test results, the detachment force when pulling to harvest the fruit is about 8.88 N–39.6 N, indicating that the designed gripper’s maximum pulling force meets the detachment requirement.

The gripping force resulting from the adaptive bending deformation of the soft fingers in contact with the fruit surface, which was measured by a thin-film pressure sensor (RP-L TDS REV C.) mounted between each finger and the silicone pad, as shown in Figure 21a. The RP-L type soft thin-film pressure sensor was composed of polyester film, high conductive material, and pressure-sensitive material. It converts the pressure acting on the thin-film area of the sensor into a change in resistance.

The test started when the finger made contact with the apple, and the output torque of the servo increased by 0.2 kg·cm (0.02 N·m) each time until it reached the rated torque of 12 kg·cm (1.2 N·m). To compare the difference in the gripping force of the finger on the surface of the fruit when the diameter of the fruit changes, apples with diameters of 70 mm, 80 mm, and 90 mm were chosen for the test, as shown in Figure 21b. In each test, the pressure output by the sensor and the servo torque was recorded, as shown in Figure 22.

As can be seen from the figure, there is an approximate positive relationship between the gripping force of the soft finger and the servo torque, and the image fits the theoretical curve well. Furthermore, it can be found that the effect on the gripping force is not very significant when the diameter of the fruit changes. Therefore, the finger output force during picking can be controlled by adjusting the servo output torque.

Nevertheless, the single-finger gripping force at a torque up to 1.2 N∙m for the fruit diameter of 90 mm does not reach the theoretically calculated maximum value, which is probably due to the lack of accuracy from the thin-film pressure sensor.

5.3. Test Analysis on the Harvesting Performance of the Soft Gripper

During the grasping and harvesting tests, the soft gripper was fixed on Franka, a seven-axis robotic arm with a high-sensitivity force-control performance, as shown in Figure 23. The tests were carried out in an orchard located in Changping Distrct, Beijing.

5.3.1. Feasibility Test Analysis of Constant-Pressure Feedback System

To ensure that the finger gripping force is less than 15.35 N, the servo output torque is set to control the maximum gripping force f_c. Assuming that the detachment force required for fruit detachment is 40 N, it can be obtained from Equation (16) that the required output torque is 10.25 kg·cm (1.025 N·m). A single finger’s grasping force f_c is 13.89 N, which is not harmful.

Therefore, a grasping comparison test was performed to verify the improvement of the soft gripper’s safe grasping performance by the force feedback system. In this experiment, a total of 20 apple samples with no damage on the fruit skin were selected and divided into two groups of ten apples each. In the first set of experiments, the force feedback system was turned on, and the clamping test was performed on each apple. The clamping process followed the logic of the flowchart in Figure 12, and the clamping posture is shown in Figure 24a. After the gripper has completely and stably grasped the apple, hold it still for 5 s before releasing the fruit. In the second set of experiments, all experimental conditions were the same except that the force feedback system was turned off. As there is no output torque control, the clamping will stop until the servo reaches the locked rotor torque, and the clamping posture is shown in Figure 24b. The contact area on the fruit was marked after each release, and the fruit was then stored at the same constant temperature for 7 days. After taking them out, make a note of the damage on the apple surface’s contact area. The radius of the damaged area was less than 10 mm for slight damage and greater than 10 mm for serious damage.

Observing the apple surface, the contact area of the apples clamped by the gripper with an open force feedback system was not damaged, so the damage rate was 0%; however, the apples were clamped by the gripper with a closed force feedback system. On the other hand, the slight damage rate was 10%, and the severe damage rate was 10%; the specific pericarp damage is shown in Figure 25. The experimental results show that activating the constant-pressure feedback system improves the soft gripper’s safe grasping performance and effectively ensures non-destructive fruit grasping.

5.3.2. Test Analysis of Harvesting Success Rate and Apple Damage Rate

We carried out picking experiments to verify the stability and safety of the soft harvesting gripper designed in this paper. The harvesting process followed the logic of the flowcharts in Figure 12 and Figure 14 with the force feedback system on. The soft finger length is 120 mm, while the effective gripping length is 100 mm. In the tests, L_s was set to 10 mm. To grab and separate the fruit, the soft picking gripper was controlled by Franka’s arm with a pulling speed of 2 mm/s.

First, we analyzed various situations that occurred in the fruit harvesting process with the fruit slip detection turned on. The process began with the gripper approaching the fruit and ended with the fruit being harvested. The condition of the fruit slip and the change in the servo output torque for the three situations of no obvious slip, first slip, and second slip was recorded afterwards, as shown in Figure 26.

The figure shows that, even if the fruit did not slip for the first time, there would be a slight relative movement to the finger during harvesting, which might be due to the fingertip not being completely in contact with the fruit. After the fruit slipped slightly, the fingertip and the fruit made complete contact, providing adequate support for the apple. It was also conceivable that the measurement distance was floating within the accuracy range due to a lack of sensor accuracy. When the fruit slipped for the first time, the occurrence time was approximately 10 s, implying that the gripper pulled the fruit 2 cm. At this point, the fruit branch was completely straightened, and sufficient force was required to detach it from the branch; if the fruit slipped for the second time, it proved that it was not enough to harvest the fruit under the premise of safe harvesting; in addition, further harvesting might damage the fruit.

It can be ascertained that, during fruit harvesting, the stable servo output torque can ensure that the fruit does not break free due to the gripper loosening.

To further verify the effectiveness of the gripper harvesting, the tests for the three cases of rigid fingers and soft fingers with or without slip detection under the gripper structure of this study were carried out, as shown in Figure 27. For each group of the experiments, 25 apples with completely undamaged pericarps were selected. The picking situation and fruit harvesting damage were observed and recorded. The experimental results are shown in

Table 2. The harvesting situation of rigid fingers.

Average Diameter (mm)	Average Mass (g)	Number of Visible Slippage 1	Number of Picking Success	Number of Picking Damage	Number of Slippage Damage
82.7536	236.748	7	25	4	3
Damaged Fruit Characteristics
Fruit Diameter (mm)	Fruit Mass (g)	Visible Slippage or Not?	Picking Success or Not?	Picking Damage or Not?	Damage Causes 2
83.04	245.7	Yes	Yes	Yes	Slippage
86.37	280	Yes	Yes	Yes	Slippage
88.86	262.5	Yes	Yes	Yes	Slippage
90.23	278	No	Yes	Yes	Grasping

¹ When the slip detection is turned off, we define the visible slippage as the fruit that is about to slide to the fingertips of the gripper or that has already broken from the gripper (the same as below). ² Because the fruit damage due to slippage in the gripper and due to grasping are quite different in character, we can distinguish them more easily (the same as below).

,

Table 3. The harvesting situation of soft fingers without slip detection.

Average Diameter (mm)	Average Mass (g)	Number of Visible Slippage	Number of Picking Success	Number of Picking Damage	Number of Slippage Damage
83.7548	232.724	9	24	2	2
Damaged fruit characteristics
Fruit Diameter (mm)	Fruit Mass (g)	Visible Slippage or Not?	Picking Success or Not?	Picking Damage or Not?	Damage Causes
82.35	235.5	Yes	Yes	Yes	Slippage
86.66	260.3	Yes	Yes	Yes	Slippage

and

Table 4. The harvesting situation of soft fingers with slip detection.

Average Diameter (mm)	Average Mass (g)	Number of First Slippage	Number of Second Slippage	Number of Picking Success	Number of Picking Damage
84.2252	242.932	13	7	20	0
Second Slippage Fruit Characteristics
Fruit Diameter (mm)	Fruit Mass (g)	Second Picking after First Failed Harvesting 1	Second Picking Success or Not?	Picking Damage or Not?	Damage Causes
82.32	226.5		Yes	No	—
83.31	233.5		No	No	—
84.45	226		No	No	—
84.65	256.1		Yes	No	—
86.19	266.6		No	No	—
90.19	279.4		No	No	—
91.11	309.8		No	No	—

¹ With slip detection on, the second slippage of the fruit means that the picking has failed. At this point, each fruit was picked twice; it implies that the fruit has failed, and the next fruit would be chosen if both pickings failed.

.

Comparing Table 2 and Table 3, the picking success rate for the rigid fingers is 100%, with a damage rate of 16%, while the success rate for the soft fingers is 96%, and the damage rate is 8%, both of these have the silicone gasket applied to the surface. This shows that the optimized Fin-Ray soft fingers in this paper are able to reduce the fruit damage better. At the same time, we can see that visible slippage of the fruit was common in both cases and that most of the damage occurred during the fruit slippage in the gripper. In the rigid fingers experiment, three fruits were damaged by slippage and one by grasping, which also shows that the rigid support structure is prone to fruit damage despite the flexible silicone gasket applied to the surface. In the soft fingers experiment, both damaged fruits were caused by slippage. Therefore, the effective control of the fruit slip in the gripper is essential to reducing the risk of fruit damage.

Comparing Table 3 and Table 4, although the picking success rate dropped to 80% with slip detection on, there was no fruit damage. It turns out that the soft gripper with slip detection can effectively reduce fruit damage. Despite the fact that the harvesting success rate will decrease, the fruit will not be harmed, and its economic value will not be impacted after manual harvesting. In addition, we can see from Table 4 that 13 fruits made the first slippage, and in 7 of them, the second slippage occurred, further demonstrating the prevalence of fruit sliding during picking. Although five fruits failed in the second picking, no fruit were damaged, which indicates that the proposed control method for slip detection is effective in preventing damage to the fruits.

According to the above experimental results, the proposed Fin-Ray soft harvesting gripper with force feedback and fruit slip detection enables stable and non-destructive fruit picking. Notably, to improve the harvesting lossless rate, it is necessary to sacrifice some harvesting success rates by detecting slippage between the fruit and the fingers.

Remark 2.

It should be noted that the experimental results of the outdoor harvesting could be regarded as the effect of combining both force feedback and slip detection on the basis of the optimized harvesting gripper.

6. Conclusions

This paper presents a three-fingered apple-harvesting soft gripper with constant-pressure feedback inspired by the Fin-Ray effect. First, the structural parameters of the single-soft-finger model were optimized using finite element analysis, and the influence of different Fin Ray finger structural parameters on the contact stress and fingertip displacement was investigated. The optimal structural parameters of the single soft finger were proposed: the front and rear beam thickness is 3.5 mm; the finger width is 16 mm, and the number of beams is 9. A three-fingered apple harvesting soft gripper was designed based on the above-optimized fingers. The determined gripper structure’s mathematical model was then statically analyzed, and the relationship between the gripping force, the pulling force, and the servo torque was obtained. Therefore, the finger output force during picking can be controlled by adjusting the servo output torque.

We also propose a dynamic control method for detecting fruit slip during apple harvesting by integrating a distance sensor in this study. The maximum static friction coefficient between the finger and the apple, the detachment force of the apple, and the damaged condition of the apple were obtained through an experimental analysis of the apple’s mechanical properties, which provides a theoretical basis for the gripper design. In indoor experiments, the results show that the servo output torque has an approximately linear relationship with the contact pressure between the fingers and the apple, and it is suitable for all sizes of apple. In the outdoor orchard experiments, turning on the constant-pressure feedback system can improve the safe grasping performance of the soft gripper, which can effectively ensure non-destructive fruit gripping. Comparing the tests for the three cases of rigid fingers and soft fingers with or without slip detection, the optimized Fin-Ray soft fingers in this paper are able to reduce the fruit damage better, and opening the slip detection can effectively avoid fruit damage. Furthermore, the stable output torque of the servo can ensure that the fruits do not break free due to the gripper loosening during harvesting.

In this study, we believe that the soft harvesting gripper is not only suitable for harvesting apples but also for harvesting some other fruits (e.g., tangerine and kiwi) and vegetables (e.g., tomato) and can provide an application reference. It has a high degree of adaptability and can effectively avoid fruit damage by adjusting the servo output torque.

However, our current research work still has some limitations. On the one hand, the complex structure of the Fin Ray fingers needs further investigation, and we will conduct more in-depth and detailed research on it in the future. On the other hand, the theoretical analysis and design of the gripper are only for the single-pulling fruit harvesting method in this study, which has significant limitations. This is only the first step in our exploration. In future work, combined with the optimal method and the posture of the fruit harvesting, the harvesting method combining gripper rotation and pulling will be studied.