*2.4. Three-Dimensional Positioning Algorithm for Pineapple Eyes*

In this paper, images of pineapples are collected using 30◦ intervals; obviously, the same pineapple eye appears on multiple consecutive images. By analyzing these images and matching the same pineapple eyes in different images, the parallax information of the pineapple eye can be obtained. The depth information of the pineapple eye can be obtained through triangulation. In this paper, two images with an angle difference of 90◦ are used as a group for stereo matching analysis to obtain the three-dimensional position information of all pineapple eyes. Considering the high similarity of pineapple images from different angles, the traditional stereo-vision-matching algorithm is not expected to perform well. In addition, a large amount of calculation is required in this algorithm, which also has low efficiency. Therefore, this algorithm is not suitable for the needs of actual production. Figure 4a,b show the comparison of the γ degree and γ + 90-degree pineapple eye images. Here, one pineapple eye appears in both images.

**Figure 4.** Epipolar constrained stereo matching. (**a**) γ degree; (**b**) γ + 90 degree; (**c**) calculate d of distance schematic diagram.

The central coordinates (*uc*, *vc*) and (*uc*1, *vc*1) are used to describe the position of the pineapple eye in the two images. Therefore, the position of the pineapple eye in the two images must satisfy the following two constraints: (1) the center of the pineapple eye is located on the same vertical line in the two images, that is, *vc*<sup>1</sup> = *vc*. (2) The row coordinates of the center of the pineapple eye on the two images can be predicted by the displacement of the center of the pineapple eye after the pineapple is rotated by 90◦, namely, *uc*<sup>1</sup> = *uc* + *d*.

In order to obtain the value *d* in Figure 4c, Figure 5 is used to describe the solution process in detail, *f* is the focal length, and S is the distance between point *O* and point p, the optical center of the camera; *R* is the radius on the contour of the pineapple cross-section through C of the pineapple eye. We can obtain Formula (1).

$$\begin{cases} \begin{aligned} \eta &= \arcsin\left(\frac{l\_0 d\_x}{f}\right) \\ R &= S \sin \eta \\ \frac{l\_1 d\_x}{R \sin r} &= \frac{f}{S - R \cos r} \\ d &= d\_1 + d\_2 = R \sin r + R \cos r \end{aligned} \tag{1} $$

where *dx* represents the physical size of a pixel on the *u*-axis, which is 0.00465 mm in this paper, *η* is the angle between the *O*p and the Ap, and *r* is the *O*G and the *OC*1.

**Figure 5.** Schematic diagram of the 90◦ rotation distance of the pineapple eye center point.

Since the contour of the pineapple cross section through C of the pineapple eye is not an ideal circle, and due to system errors such as installation and imaging, *uc*<sup>1</sup> and *vc*<sup>1</sup> cannot fully meet the above constraints; therefore, a certain tolerance Δ is added when finding a matching pineapple eye in the γ + 90 degree image. In other words, we search for the target pineapple eye within the rectangular box (*uc*<sup>1</sup> − Δ,*vc*<sup>1</sup> − Δ,*uc*<sup>1</sup> + Δ,*vc*<sup>1</sup> + Δ). To ensure that there is only one pineapple eye in the rectangular box, Δ is set to a third of the minimum distance between the two pineapple eyes in the image. Obviously, according to the above constraints, the pineapple eye below the rotation axis in Figure 4a is not found in Figure 4b, so there is no need to perform a matching operation.

In this paper, a 3D localization algorithm for pineapple eyes based on monocular multiangle image matching is proposed. After obtaining the image coordinates of the same pineapple eye in two images with a difference of 90◦, the depth of the pineapple eye is calculated by triangulation. The information is then used to calculate the three-dimensional position information of the pineapple eye. The camera coordinate system *Oc*\_*XcYcZc* is established with the camera optical center as the origin, as shown in Figure 6. The center point C of any pineapple eye is selected as the measurement object. (*uc*, *vc*) represents the pixel value of pineapple eye center C under the imaging plane, *O*<sup>1</sup> is the intersection of the imaging plane of the pineapple eye center point C and the camera optical axis, and the pixel value is (*u*0, *v*0) .

The circle in Figure 7 is the cross-sectional profile of the pineapple through point C. *ψ* is the angle between the line segment *OC* and the optical axis of the camera, which satisfies the formula *<sup>ψ</sup>* <sup>=</sup> arctan *h*1 *h*2 . The pineapple is rotated clockwise in the direction indicated by the arrow in the figure. p refers to the optical center of the camera. The distance between point C and point p of the camera optical center is *W*, the distance between point *O* and point p of the camera optical center is S, and *l*<sup>1</sup> is the number of pixels in the axial direction of the pineapple eye imaging plane. When the pineapple rotates clockwise by 90◦, which is equivalent to a 90◦ counterclockwise rotation of the camera, as shown in the

dotted line in Figure 8, *l*<sup>2</sup> is the number of pixels in the axial direction of the pineapple eye imaging plane after rotation. The following formula can be obtained from Figure 7:

$$\begin{cases} \alpha = \arctan\left(\frac{l\_1 \times d\_x}{f}\right) \\ \beta = \arctan\left(\frac{l\_2 \times d\_x}{f}\right) \end{cases} \tag{2}$$

**Figure 6.** Camera coordinate system for the pineapple eye.

**Figure 7.** Schematic diagram of the pineapple eye depth information calculation.

**Figure 8.** Three-dimensional positioning schematic diagram.

In this formula, *dx* represents the physical size of a pixel on the *u*-axis, which is 0.00465 mm in this paper. *α* and *β* can be solved by using Formula (2), and *h*<sup>1</sup> and *h*<sup>2</sup> in Figure 7 can be simultaneously solved according to the following equations.

$$\begin{cases} h\_1 = \frac{\mathcal{S}(1 - \tan(\beta))\tan(a)}{1 - \tan(a)\tan(\beta)}\\ h\_2 = \frac{\mathcal{S}(1 - \tan(a))\tan(\beta)}{1 - \tan(a)\tan(\beta)} \end{cases} \tag{3}$$

In Formula (3), *h*<sup>1</sup> is the distance between point C and point *O*1, mm. *h*<sup>2</sup> is the distance between point C and point K, mm. Figure 7 shows that the center point C of the pineapple eye is imaged at time *t*, and the object distance of the imaging plane is *W* = *S* − *h*2. Then, the number of pixels of pineapple eye point C on the imaging plane and in the camera coordinate system are determined using the following equation:

$$\begin{cases} \frac{d\_x(u\_c - u\_0)}{f} = \frac{\chi\_c}{W} \\ \frac{d\_x(v\_c - v\_0)}{f} = \frac{\chi\_c}{W} \end{cases} \tag{4}$$

In other words, at time *t*, the center point C of the pineapple eye fulfills the matrix in the camera coordinate system, with the camera optical center serving as the origin:

$$
\begin{bmatrix} X\_{\mathcal{L}} \\ Y\_{\mathcal{L}} \\ Z\_{\mathcal{L}} \end{bmatrix} = \mathcal{W} \begin{bmatrix} u\_{\mathcal{L}} & -u\_0 & 0 \\ v\_{\mathcal{L}} & -v\_0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \frac{d\_{\mathcal{L}}}{f} \\ \frac{d\_{\mathcal{L}}}{f} \\ 1 \end{bmatrix} \tag{5}
$$

To facilitate subsequent experiments and the operation of removing pineapple eyes in practical engineering applications, the pineapple three-dimensional space coordinate with *O* as the center is established *O*\_*XYZ*. The geometric vector approach is used to translate the camera coordinates into the 3D space coordinates of the pineapple eye, as shown in Figure 8. <sup>⎧</sup>

$$\begin{cases} X = -X\_c \\ Y = Y\_c \times \cos(-\alpha) + (\mathcal{S} - Z\_c) \times \sin(-\alpha) \\ Z = \mathcal{S} - ((\mathcal{S} - Z\_c) \times \cos(\alpha) + Y\_c \times \sin(\alpha)) \end{cases} \tag{6}$$

Furthermore, the three-dimensional coordinates of the pineapple eye (*X*,*Y*, *Z*) are converted to the probe, which can be used for eye removal after changing to the eye-removal tool. Position *L* and pineapple rotation angle *θ* are represented by the space coordinates

(*L*, *θ*), and *O*<sup>2</sup> is the starting point of the probe. As shown in Figure 9, the corresponding conversion formula is as follows:

$$\begin{cases} \quad L = \mathbb{X}\_1 - X \\ \quad \theta = \frac{Y}{\mathbb{S} - \mathbb{Z}} \times 180 / PI \end{cases} \tag{7}$$

**Figure 9.** Flow diagram of the 3D positioning method for pineapple eyes.

In Equation (7), X1 is the distance from the optical center of the horizontal axis camera to the starting point of the probe. Because the pineapple is rotating during the image acquisition process, all the calculated coordinates of the pineapple eyes are the result of the calculation of the current pineapple angle conditions. To obtain the coordinates of all pineapple eyes for the whole pineapple in the same coordinate space, we should reverse rotate the coordinates of all pineapple eyes to the 0◦ position. Therefore, Formula (7) should be modified to the following:

$$\begin{cases} \ L = \mathbb{X}\_1 - X \\ \theta = \frac{\mathbb{Y}}{\mathbb{S} - \mathbb{Z}} \times \frac{180}{PI} - \gamma \end{cases} \tag{8}$$

The position information of all pineapple eyes can be obtained after image stereo matching and pineapple eye position computation. To ensure that the position information of each pineapple eye is calculated, the image acquisition angle interval is set to 30 degrees, which leads to the same pineapple eye being calculated in multiple sets of images. This results in more calculated pineapple eyes than the actual number of pineapple eyes. To avoid the same pineapple eye being repeatedly calculated, a successful match of a pineapple

eye in the image is marked. When using the image and the next picture, the marked pineapple eye does not participate in the matching calculation.
