Research on a Novel Citrus Reticulata ‘Chachi’ Orientation Adjustment Mechanism (COAM) and Machine Vision Guidance Control

Abstract

The initial processing of Citrus Reticulata ‘Chachi’ involves peeling as a crucial step. Currently, there is some semi-automatic peeling equipment available. However, due to the requirement of adjusting the orientation of Citrus Reticulata ‘Chachi’ to ensure the stem (or navel) is facing upwards before peeling and because the peeling process must retain the stem as a marker for fresh fruit picking, the loading of Citrus Reticulata ‘Chachi’ for peeling still solely relies on manual operation, resulting in low efficiency and poor standardization. With the rapid growth of the pericarp of the Citrus Reticulata ‘Chachi’ industry, semi-automatic processing equipment is no longer able to meet production demands. The loading issue before peeling Citrus Reticulata ‘Chachi’ is a complex hand–eye coordination problem. In response to this issue, this paper proposes a novel Citrus Reticulata ‘Chachi’ orientation adjustment mechanism (COAM). This mechanism utilizes frictional force to adjust the orientation of Citrus Reticulata ‘Chachi’. First, the conceptual design and kinematic modelling analysis of the mechanism were conducted. Next, the omnidirectional friction-driven wheels were optimized in design. Subsequently, a prototype was manufactured and assembled to conduct validation tests on its open-loop motion performance. Finally, a visual feedback-guided algorithm was introduced to complement the kinematic model, enabling the automatic and rapid adjustment of Citrus Reticulata ‘Chachi’ orientation. The experimental results indicate that the COAM designed in this study can effectively and rapidly adjust the orientation of Citrus Reticulata ‘Chachi’ fruits of different sizes and shapes. It demonstrates strong adaptability, and under visual feedback guidance, the orientation adjustment error is less than 10% of the fruit’s diameter. This meets the requirements for automated production in the initial processing of Citrus Reticulata ‘Chachi’. The research presented in this paper also provides new insights for the orientation adjustment and loading of similar spherical fruits.

1. Introduction

Citrus Reticulata ‘Chachi’ belongs to the Rutaceae family and Citrus genus. It is a small deciduous tree cultivated in regions such as South China, Tibet, and Taiwan Province [1,2]. Currently, the main production area of Citrus Reticulata ‘Chachi’ is Xinhui District, Guangdong Province, and it has become a characteristic agricultural product of Jiangmen City, known locally as Xinhui Citrus [3]; its primary processed product is the “pericarp of the Citrus Reticulata ‘Chachi’” [4]. As shown in Figure 1, the pericarp of the Citrus Reticulata ‘Chachi’ is often divided into three segments, with a thickness of approximately 1~4 mm, and its outer surface features abundant sunken oil glands [5]. From the perspective of traditional Chinese medicine, the pericarp of the Citrus Reticulata ‘Chachi’ has the functions of drying dampness, resolving phlegm, regulating qi, and invigorating the spleen [6]. It provides excellent health benefits in cardiovascular, respiratory, endocrine, gastrointestinal systems [7,8], as well as in beauty and skin care, possessing the characteristics of both food and medicine [9,10]. From the perspective of Western medicine, the flavonoids contained in the pericarp of the Citrus Reticulata ‘Chachi’ not only have anti-inflammatory and free radical scavenging effects [11,12] but can also inhibit cancer cell growth to some extent [13]. Additionally, the alkaloids and inositol contained in it are also beneficial substances for the human body [14]. These characteristics determine its high medicinal and economic value, leading to annual increases in the production and output value of Citrus Reticulata ‘Chachi’ [15].

The traditional process of making the pericarp of the Citrus Reticulata ‘Chachi’ involves multiple steps, as shown in Figure 2. Generally, it includes picking, washing, cutting, peeling, flipping, sun-drying, aging, and storing [16,17,18]. Among these, peeling is a crucial step in the initial processing of Citrus Reticulata ‘Chachi’. The traditional peeling methods for Citrus Reticulata ‘Chachi’ can be divided into the “three cuts on the front” and “two cuts on the back” methods. The back-cutting method is used when the stem of the Citrus Reticulata ‘Chachi’ is facing upward, while the front-cutting method is used when the stem is facing downward. Both methods result in the peel being divided into a tripetalous form [5]. As seen in Figure 3, regardless of the peeling method used, it is essential to preserve the intact stem to distinguish between freshly picked and fallen fruits. Both the front-cutting and back-cutting methods divide the orange peel into a tripetalous form, with a larger curled surface area, facilitating the preservation of peel integrity during the flipping and sun-drying processes [19].

With the rapid growth of the pericarp of the Citrus Reticulata ‘Chachi’ industry in recent years, it has gradually diversified [20]. Various deep-processed products of the pericarp of the Citrus Reticulata ‘Chachi’ continue to be introduced [21], achieving certain results in the research of medicinal and edible products. Market demand has surged, posing higher requirements for the automation of Citrus Reticulata ‘Chachi’ pericarp processing [22,23]. Currently, available semi-automatic equipment exhibits randomness in the positioning of the stem when peeling Citrus Reticulata ‘Chachi’, which may lead to stem damage or damage to the cutting tools, failing to meet the technical requirements of the initial processing of the pericarp of the Citrus Reticulata ‘Chachi’.

Therefore, semi-automatic cutting equipment for Citrus Reticulata ‘Chachi’ mainly relies on manual loading to accomplish stem identification and orientation adjustment. For example, a rapid cutting machine for Citrus fruits disclosed in a Chinese patent [24] (Figure 4) allows for the placing of the Citrus fruit on a base and the even pressing of the operating handle to cut the peel into three segments (tripetalous). The combined pneumatic–electric cutting machines designed by a local enterprise in Xinhui, Guangdong are shown in Figure 5. When working, one must manually load the Citrus Reticulata ‘Chachi’ onto a workstation and position the stem facing the correct orientation; then, an electric platform or turntable intermittently moves below the cutting knife, and the pneumatic-driven cutting knives moves downward to evenly cut the peel of Citrus Reticulata ‘Chachi’ into three segments (tripetalous). Another piece of equipment designed for the automatic separation of the Citrus peel and pulp, as disclosed in a Chinese patent [25], is shown in Figure 6; a conveyor belt, segmenting machine cylinder, and separating machine cylinder are all connected to a pneumatic controller, enabling the automatic batch segmentation of Citrus fruits and the separation of the peel and pulp. However, none of the mentioned equipment can automatically identify the stem and adjust the orientation of the fruit. All current equipment requires a human eye to recognize the stem and human personnel to manually load the fruit in the correct orientation, which directly impacts the efficiency and quality of initial processing. It is evident that automatic stem identification and fruit orientation adjustment before cutting are bottleneck issues restricting the full automation of the initial processing of Citrus Reticulata ‘Chachi’.

To adapt to the current production situation of the pericarp of the Citrus Reticulata ‘Chachi’ and fill the gap in modern intelligent equipment technology in fruit processing, this study focuses on a control method for the automatic stem identification and fruit orientation adjustment of Citrus Reticulata ‘Chachi’. A novel Citrus Reticulata ‘Chachi’ orientation adjustment mechanism (COAM) based on visual feedback guidance was designed. Through a visual feedback system, the coordinates of a stem can be obtained in real time, and the motion path of the stem adjustment can be planned and feedback-corrected in real time. This mechanism enables Citrus Reticulata ‘Chachi’ in any orientation to be quickly adjusted to the stem-up (or stem-down) position via the shortest path. The COAM can prevent stem damage, reduce the risk of tool damage, improve the efficiency of the initial processing of Citrus Reticulata ‘Chachi’, and ensure the quality of the pericarp of the Citrus Reticulata ‘Chachi’. The orientation adjustment process is fully automated, requiring no manual intervention, and can be easily integrated into industrial production lines, making subsequent peeling processes of Citrus Reticulata ‘Chachi’ simpler and more reliable.

2. The Working Principle and Structure Design of the COAM

2.1. Mechanical Structure Design

The mechanical structure design of the COAM is inspired by the ball-balancing robot. Utilizing a standard sphere (nylon) with a radius of R (35 mm) to simulate the spherical shape of Citrus Reticulata ‘Chachi’, the overall structure of the mechanism is shown in Figure 7. Viewed from above, the main body of the hexagonal structure is equipped with three symmetrically arranged omnidirectional friction-driven wheels, each one mounted on a pair of symmetric axes separated by 120°. The design of this three-wheel structure allows each individual wheel’s enveloping circle to have a contact point with the Citrus Reticulata ‘Chachi’, providing three rotational degrees of freedom in three-dimensional space. The shaft of each wheel is inclined at an angle of 30~60° with respect to the base of the mechanism to prevent the contact points of the three enveloping circles (R = 32 mm) from being too close. In this study, the angle between the wheel shaft and the base of the mechanism was set to 45° to ensure excellent omnidirectional rotation performance while achieving better motion control and stability. Additionally, symmetric motor mounting holes were designed on both sides of each individual wheel, and four mechanism mounting holes were designed on each side of the base for installation on different platforms. Wire outlets were also designed on the front and rear sides of the mechanism for cable routing.

The single-wheel structure of the COAM is shown in Figure 8. The single-wheel structure mainly consists of a stepper motor, flange shaft, shaft–wheel connecting joint, locking nut, and flexible rubber ball. The outer ring consists of 21 embedded flexible rubber balls (with a diameter of 6 mm) connected by iron wire and fixed in the outer ring’s grooves. The gap between these rubber balls is approximately 5 mm, providing the necessary flexibility and friction for the contact of Citrus Reticulata ‘Chachi’ and allowing the single wheel to have a larger enveloping circle than its radius. This arrangement also reduces the phenomenon of side-slipping when one or two wheels are rotating, thus achieving the decoupling of friction between the wheels.

2.2. Electronic Control System Design

Each single-wheel structure requires a stepper motor for driving; thus, the control system of the COAM needed to be designed. The control system diagram shown in Figure 9 includes a power supply module, motor driver module, MCU module, host computer module, and visual feedback module.

The expansion board of the motor driver module contains three A4988 stepper motor driver boards (12VDC rated voltage, up to 16 micro steps) (Allegro Microsystems, Manchester, NH, USA) powered by a 12 V lithium battery or DC adapter. Additionally, the power is run through a 5 V DC voltage regulator module to supply power to the MCU module. The MCU module utilizes an Arduino UNO (Arduino.cc, Ivera, Italy) development board, which is responsible for sending direction and pulse signals to the expansion boards to control the position and direction of the three stepper motors. It also communicates with the host computer via USART for tasks such as program downloading and command sending. The LDO Nema14 36 mm Pancake Stepper Motors (Ldo Motors Co., Ltd., Shenzhen, China) (LDO-36STH20-1004AHG, 12VDC rated voltage, 1.8-degree step angle) on the single-wheel structure have a high torque, small volume, and light weight, and they are coaxially connected with every single wheel through the motor shaft. The industrial camera (Catchbest photoelectric equipment Co., Ltd., Beijing, China) connects to the host computer via USB and can displays real-time images. The host computer program was written in the C# language with the OpenCV-Sharp library; it calculates the stem coordinates through image processing algorithms and sends them to the MCU module in real time. Upon receiving the stem coordinates, the MCU module uses a kinematic inverse solution and feedback control algorithms to obtain control signals to the motors. These signals are then sent to the stepper motor drivers via I/O ports, thereby controlling the coordinated motion of the entire mechanism.

2.3. Principle of Mechanical Transmission

From Section 2.1, it can be inferred that the three single wheels of the COAM are connected to three stepper motors via flange shafts. The rotation of the stepper motors can drive the connected single wheels to rotate clockwise or counterclockwise in a plane perpendicular to the motor axis. As shown in Figure 10a, when only wheel 1 rotates counterclockwise, it generates an angular velocity ω₁ right-hand rule along the direction of the motor axis, perpendicular to the maximum circular cross-section. At the contact point with the standard sphere, the flexible rubber balls on the enveloping circle of wheel 1 produce a friction force f_h1, thereby driving the standard sphere to produce an angular velocity ω_b1 in the direction perpendicular to the equatorial plane (tangent to the outer circle cross-section of wheel 1). This motion is like the meshing transmission of gears. Meanwhile, the flexible rubber balls on the enveloping circles of wheels 2 and 3 also experience lateral rolling at the contact points with the standard sphere, acting as passive elements to prevent slippage and thereby decoupling the friction forces between the three driving wheels and the standard sphere. As shown in Figure 10b, when wheel 1 rotates clockwise and wheel 3 rotates counterclockwise, they each generate angular velocities ω₁ and ω₃ along the direction of the motor axis, perpendicular to the maximum circular cross-section. The flexible rubber balls on the enveloping circles of wheels 1 and 3 each produce a friction force f_h1 and f_h3 at the contact points with the standard sphere, driving the standard sphere to produce an angular velocity ω_b13 in the direction perpendicular to the plane dividing the two wheels by 120°. Meanwhile, the remaining small wheels on wheel 2 continue to roll laterally, serving as passive elements to prevent slippage.

Therefore, by combining the control of the three stepper motors, the standard sphere can be rotated to different degrees around the x, y, and z axes (3 DOFs), so standard sphere orientation adjustment can be realized.

2.4. Kinematic Modelling

As shown in Figure 11a, a standard spherical coordinate system for Citrus Reticulata ‘Chachi’ was established.

In this system, the control position vector r(x,y,z) represents a vector pointing from the origin to an arbitrary position of the fruit stem, while the target position vector r₁(0,0,R) points from the origin to the position where the fruit stem is upward. $\theta$ denotes the spatial angle between the control position vector r and the target position vector r₁, and $\omega$ represents the composite angular velocity vector of the standard sphere coinciding with the rotation axis k. Essentially, to adjust the fruit stem from any position to an upright state, friction serves as the driving force. This involves controlling the Citrus Reticulata ‘Chachi’ to rotate around a rotation axis $k$ synthesized in three-dimensional space, ensuring that the control position vector r aligns with the target position vector r₁.

The rotation matrix for Citrus Reticulata ‘Chachi’ around the rotation axis k is as follows:

$$\mathit{Rot}(k,\theta )=\left[\begin{array}{ccc}\cos \beta \cos \gamma & -\cos \beta \sin \gamma & \sin \beta \\ \sin \alpha \sin \beta \cos \gamma +\cos \alpha \sin \gamma & -\sin \alpha \sin \beta \sin \gamma +\cos \alpha \cos \gamma & -\sin \alpha \cos \beta \\ -\cos \alpha \sin \beta \cos \gamma +\sin \alpha \sin \gamma & \cos \alpha \sin \beta \sin \gamma +\sin \alpha \cos \gamma & \cos \alpha \cos \beta \end{array}\right]$$

(1)

The coordinates of the control position vector r at a certain moment of rotation around the rotation axis $k$ are as follows:

$$\left[\begin{array}{c}{x}_t\\ y_t\\ z_t\end{array}\right]=\left[\begin{array}{ccc}\cos \beta \cos \gamma & -\cos \beta \sin \gamma & \sin \beta \\ \sin \alpha \sin \beta \cos \gamma +\cos \alpha \sin \gamma & -\sin \alpha \sin \beta \sin \gamma +\cos \alpha \cos \gamma & -\sin \alpha \cos \beta \\ -\cos \alpha \sin \beta \cos \gamma +\sin \alpha \sin \gamma & \cos \alpha \sin \beta \sin \gamma +\sin \alpha \cos \gamma & \cos \alpha \cos \beta \end{array}\right]\left[\begin{array}{c}x\\ y\\ z\end{array}\right]$$

(2)

where $\alpha ,\beta ,\gamma$ represent the rotation angles of Citrus Reticulata ‘Chachi’ around the x, y, and z axes, respectively. As shown in Figure 11b, assuming the transformed new coordinate system is o-x’ y’ z’, where the z’ axis coincides with the rotation axis k, the z’ axis is aligned with the direction of the control position vector r, and y’ axis is perpendicular to the x’ o z’ plane, a coordinate transformation can be performed to simplify calculations. Then, the coordinate transformation matrix T is given by:

$$\mathit{T}={\left[\begin{array}{ccc}{\displaystyle \frac{\mathit{r}}{\left|\mathit{r}\right|}}& {\displaystyle \frac{\mathit{r}\times \mathit{dir}}{\left|\mathit{r}\times \mathit{dir}\right|}}& \mathit{dir}\end{array}\right]}_{3\times 3}={\left[\begin{array}{ccc}{\displaystyle \frac{\mathit{r}}{\left|\mathit{r}\right|}}& {\displaystyle \frac{\mathit{r}\times \left(\frac{\mathit{r}\times {\mathit{r}}_1}{\left|\mathit{r}\times {\mathit{r}}_1\right|}\right)}{\left|\mathit{r}\times \left(\frac{\mathit{r}\times {\mathit{r}}_1}{\left|\mathit{r}\times {\mathit{r}}_1\right|}\right)\right|}}& {\displaystyle \frac{\mathit{r}\times {\mathit{r}}_1}{\left|\mathit{r}\times {\mathit{r}}_1\right|}}\end{array}\right]}_{3\times 3}$$

(3)

where dir is the unit directional vector of the z’ axis, aligned with the direction of $\mathit{\omega }$, obtained by normalizing the cross product of the control position vector r and the target position vector r1. After combining (3) and r(x,y,z), in the new coordinates, the rotation of the standard sphere around the rotation axis k simplifies to rotation around the z’ axis. At a certain moment t, the coordinates of r rotating from r to r1 by angle $\phi$ around the rotation axis k simplify as follows:

$$\left[\begin{array}{c}{x}_t\\ y_t\\ z_t\end{array}\right]=\mathit{T}\times \mathit{Rot}\left(z\prime ,\phi \right)\times {\mathit{T}}^{-1}\times \mathit{r}$$

(4)

where $\mathit{Rot}\left(z\prime ,\phi \right)=\left[\begin{array}{ccc}\cos \phi & -\sin \phi & 0\\ \sin \phi & \cos \phi & 0\\ 0& 0& 1\end{array}\right]$, the spatial angle is $\theta$, the average angular velocity is $\left|\mathit{\omega }\right|$, and the angle $\phi$ that r rotates towards r1 around the rotation axis k can be obtained with r, r1 and the total control time $\Delta t$:

$$\theta =ar\cos \left({\displaystyle \frac{\mathit{r}\cdot {\mathit{r}}_1}{\left|\mathit{r}\right|\left|{\mathit{r}}_1\right|}}\right)=ar\cos \left({\displaystyle \frac{z}{R}}\right)$$

(5)

$$\left|\omega \right|={\displaystyle \frac{\theta \pi }{180\Delta t}}$$

(6)

$$\phi ={\displaystyle \frac{180\left|\omega \right|t}{\pi }}={\displaystyle \frac{\theta t}{\Delta t}}$$

(7)

Therefore, by combining Equations (3)–(7), the coordinates of the control position vector r rotating around the rotation axis k towards r₁ at each moment t can be obtained, allowing for subsequent sampling to record the adjustment trajectory of the standard sphere and the Citrus Reticulata ‘Chachi’ stem.

As shown in Figure 12, the three individual wheels of the COAM are labelled as wheel 1, wheel 2, and wheel 3. The origin O_b of the main body coordinate is located at the center of the bottom of the main body of the COAM, where X_b, Y_b, and Z_b, respectively, point to the right, front, and upward directions of the mechanism. The origins O₁, O₂, and O₃ of the three individual wheel coordinates are located at the center of the wheel.

Taking wheel 3 as an example, the directional vector is as follows:

$${\mathit{p}}_3={\left[\begin{array}{ccc}{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi & l\sin \psi \end{array}\right]}^{{\rm T}}$$

(8)

where l represents the straight-line distance between the origin O₃ of the wheel 3 coordinate and the origin O_b of the COAM global coordinate and $\psi$ is the spatial angle between the wheel axis and the bottom surface. Similarly, the P₁ and P₂ direction vectors for wheel 1 and wheel 2, as well as the position matrix P for all three wheels, are as follows:

$${\mathit{p}}_1={\left[\begin{array}{ccc}0& l\cos \psi & l\sin \psi \end{array}\right]}^{{\rm T}}$$

(9)

$${\mathit{p}}_2={\left[\begin{array}{ccc}-{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi & l\sin \psi \end{array}\right]}^{{\rm T}}$$

(10)

$$\mathit{P}=\left[\begin{array}{ccc}{\mathit{p}}_1& {\mathit{p}}_2& {\mathit{p}}_3\end{array}\right]=\left[\begin{array}{ccc}0& -{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi & {\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi \\ l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi \\ l\sin \psi & l\sin \psi & l\sin \psi \end{array}\right]$$

(11)

The forward kinematic equations are as follows:

$$\mathit{\omega }=\left[\begin{array}{c}{\mathit{\omega }}_x\\ {\mathit{\omega }}_y\\ {\mathit{\omega }}_z\end{array}\right]=\mathit{PW}=\mathit{P}\left[\begin{array}{c}{\mathit{\omega }}_1\\ {\mathit{\omega }}_2\\ {\mathit{\omega }}_3\end{array}\right]=\left[\begin{array}{c}-{\mathit{\omega }}_2{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi +{\mathit{\omega }}_3{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi \\ {\mathit{\omega }}_{1}l\cos \psi -{\mathit{\omega }}_2{\displaystyle \frac{1}{2}}l\cos \psi -{\mathit{\omega }}_3{\displaystyle \frac{1}{2}}l\cos \psi \\ {\mathit{\omega }}_{1}l\sin \psi +{\mathit{\omega }}_{2}l\sin \psi +{\mathit{\omega }}_{3}l\sin \psi \end{array}\right]$$

(12)

Meanwhile, the input matrix W of the forward kinematic equations consists of the angular velocity vectors of the three individual wheels ${\mathit{\omega }}_1,{\mathit{\omega }}_2,{\mathit{\omega }}_3$ and the output matrix $\mathit{\omega }$ represents the matrix of angular velocity vectors ${\mathit{\omega }}_x,{\mathit{\omega }}_y,{\mathit{\omega }}_z$ generated by the standard sphere on the three coordinate axes. Derived from the forward kinematic equations, the inverse kinematic equations are as follows:

$$\mathit{W}=\left[\begin{array}{c}{\mathit{\omega }}_1\\ {\mathit{\omega }}_2\\ {\mathit{\omega }}_3\end{array}\right]={\mathit{P}}^{-1}\mathit{\omega }={\left[\begin{array}{ccc}0& -{\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi & {\displaystyle \frac{\sqrt{3}}{2}}l\cos \psi \\ l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi & -{\displaystyle \frac{1}{2}}l\cos \psi \\ l\sin \psi & l\sin \psi & l\sin \psi \end{array}\right]}^{-1}\left[\begin{array}{c}{\mathit{\omega }}_x\\ {\mathit{\omega }}_y\\ {\mathit{\omega }}_z\end{array}\right]$$

(13)

This establishes the forward and inverse kinematic models of the COAM, which was an essential step for the subsequent simulation examples and motion control experiment. The simulation parameters are shown in

Table 1. Simulation parameters.

Sphere Radius/mm	Wheel Radius/mm	Line Trajectory/mm	Circular Trajectory/mm
35	25	$l:\left\{\begin{array}{c}{x}_t=10t\\ y_t=10t\end{array}\right.$	$C:\left\{\begin{array}{c}{x}_t=33\cos \left(2\pi t\right)\\ y_t=33\sin \left(2\pi t\right)\end{array}\right.$

, where the line l was selected from the xoy plane, and the simulation results are shown in Figure 13.

Similarly, when selecting a circular trajectory C from the xoy plane, the simulation results were as shown in Figure 14.

The simulation results are shown in Figure 13 and Figure 14. It can be observed that under different trajectory scenarios, the motion patterns of the three individual wheels exhibited significant differences. When the motion trajectory of the COAM was a line, due to movement in only one direction, two of the wheels rotated at high angular velocities to facilitate differential motion, dominating the motion direction and angular velocity magnitude of the standard sphere’s motion. The remaining wheel, positioned farther from the direction of motion and approximately orthogonal to it, rotated at a lower angular velocity. Thus, utilizing flexible rubber balls on the side to act as passive components enabled the decoupling of frictional forces. Additionally, when the motion trajectory of the COAM was circular, the motion patterns of each wheel were similar, with a phase difference of 120°. The distance between each wheel’s position and the direction of motion varied over time, resulting in corresponding changes in the magnitude of their angular velocities.

3. Open-Loop Orientation Adjustment Experiment

3.1. The Experimental Platform Setup and Machine Vision Recognition

In Section 2.4, the kinematic model of the COAM was validated through simulation analyses using given line and circular trajectories. To further validate the correctness of the kinematic model, an open-loop orientation adjustment experiment was conducted using a standard sphere made of nylon on the constructed experimental platform. as shown in Figure 15a.

The radius of the standard sphere was equivalent to the average transverse radius of Citrus Reticulata ‘Chachi’ (35 mm), and its weight matched the average weight of Citrus Reticulata ‘Chachi’ (150 g). Additionally, to better match the look of Citrus Reticulata ‘Chachi’, the surface of the standard sphere was coated with orange matte paint. Furthermore, considering that the stem of the Citrus Reticulata ‘Chachi’ is dark green, a green marker was applied to simulate the position of the stem, serving as the end-point (Figure 15b). The trajectory of the end-point during the relevant motion processes was observed with an industrial camera.

The first step of the visual guidance algorithm is stem identification. Stem identification mainly utilizes color threshold segmentation and connected component traversal methods. In this case, the overhead view imaging by an industrial camera separated the images of the standard sphere (or Citrus Reticulata ‘Chachi’) and the end-point (green marker point or dark green stem), which was projected onto the xoy plane. While marking the standard sphere (or Citrus Reticulata ‘Chachi’), the two-dimensional pixel coordinates of the end-point were extracted to calculate the inverse kinematics and closed-loop visual guidance algorithm. The orientation adjustment of the Citrus Reticulata ‘Chachi’ was based on its size, and its stem position was adjusted to near the center of its minimum circumscribed circle. This adjustment relied on a proportional relationship, so the image processing in this study was pixel-based and no additional calibration was required. As the determination of whether the end-point fell within the error circle (with a diameter of 10% of the pixel diameter of the minimum circumscribed circle of the standard sphere or Citrus Reticulata ‘Chachi’) was based on pixel proportions, camera coordinate transformation and pixel equivalence calibration were not required, making the process relatively simple. Figure 16 outlines the main steps of visual recognition when using the Citrus Reticulata ‘Chachi’. According to statistics, the diameter of the stem is in the range of 4~7 mm, assuming that the diameter of the Citrus Reticulata ‘Chachi’ is 65 mm, and the error band is set to 10% (that is 6.5 mm), which is within the diameter range of the stem; there is a small error as fault tolerance. Of course, in the actual use on a production line, this error band of 10% is a process parameter that can be adjusted according to actual working conditions. The larger the error band, the shorter the adjustment time, and the smaller the error band, the greater the adjustment time. In this paper, it was set to 10% because the diameter distribution of the Citrus Reticulata ‘Chachi’ is 65~70 mm most of the time, so it was reasonable to choose 10% as the error band.

The host software interface programmed in the C# language (Figure 17) eventually displays the pixel coordinates of the end-point. This interface was integrated with the constructed experimental platform of the COAM, facilitating the recording and observation of subsequent experiments.

3.2. Open-Loop Orientation Adjustment Experiment with Standard Sphere

3.2.1. End-Point Trajectory Validation

To validate the end-point trajectory, the circular trajectory in Table 1 was divided into 50 equal parts. Then, an inverse kinematics model was used to calculate the input vectors for each segmented trajectory of three driving wheels. These input vectors PW were then applied to the three individual wheels. The resulting actual end-point trajectory of the green marker point (simulating the fruit stem) on the standard sphere was compared with the given circular trajectory. As shown in Figure 18, the pixel equivalent was 0.214 millimeters per pixel. It can be observed that after applying the input vectors PW, the actual end-point path of the green marker point closely resembled the given circular path (the radius error was 2.03% and the Fréchet distance was 9.43). Thus, through the open-loop trajectory experiment, the correctness and feasibility of the inverse kinematic model were further validated.

3.2.2. Open-Loop Orientation Adjustment Experiment

After validating the kinematic model of the COAM in the previous chapters, based on the production requirements of cutting Citrus Reticulata ‘Chachi’ when fruit stems need to be adjusted to face upwards or downwards, an open-loop control experiment was conducted using a standard sphere to simulate the Citrus Reticulata ‘Chachi’. The end-point target was set as the center of the standard sphere projected onto the xoy plane.

The experimental procedure, as shown in Figure 19, involved setting up an industrial camera positioned directly above the COAM to capture and identify the standard sphere and the end-point. If the camera failed to recognize the end-point, the host software instructed the COAM to flip the standard sphere 180 degrees upwards. Once the industrial camera detected the end-point during the overhead capture, it sent the two-dimensional pixel coordinates of the end-point projected on the xoy plane to the host software via USB. The host software used the USART protocol to communicate with the Arduino MCU and sent the two-dimensional pixel coordinates to it. The Arduino MCU running the COAM program first transformed the two-dimensional pixel coordinates back into three-dimensional coordinates and then calculated the rotation angles of the stepper motor shafts for the three individual wheels using the inverse kinematics model, thereby allowing the COAM to adjust the orientation of the standard sphere.

In the xoy plane (unit: pixel), with the center of the recognized circular region as the coordinate origin, five points were randomly selected from each quadrant as the starting points for the motion of the end-point. The open-loop control experiment for standard sphere orientation adjustment was then conducted. The paths of the end-point are shown in Figure 20a–d. The orange circles represent the projection of the standard sphere outline recognized by the industrial camera. The black lines connecting the green points represent the motion paths of the end-point recognized by the industrial camera. The black arrows on the lines indicate the direction of motion. The blue circles represent the error circle (within 10% of the standard sphere diameter) at the top center of the fruit.

Whether the end-point motion path could fall within the error circle under open-loop control was analyzed. According to Figure 20e, only about a quarter of the end-point paths ultimately fell within the error circle. The accuracy of orientation adjustment under open-loop control was not ideal because of machining accuracy and assembly errors, as well as uncertainties in friction. Relying solely on open-loop control made it difficult to achieve good adjustment results. Therefore, the COAM needed to introduce machine vision guidance algorithms with feedback control to improve the accuracy of orientation adjustment.

4. Closed-Loop Experiments with Different Algorithms

4.1. Closed-Loop Experiments Using PID Control

4.1.1. Standard Sphere Experiments (PID)

After conducting the open-loop control experiment, the visual guidance control experiment based on end-point coordinate feedback using the standard sphere was continued. The experimental procedure is shown in Figure 21. After each orientation adjustment, the host software evaluated the position of the end-point; if the end-point did not fall within the error circle, the COAM continued to correct the orientation of the standard sphere. During this process, the industrial camera identified the end-point at a frequency of 20 Hz until the end-point fell within the error circle.

Similarly to the open-loop experiment, the paths of the end-point are shown in Figure 22a–d. To ensure consistency in the experiments, the standard remained the same as in the previous experiment, and it was observed whether the end-points of the path fell within the error circle under closed-loop control based on visual feedback control. From Figure 22e, it can be observed that after introducing visual feedback, the end-point of the standard sphere could eventually fall into the error circle no matter what quadrant it started from. The closed-loop control significantly improved the accuracy of orientation adjustment.

To evaluate the speed and stability of the closed-loop control with the standard sphere, further analysis was conducted on the five points’ trajectories selected from each quadrant. Time was set as the horizontal axis. The trends of pixels X and Y of the end-point during orientation adjustment from different quadrants were observed, as shown in Figure 23. While ensuring an error band of 10%, the pixel X and Y coordinates of the end-points tended to stabilize within 3 s. This indicates that under closed-loop control with visual feedback, the COAM exhibited a level of speed and stability that met the basic requirements for the initial processing and adjustment of Citrus Reticulata ‘Chachi’.

4.1.2. Citrus Reticulata ‘Chachi’ Experiments (PID)

To verify whether the COAM designed in this paper can be applied in actual production, a closed-loop control experiment was conducted using a fresh Citrus Reticulata ‘Chachi’. The experimental steps were like those shown in Figure 21 in Section 4.1, though the standard sphere shown in Figure 15a was replaced; the closed-loop experiment can be viewed on the following link: (https://www.youtube.com/watch?v=VxknxNjx85k, accessed on 1 September 2024).

Some of the stem paths achieved following the previous steps are shown in Figure 24a–d. From Figure 24e, it can be observed that after introducing visual feedback, the dark green stem could eventually fall into the error circle, no matter what quadrant it started from. The orientation adjustment accuracy was comparable to the experiments using the standard sphere, indicating a similarly high level of accuracy. As shown in Figure 25, while ensuring the error band of 10%, the pixel X and Y coordinates of the stem tended to stabilize within 5 s. The rapidity and stability were comparable to the results with the standard sphere, meeting the basic requirements for the initial processing of Citrus Reticulata ‘Chachi’.

4.2. Closed-Loop Experiments Using LQR Control

4.2.1. Standard Sphere Experiments (LQR)

Although PID control was used to achieve desired adjustments, there was a lot of oscillation. Therefore, an LQR algorithm was used for improvement. Continuing to utilize standard spheres, the experimental procedure for vision-guided control based on LQR state feedback was as depicted in Figure 26.

Most of the steps were the same as in the previous PID experiments. The primary difference was the MCU program, as the LQR algorithm within the MCU program calculated the gain matrix K based on the system’s state-space expression and ultimately calculated the output control quantity u = −Kx provided by the LQR algorithm. This ensures that the system’s state optimally approaches zero in a fast and stable manner under given performance metrics, thereby achieving system stability. Firstly, consider the state-space representation of a linear time-invariant system:

$$\dot{\mathit{x}}=\mathit{A}\mathit{x}+\mathit{B}\mathit{u}$$

(14)

where x is the state vector of the system and A and B are system matrices. The matrices A and B are as follows:

$$\mathit{A}=\left[\begin{array}{ccc}0& -{\displaystyle \frac{\sqrt{6}}{4}}& {\displaystyle \frac{\sqrt{6}}{4}}\\ {\displaystyle \frac{\sqrt{2}}{2}}& -{\displaystyle \frac{\sqrt{2}}{4}}& -{\displaystyle \frac{\sqrt{2}}{4}}\\ {\displaystyle \frac{\sqrt{2}}{2}}& {\displaystyle \frac{\sqrt{2}}{2}}& {\displaystyle \frac{\sqrt{2}}{2}}\end{array}\right]$$

(15)

$$\mathit{B}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$

(16)

The performance criterion for the system is typically represented by a quadratic cost function:

$$J={\displaystyle \frac{1}{2}}{\displaystyle {\int }_0^{\infty }\left({\mathit{x}}^{\mathrm{T}}\mathit{Q}\mathit{x}+{\mathit{u}}^{\mathrm{T}}\mathit{R}\mathit{u}\right)\mathrm{d}t}$$

(17)

where Q is a semi-positive definite diagonal matrix representing the state vector parameters and R is a positive definite diagonal matrix representing the control vector parameters. Substituting the linear state feedback controller u = −Kx into Equation (17) results in the following:

$$J={\displaystyle \frac{1}{2}}{\displaystyle {\int }_0^{\infty }{\mathit{x}}^{\mathrm{T}}\left(\mathit{Q}+{\mathit{K}}^{\mathrm{T}}\mathit{R}\mathit{K}\right)\mathit{x}\mathrm{d}t}$$

(18)

A common practice is to define an auxiliary constant matrix P such that

$${\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}\left({\mathit{x}}^{\mathrm{T}}\mathit{P}\mathit{x}\right)=-{\mathit{x}}^{\mathrm{T}}\left(\mathit{Q}+{\mathit{K}}^{\mathrm{T}}\mathit{R}\mathit{K}\right)\mathit{x}$$

(19)

Then, the Riccati algebraic equation is solved using MATLAB’s (MATLAB9.9, MathWorks, Natick, MA, USA) solver toolbox to obtain the value of the auxiliary constant matrix P.

$${\mathit{A}}^{\mathrm{T}}\mathit{P}+\mathit{P}\mathit{A}+\mathit{Q}-\mathit{P}\mathit{B}{\mathit{R}}^{-1}{\mathit{B}}^{\mathrm{T}}\mathit{P}=0$$

(20)

where the matrices Q and R are obtained through an empirical fitting method:

$$\mathit{Q}=\left[\begin{array}{ccc}0.8& 0& 0\\ 0& 0.8& 0\\ 0& 0& 0.8\end{array}\right]$$

(21)

$$\mathit{R}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$

(22)

Combining Equations (20)–(22) results in the optimal state feedback gain matrix K:

$$\mathit{K}=-{\mathit{R}}^{-1}{\mathit{B}}^{\mathrm{T}}\mathit{P}=\left[\begin{array}{ccc}1.20& 0.14& 0.90\\ 0.14& 0.65& 0.23\\ 0.90& 0.23& 1.88\end{array}\right]$$

(23)

Finally, the output control quantity u that minimizes the cost function J can be obtained:

$$\mathit{u}=-\mathit{K}\mathit{x}={\mathit{R}}^{-1}{\mathit{B}}^{\mathrm{T}}\mathit{P}\mathit{x}$$

(24)

Substituting u = −Kx into Equation (14) results in the following:

$$\dot{\mathit{x}}=\left(\mathit{A}-\mathit{B}\mathit{K}\right)\mathit{x}$$

(25)

Using the MATLAB toolbox to solve the eigenvalues of the matrix A − BK revealed that the real parts of the three eigenvalues were −0.52, −1.43, and −1.43. All three eigenvalues had negative real parts, indicating that this system under LQR control was asymptotically stable.

This output control quantity was then used in inverse kinematics to determine the angular displacements of the three stepper motors. After each adjustment of the orientation, the host computer assessed the position of the green marker point. If the green marker point remained outside the error circle, the COAM continued to correct the orientation of the standard sphere; the sample frequency remained at 20 Hz for results comparison.

The experimental data were recorded in the same way as in Section 4.1. The motion paths of some green marker points are depicted in Figure 27a–d. As shown in Figure 27e, incorporating state feedback resulted in the end-points of the motion paths of the green marker points, regardless of the starting quadrant, eventually falling within the error circle.

Using time as the horizontal axis, it was observed that the trends of the pixel X and Y coordinates of the end-points fit within the 10% error, as depicted in Figure 28. Furthermore, both the pixel X and Y coordinates of the end-points tended to stabilize within 2 s. These results indicate that the LQR state feedback exhibited certain levels of both speed and stability.

4.2.2. Citrus Reticulata ‘Chachi’ Experiments (LQR)

The control object of the standard sphere was then replaced by the Citrus Reticulata ‘Chachi’. The steps were the same as those described in Section 4.2.1, and the experimental data were also recorded in the same way. The motion paths of some green marker points are depicted in Figure 29a–d. Additionally, as shown in Figure 29e, regardless of the starting quadrant, the motion paths of the deep green stems, aided by state feedback, eventually fell within the error circle.

Using time as the horizontal axis, the trends of the pixel X and Y coordinates of the deep green stems of the Citrus Reticulata ‘Chachi’ during orientation adjustment motion as initiated from different quadrants were observed, as shown in Figure 30. It can be observed that, with the error maintained within 10%, both the pixel X and Y coordinates of the stem tended to stabilize within 2 s.

4.3. Results Comparison of Different Algorithms

To compare these two algorithms and ensure the universality of the COAM, this study repeated the above experiments using four Citrus Reticulata ‘Chachi’ fruits with sizes and weights close to the average (approximately 70 mm in long-axis size and 165 g). The experimental results of PID control showed that out of 80 trials conducted across the four quadrants with the four fruits, the average error of orientation adjustment was 3.93% (<10%) and the average stable adjustment speed was 15.64 mm/s (

Table 2. The experimental results of four Citrus Reticulata ‘Chachi’ fruits (PID control).

Numbers of Citrus Reticulata ‘Chachi’ and Points		1st Quadrant		2nd Quadrant		3rd Quadrant		4th Quadrant
		Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)
No.1 (70.90 mm in long-axis size, 166.78 g)	Point 1	3.83	11.75	4.12	18.31	7.55	13.62	4.62	18.17
	Point 2	2.61	19.82	1.81	17.16	4.43	14.70	4.58	18.01
	Point 3	8.91	17.04	1.49	16.67	6.37	19.98	4.83	15.97
	Point 4	8.74	17.04	6.81	11.65	2.52	11.73	3.45	12.95
	Point 5	5.07	10.45	2.37	15.10	5.74	16.20	6.32	17.64
No.2 (70.55 mm in long-axis size, 164.52 g)	Point 1	1.81	19.23	7.86	13.14	0.85	9.84	1.70	11.07
	Point 2	8.73	14.63	2.30	15.83	6.37	14.30	0.68	16.65
	Point 3	2.24	12.65	2.65	15.73	2.41	18.63	1.54	15.89
	Point 4	4.69	16.69	1.17	16.03	1.28	13.98	0.21	15.26
	Point 5	0.94	21.58	3.55	17.23	5.80	10.26	8.52	14.67
No.3 (69.56 mm in long-axis size, 165.22 g)	Point 1	5.11	12.77	8.43	17.63	2.40	14.52	1.98	15.69
	Point 2	3.24	16.07	1.35	20.42	4.25	11.99	1.25	18.54
	Point 3	2.89	18.16	2.59	17.60	4.83	17.10	4.63	12.51
	Point 4	3.86	16.65	7.74	14.37	1.86	12.78	5.84	14.99
	Point 5	4.95	13.88	0.51	15.37	5.12	17.52	1.66	15.47
No.4 (68.08 mm in long-axis size, 164.21 g)	Point 1	2.64	16.25	4.11	16.52	2.16	11.55	8.92	12.68
	Point 2	5.85	17.42	5.68	18.55	4.21	16.91	2.65	16.85
	Point 3	4.41	14.63	1.54	18.96	1.80	12.96	0.45	20.65
	Point 4	2.63	14.27	6.32	16.41	7.66	15.24	6.33	17.42
	Point 5	1.73	16.16	0.83	10.42	3.30	21.63	5.42	14.07
Average		4.24	15.86	3.66	16.16	4.05	14.77	3.78	15.76
Variance		5.25	7.37	6.32	5.54	4.18	9.52	6.51	5.29

). Additionally, the experimental results of LQR control showed that out of 80 trials conducted across the four quadrants with the four fruits, the average error of orientation adjustment was 3.62% (<10%) and the average stable adjustment speed was 22.81 mm/s (

Table 3. The experimental results of four Citrus Reticulata ‘Chachi’ fruits (LQR control).

Numbers of Citrus Reticulata ‘Chachi’ and Points		1st Quadrant		2nd Quadrant		3rd Quadrant		4th Quadrant
		Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)	Error (%)	Speed (mm/s)
No.1 (70.90 mm in long-axis size, 166.78 g)	Point 1	1.93	16.62	4.19	18.35	3.53	18.93	6.55	26.25
	Point 2	2.38	21.76	3.36	19.66	0.67	21.50	2.92	25.63
	Point 3	2.80	26.42	2.34	27.50	2.95	20.76	4.70	16.93
	Point 4	2.66	34.07	4.42	22.56	7.46	14.60	4.82	24.61
	Point 5	6.13	28.98	5.06	17.83	2.87	20.35	1.99	22.83
No.2 (70.55 mm in long-axis size, 164.52 g)	Point 1	1.93	23.71	3.85	29.52	4.30	27.15	0.85	19.26
	Point 2	6.69	34.51	5.65	23.99	4.36	30.23	0.74	23.69
	Point 3	4.51	30.77	5.10	20.98	7.85	14.11	4.38	23.98
	Point 4	3.78	27.24	1.85	26.56	2.95	18.76	1.45	30.44
	Point 5	2.58	16.37	3.50	20.55	6.42	26.36	2.65	22.55
No.3 (69.56 mm in long-axis size, 165.22 g)	Point 1	4.58	20.70	4.45	18.03	5.12	23.80	4.51	27.15
	Point 2	0.95	32.63	1.70	24.83	1.11	14.69	2.44	24.64
	Point 3	2.67	27.02	3.85	29.12	0.85	30.83	6.85	26.61
	Point 4	4.11	21.38	3.37	16.79	2.79	19.41	1.42	19.89
	Point 5	4.55	20.75	1.25	35.28	2.63	15.63	2.10	32.37
No.4 (68.08 mm in long-axis size, 164.21 g)	Point 1	1.25	24.37	2.75	17.83	5.41	24.06	5.25	18.83
	Point 2	2.78	17.58	7.04	15.05	2.78	16.89	2.66	30.26
	Point 3	3.45	16.31	3.42	18.26	4.50	17.37	3.30	17.60
	Point 4	2.12	13.13	4.50	24.91	3.12	25.74	3.62	22.86
	Point 5	2.76	23.08	5.85	16.56	6.30	29.89	6.17	15.57
Average		3.23	23.87	3.88	22.21	3.90	21.55	3.47	23.60
Variance		2.14	37.39	2.07	27.30	4.01	28.03	3.30	20.28

). The stable adjustment speed can be calculated with the following formula.

$$v={\displaystyle \frac{p\times \sqrt{{\left(x_{sp}-x_{ep}\right)}^2+{\left(y_{sp}-y_{ep}\right)}^2}}{t}}$$

(26)

where p is the pixel equivalent (0.214 mm/pixel), x_sp and y_sp are the coordinates of the starting points, x_ep and y_ep are the coordinates of the end-points, and t is the stable adjustment time.

The experimental results show that the COAM control with two algorithms all met the actual production requirements for the initial processing of Citrus Reticulata ‘Chachi’ in terms of accuracy, stability, and speed. In terms of accuracy, the total average adjustment error of the LQR algorithm was slightly higher than that of PID control. However, in each quadrant with the circular domain center as the origin, the average adjustment errors of the two algorithms varied, showing similar levels of accuracy. In terms of speed, it is evident that both the total average adjustment speed and the average adjustment speed in each quadrant with the circular domain center as the origin were higher for the LQR algorithm compared with PID control. This indicates that the LQR algorithm has a faster response speed and better agility.

Furthermore, a corresponding closed-loop orientation adjustment experiment was conducted on other types of spherical fruits (Figure 31). The COAM could equally achieve fully automatic orientation adjustments for them, maintaining errors within 10% and demonstrating the strong universality of the COAM and the visual guidance control algorithm for other types of spherical fruits.

5. Conclusions and Outlook

To address the issues of low automation, poor efficiency, and standardization in the initial processing of Citrus Reticulata ‘Chachi’, this study innovatively designed a friction-driven Citrus Reticulata ‘Chachi’ orientation adjustment mechanism (COAM). Motion analysis and mechanical design were conducted, and open-loop experiments indicated that the mechanism could adjust the orientation of Citrus Reticulata ‘Chachi’ fruits of different sizes and weights, demonstrating strong adaptability. To improve the performance of the COAM, machine vision-based feedback was introduced in two different algorithms for visual guidance. The experimental results under closed-loop conditions showed that the COAM could adjust the Citrus Reticulata ‘Chachi’ stem along the shortest path. The mechanism could quickly and efficiently realize the automatic adjustment of some kinds of spherical fruits, and the LQR control was better than the PID control in both speed and precision. The COAM exhibited good stability and standardization, so it could become an indispensable part of the automation production line for the initial processing of Citrus Reticulata ‘Chachi’. The friction-driven Citrus Reticulata ‘Chachi’ orientation adjustment mechanism designed in this study can become an important step on the road to the mechanization and automation of the pericarp of the Citrus Reticulata ‘Chachi’ industry.