**1. Introduction**

Agricultural robotics is an exciting, emerging research field that offers a potential solution to the problem of increasing global demand for food production due to exponential population growth and labor shortages [1]. Existing automated agricultural technologies use large and expensive machines that are strictly tied to a specific production process and usually have a significant environmental impact, especially ones used for pesticide, insecticide and herbicide application [2].

The research presented in this paper is a part of the HEKTOR project [3,4], which aims to replace such machines with heterogeneous autonomous robotic systems, capable of jointly performing different tasks in viticulture and mariculture. As a part of the project, a custom flipper-tracked mobile base was developed and equipped with a torque-controlled 7-DoF Kinova Gen3 robotic arm (Figure 1). For the task of vineyard spraying, a spray nozzle is attached to the manipulator end-effector, and its position and orientation are controlled by the robotic arm (Figure 1). This allows for precise control of the spray area, and is suitable for treating the entire vine canopy or specific areas of the plant. The HEKTOR project addresses viticulture activities that need to be carried out on steep terrain, typical of the Mediterranean islands. However, the method presented in this paper is general and was not developed specifically for steep terrains. The design of the mobile robot and the high torques of the track drives allow the storage and transport of a sufficient quantity of protective agent for spraying in conditions that are otherwise difficult for human workers.

This paper attempts to present a solution to the following problem: given a description of a row of grapevines, one must select coordinated mobile vehicle and robot arm commands that result in satisfactory canopy coverage, while aiming to minimize spraying agent waste, and perform the task as quickly as possible. This kind of problem setup calls for a control method that is able to adapt to different row structures, accelerate in areas of the row without grapevines and slow down in areas with the largest foliage heights.

**Citation:** Vatavuk, I.; Vasiljevi´c, G.; Kovaˇci´c, Z. Task Space Model Predictive Control for Vineyard Spraying with a Mobile Manipulator. *Agriculture* **2022**, *12*, 381. https://doi.org/10.3390/ agriculture12030381

Academic Editors: Jin Yuan, Wei Ji and Qingchun Feng

Received: 8 February 2022 Accepted: 7 March 2022 Published: 9 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Figure 1.** Mobile manipulator developed for the HEKTOR project, with an emphasis on manipulation ability and maneuverability in steep terrain.

#### *1.1. Related Work*

Research efforts have already been directed towards the development of robots for vineyard-related tasks [5–22]. The same applies to robots in agricultural spraying tasks [2,6–9,11,13,23]. In [21], Bouloumpasi et al. discuss the possibilities and limitations of autonomous robot technology for performing different tasks in viticulture. Vrochnidou et al. [22] present the system architecture of an autonomous robot for grape harvesting, and discuss the vision system used for the task of grape harvesting.

Monta et al. [5] designed a multipurpose robot for viticulture applications as early as 1995. Approaches for robotic harvesting, berry thinning, spraying and bagging are presented in the paper. At a conceptual level, the authors' approach to vineyard spraying is similar to the one presented in this paper, in which a spray nozzle is mounted on a robotic manipulator that follows a lawnmower pattern path. Monta et al. conclude that this kind of approach results in uniform spraying of the fixed-height foliage, but they do not discuss in detail the control algorithm used to achieve this. In this paper, an expansion of the following idea is given, able to handle different foliage shapes, with controls that allow for spraying where the mobile manipulator is able to adapt to a specific vine row description. Oberti et al. [7,8] mounted a precision spraying end-effector on a robotic manipulator and used it to spray specific disease-affected areas of the vine. In [13], Cantelli et al. present a mobile vehicle developed for autonomous spraying in agriculture, with emphasis on vehicle navigation and mission planning. Berenstein et al. [6] report on grape cluster and foliage detection algorithms for autonomous selective vineyard spraying. The authors present a mobile vehicle with multiple spray nozzles mounted at different heights and use visual feedback to select which nozzles should be active.

Similar work has been done with mobile manipulators for paint spraying [24–26]. Here, a lawnmower path is often used as a reference for the position of the tool, an approach that was adopted in the currently presented work.

Model Predictive Control (MPC) has been a subject of a number of research applications in robotics, such as manipulation, autonomous vehicle control and legged robot control [27–31]. Wieber [27] presents the application of linear MPC to the bipedal walking robot, and discusses the advantages of this kind of controller for this particular task. In [29], Elsisi presents an optimally designed nonlinear model predictive controller, and its application to the robot manipulator. The parameters for the MPC algorithm presented by Elsisi are selected by a modified multitracker optimization algorithm, rather than by trial and error. In later work, Elsisi et al. focus on autonomous vehicle control [30,31]. In [30], an MPC algorithm with a small number of parameters is presented, which is optimally

designed using the social ski diver algorithm. In [31], an adaptive MPC algorithm is used for autonomous vehicle control, utilizing a modified grey wolf optimizer, based on opposition-based learning and quasi opposition-based learning. In this paper, MPC parameters are selected experimentally, while the more sophisticated MPC tuning algorithms will be considered as part of future work.

The use of different trajectory optimization algorithms in mobile manipulation is also a well-researched topic [28,32–35]. These trajectory planning algorithms are often designed to be as general as possible, capable of performing multiple tasks while considering obstacles, mobile manipulator dynamics and stability. In [28], Pankert et al. present a nonlinear MPC scheme for continuous mobile manipulation, and show its ability to perform several tasks. The strength of the authors' approach lies in the variety of tasks it can perform, involving position or force control, while avoiding obstacles. One of the experiments presented in the paper is spraying a pattern on a flat surface, which is similar to the presented vision of vineyard spraying with a mobile manipulator. However, the method presented in this paper simplifies the planning problem by using certain insights into this particular task. The complexity of the general mobile manipulation problem includes vehicle path planning while considering obstacles, and controlling the forces or positions of the robot arm end-effector. Grapevines are typically planted in structured rows, the area between the rows being obstacle free. During vineyard spraying, the mobile base is limited to movement between the rows. For this particular application, this paper proposes a task space MPC algorithm that considers only the motion of the mobile base along the row and the two-dimensional motion of the manipulator tool. The result is a low-dimensional linear MPC algorithm that can be solved in real time even with large prediction horizons. Additionally, in the spraying experiment presented in [28], the orientation of the spray nozzle is kept constant throughout the task execution. This greatly reduces the maximum and minimum heights that the spray area can reach, which also depends on the reach of the particular robot arm. Knowing the typical foliage heights and the reach of the Kinova Gen3 robotic arm, it was concluded that constant tool orientation was not appropriate for the vineyard spraying task.

#### *1.2. Contribution*

This paper presents a novel method for vineyard spraying with a mobile manipulator that combines the following methodological elements:


Compared to other state-of-the-art autonomous vineyard spraying methods, the presented method includes multiple novelties. The lawnmower spraying reference trajectory is generated based on a specific grapevine row description, aiming to minimize spraying waste and reduce the risk of excessive pollution. The control design based on task space model predictive control allows the spraying agent to follow this reference while optimizing coordinated mobile base and robot arm movements. This results in the desired behavior, where the mobile base keeps constantly moving, accelerating in areas of the row without grapevines and slowing down in areas with foliage of the largest heights. The predictive nature of MPC allows for such behavior, which would be difficult to achieve using standard, instantaneous control methods. Predictive control allows the controller to anticipate any changes in the reference trajectory and optimally select how the mobile base and the robot arm should react to it.

Compared to the state of the art in mobile manipulation, a control algorithm for the specific purpose of vineyard spraying is developed instead of using a general trajectory optimization algorithm. Unlike the general trajectory optimization algorithms, which usually include end-effector pose planning, the presented algorithm combines MPC and task space control in such a way that the MPC plans only the positions of the end-effector while disregarding its orientations, which are handled by an instantaneous task space control algorithm.

This kind of approach results in two quadratic programming problems, for MPC and task space control, which can be solved in real time even for large prediction horizons.

To summarize, our contributions are as follows:


#### **2. Task Space Model Predictive Control Approach**

Since the vines are usually planted in structured rows, it is assumed that the vehicle moves in a straight line at a constant distance from the vines, and, because of this, the spray reference for the MPC is generated in a two-dimensional space. As shown in Figure 2, three coordinate frames are defined: a ground-level global frame *LG*, a mobile base frame *LB* and the spray frame *LS*. The spray frame *LS* is defined at a fixed distance from the spray nozzle, and its position is computed using a single static transformation from the last link of the robot manipulator. The goal is to control the global position of the *LS* frame, which depends on the position of the *LB* frame with respect to *LG*, and the pose of the robot arm. The *y*-coordinate of the *LS* frame is assumed to be constant at the MPC stage, which is enforced by task space control. The *z*-coordinate of the *LS* frame, *pS*,*z*, is controlled by the robot arm alone, while the *x*-coordinate *pS*,*<sup>x</sup>* is a sum of two components

$$p\_{S,x} = p\_{B,x} + p\_{A,x} \tag{1}$$

where *pA*,*<sup>x</sup>* is the *x* coordinate of the position of the *LS* frame with respect to *LB*, controlled by the robot arm, and *pB*,*<sup>x</sup>* is the *x* coordinate of the position of *LB* with respect to *LG*, controlled by the mobile base.

**Figure 2.** Three coordinate frames are defined: a global frame at the ground level *LG*, mobile base frame *LB* and the spray frame *LS*. The *x*, *y* and *z* axes of the coordinate frames are represented with red, green and blue arrows, respectively.

The overall control diagram of the system is shown in Figure 3. The reference trajectory for the spray frame is computed based on the canopy description, explained in Section 2.1. This reference is used as an input to the MPC solver, which selects optimal mobile base and manipulator task space trajectories. Only the first set of optimized control inputs is sent to the robot, after which the trajectories are recalculated based on the system state feedback and the updated reference trajectory. The task space commands for the manipulator are converted into joint velocities by the task space controller.

**Figure 3.** Overall system control diagram. The trajectory of the reference spray frame is generated based on the canopy description and used as input to the MPC solver. The MPC solver provides the velocity of the mobile base along the row and the velocities of the robot arm in the task space. The task space control solver converts the desired velocities in the task space into joint velocity commands *q*˙ .

#### *2.1. Reference Spray Frame Trajectory*

A reference lawnmower trajectory is generated within the upper and lower boundaries of the foliage, with the aim of providing spray coverage while reducing waste (Figure 4). The foliage canopy description is used to compute the vertices of the lawnmower pattern, along with the Spray Width and Height Offset parameters (Figure 4). These parameters are tuned with respect to the spray pattern of the nozzle mounted at the end-effector of the robot arm. A constant velocity piecewise linear function is used as the reference trajectory between the lawnmower pattern vertices. The reference velocity of the spray frame, together with the flow of the spraying agent through the nozzle, affects the coverage of a fixed area of the canopy. For a fixed amount of spray flow, lower reference velocity will result in more coverage.

**Figure 4.** A reference lawnmower trajectory is shown with an orange line. The canopy description is represented by a blue and a red line, representing the upper and lower boundaries of the foliage, respectively.

The canopy description can be created either manually, by an operator, or using foliage detection. In the experiments presented in this paper, the manual approach was used to select the boundaries. In the future work, the plan is to use a foliage detection algorithm for this purpose.
