Soft Robotics as an Enabling Technology for Agroforestry Practice and Research

Abstract

The shortage of qualified human labor is a key challenge facing farmers, limiting profit margins and preventing the adoption of sustainable and diversified agroecosystems, such as agroforestry. New technologies in robotics could offer a solution to such limitations. Advances in soft arms and manipulators can enable agricultural robots that can have better reach and dexterity around plants than traditional robots equipped with hard industrial robotic arms. Soft robotic arms and manipulators can be far less expensive to manufacture and significantly lighter than their hard counterparts. Furthermore, they can be simpler to design and manufacture since they rely on fluidic pressurization as the primary mechanisms of operation. However, current soft robotic arms are difficult to design and control, slow to actuate, and have limited payloads. In this paper, we discuss the benefits and challenges of soft robotics technology and what it could mean for sustainable agriculture and agroforestry.

1. Introduction

Conventional annual row-crop agriculture, while productive and labor efficient, has resulted in numerous well-documented negative externalities [1,2,3]. With the advent of modern mechanization and coincident increase in rewards to non-agriculture labor, modern agriculture has rapidly and decisively shifted towards simple, large-scale, annual monocultures. By contract, agroforestry and perennial polycultures have been proposed as a sustainable alternative with well-documented environmental benefits, but these systems generally require significantly more labor, skill, and knowledge [4]. This concept paper introduces a set of technological innovations in that will enable viable agroforestry and sustainable agriculture systems that include a wide diversity of species. The coming age of robotics and machine intelligence could play a key role in allowing a profitable and practical sustainable agricultural system at the large scale necessary to contribute meaningfully to global food security.

The environmental benefits of the integration of a diverse set of trees and shrubs in agroforestry systems include increasing capacity for carbon sequestration, nutrient cycling, soil stabilization, and biodiversity conservation compared with conventional agricultural systems [5]. Recently, research on agroforestry has expanded beyond regulating services, to consider the potential for trees and shrubs to contribute directly to food production, through the purposeful selection of species that yield edible fruits and nuts [4,6]. The products from agroforestry can contribute to healthier human diets and greater economic diversity for landowners. An example of a Midwest US alley cropping system that includes tree/shrub polycultures is described in greater detail by Lovell et al (2019) and the concept is visualized in Figure 1.

Despite the benefits of productive agroforestry, adoption in temperate regions has been limited. Many of the concerns from landowners are related to the complexity of systems that contain a mix of different species with perennial growth habits. Labor demands are high for crops that are maintained and harvested manually and new knowledge and learning is needed to manage mixed species [7,8]. Furthermore, research on the performance of unfamiliar tree and shrub species in a variety of different environments is lacking. These factors are critical barriers that seriously limit widespread adoption when the upfront investment in perennial systems is high and the economic return is long-term.

In this paper, we discuss a recent technological advance—soft robotics—and its implications for productive agroforestry and additional diversified sustainable agriculture systems. Soft robots and other such recent advances in computer science and robotics could contribute substantially to overcoming these barriers, making these agricultural systems a more viable option, even in regions with limited labor resources and seasonal needs. In order to be feasible, the new technologies must be inexpensive and highly mobile on the rough terrain of a field setting. They must be able to identify the targets and carefully manipulate plants and their harvestable products. Ideally, the devices would have the capacity to adaptively learn how to navigate the landscape and to optimally perform activities with the plants. They could even contribute to research by deploying the sensors and collecting data necessary to study system performance for future improvements. Below, we provide evidence that suggests that robotics and machine intelligence could enable a transition to profitable large-scale implementation of these productive, environmentally beneficial agricultural systems.

2. Background

2.1. Problem: Human Labor for Complex Systems

For many food crops, and particularly for specialty crops such as fruits and berries, the US is experiencing a serious labor shortage [9], both in terms of quantity of labor available, and the cost of available labor. The situation is even more critical with sustainable complex systems like productive agroforestry and perennial polycultures that include multiple species requiring a broad range of specialized tasks throughout the growing season [4,10]. Timely harvest of berries can be particularly critical in order to maintain the most antioxidants provided by the plants [11]. Some berry harvesters are available, but they are not designed to navigate the complexity of mixed species and they offer limited quality detection capabilities. Even for commodity crops, increasingly autonomous large equipment [12,13] only addresses part of the problem, especially since it cannot provide precision care once the canopy closes and because it causes soil compaction. This has led to recent interest in smaller-sized robots [14,15,16,17]. There has also been interest in commercializing some of these robots, including the Naio robot, the Fendt Xavier concept, and the Bosch robots. However, robots with traditional hard, heavy, and expensive arms have proven to be inadequate at providing the required reach, delicacy, and strength while meeting demands on cost, field-reliability, and ease-of-use [15,18,19,20]. We refer the reader to a recent survey on agricultural robots for further details on the state-of-the-art in this area [21].

2.2. Solution: Agbots to Perform Complex Tasks

Soft robots are making significant advances [22] due to their low-cost, high dexterity, and high strength-to-weight ratio, yet this technology has not been utilized in agriculture. This technology is unique in its scalability because it is inexpensive and has unprecedented reach and dexterity. The hope with this technology is that it can be employed on a team of small robots. Such a team of robots can be scaled freely from small farms, which would need a few robots in the team, to large farms, needing many. The robots could work in collaboration with each other, completing intricate tasks together, as well as alongside humans, augmenting their capabilities. This feature is in stark contrast with existing heavy equipment, which is only profitable at the large scale and results in high energy use and soil compaction. The technology is distinctive in its customizability, with soft-arms and manipulators that are multipurpose and dexterous, capable of accomplishing a variety of agricultural tasks due to their high degrees of freedom, compliant nature, and varied force profile. Agbots will be able to quickly adapt to multiple tasks, in contrast with large equipment and hard-armed robots. The technology is attainable, with simple construction that is inexpensive to manufacture and maintain. With new designs, agbots will be far more effective and easy to use, yet cost far less, lowering barriers to entry for agrobotics on small, diversified farms and large commodity farms. Finally, agbots are socially responsible in addressing the agricultural labor crisis for both large-scale and small-scale farming to be more profitable and sustainable.

2.3. Research Concept

A research team at the University of Illinois is currently developing small, low-cost, and autonomous agricultural robots (agbots) with “soft” arms and manipulators (Figure 2) that could be the dexterous and multipurpose co-robotic tool to address the complex requirements of productive agroforestry systems. The project is jointly funded by NIFA through the NIFA-NSF National Robotics Initiative (USDA 2019-67021-28989) and supported by EarthSense Inc. The major challenge in realizing agbots for commodity as well as specialty crops is the diverse and dexterous nature of the tasks required to be performed. For example, in a berry/nut polyculture, such tasks could include scanning berries for optimal ripeness, harvesting, controlling weeds, detecting insects and diseases throughout the dense plant canopy, and pruning and thinning branches. The complex and dexterous nature of agricultural tasks is the primary reason why traditional industrial robots with arms and manipulators made of rigid and heavy materials actuated with expensive servo-motors have not been successfully used at scale in production agriculture. Indeed, manipulating squishy berries in difficult to reach places with “hard” arms is difficult at low-cost.

Our preliminary and ongoing work described here suggests that equipping small robots with soft and flexible arms could lead to heterogeneous teams of collaborative low-cost (<$1000) and lightweight (<15 kg) agbots. Thus, these agbots have an unprecedented ability to reach difficult targets and apply a wide range of forces to achieve objectives that are difficult or nearly impossible for traditional “hard” armed robots. Moreover, agbots with soft arms could cost orders of magnitude less owing to simple construction with plastic tubes and relatively low-cost pneumatic compressors, weigh far less than metal arms, and can be highly modular.

The long-term research goal is to make these soft-armed multipurpose and dexterous agbots a reality in agriculture by driving fundamental advances in soft arm design, integration on low-cost agbots, and autonomous control. In this concept paper, we describe the different research thrusts that we believe are necessary to bring soft robots to fruition in food production systems. Our approach is highly interdisciplinary, relying heavily on expertise in agroforestry, robotics, modeling, sensing, and control. It is ambitious in scope, yet we expect that the fundamental advances made in pursuing the research directions laid out here will bring us closer to cost effective and more sustainable practices.

3. Soft Continuum Robots

3.1. Tradeoffs in Dexterity, Load Capacity, and Energy

Soft continuum robotic arms are inspired by octopus tentacles and elephant trunks. They are hyper-redundant in nature and undergo large continuum deformations by utilizing material elasticity [22,23,24,25,26]. As a result, they are dexterous [27], have large accessible workspace, and are safer for human interaction, as they tend to absorb rather than transmit impact energy [28]. This has prompted a number of applications from soft-gripping, grasping, manipulation devices with applications in manufacturing products, assisting the elderly, and performing surgery [29,30,31,32,33,34,35,36]. Soft robots are ideal for agricultural applications but unaddressed design challenges remain as they depend on mostly two actuating principles: (1) large strain elastomeric members deferentially pressurized by air [27,30,37] and (2) a soft continuum backbone driven by tendons [38], with programmable stiffness due to varying material or geometric properties [39]. In the design of soft-arms, there is a clear trade-off between dexterity and load capacity on the one hand and energy consumption on the other (see

Table 1. Qualitative comparison of the state-of-the-art soft continuum manipulators.

Continuum Manipulator	Dexterity	Load Capacity	Whole Arm Manipulation	Energy Savings
OctArm [40,41]	High	Medium	High	Low
Tendon Driven [39]	Medium	Low	Low	Medium
sPAM Manipulator [42]	Low	High	Low	High
Stiff-Flop [38]	Medium	High	Low	Medium

). For applications in productive agroforestry, researchers must address these gaps in the design to allow for the dexterity and load capacity needed to perform in field environments and to overcome the energy tradeoff. To achieve this, advances in modeling-based tools for principled soft robot design and techniques for sensing and autonomous control of soft and continuum robots are needed.

3.2. Fiber Reinforced Electromeric Enclosures (FREEs)

Novel pneumatic soft building blocks known as fiber-reinforced electromeric enclosures (FREEs) [43,44] have been developed to allow greater freedom in the robotic movements. FREEs are hollow cylindrical elastomeric membranes with two families of fibers wrapped on the surface at angles a and b with respect to its longitudinal axis as shown in Figure 3a,b. FREEs are similar in construction to McKibben artificial muscles [45], but are capable of generating different deformation patterns including extension, contraction, and rotation. The nature of this deformation is determined by the fiber angles. The fabrication methodology for FREEs has been published previously [43,44].

3.3. Asymmetric Continuum Arms

Further optimization of the movement of robotic arms could come with the use of a unique family of continuum manipulators that utilize asymmetric combinations of FREEs. State-of-the-art soft continuum manipulators constitute several symmetric bending segments serially linked to enhance a spatial workspace [27,40]. In another approach using a parallel design, each FREE actuator behaves alike and their parallel architecture results in spatial bending in different directions (dotted lines in Figure 4a). In contrast, new asymmetric FREEs could potentially yield more complex motion shapes. Figure 4b illustrates a $B{R}^2$ architecture where an extending FREE (B) is combined with two FREEs that axially rotate ($R^2$) upon actuation. The resulting combination yields complex spatial deformation as shown in Figure 4c,d without the necessity of serially linked segments. The greatest advantage of the proposed asymmetric $B{R}^2$ architecture over the symmetric one is the former’s ability to deform spirally. This is a new capability for whole-arm manipulation and especially useful in handling large irregular shaped objects using the entire body (see Figure 4d), a feature of the great use for agbots pulling out weeds, pruning branches, or other complex activities required in productive agroforestry systems (Figure 1).

4. Dynamic Modeling and Simulation

From a modeling and mathematical perspective, soft robots, unlike conventional ones, can leverage structural instabilities to cope effectively with complex and dynamic environments such as productive agroforestry systems (Figure 1). In other words, non-linear mechanics can be exploited to outsource control tasks to the design, a new ‘morphological computation’ or ‘embodied cognition’ [46,47] paradigm. This is exciting but it is also difficult to incorporate these strategies into a practical robot, partly because of the lack of principled simulation, analysis and design tools [26]: indeed, virtually all studies on soft and continuum robots primarily rely on trial and error. Broadly for robotics, and specifically for cost-sensitive ag-robotics, this approach is not sustainable or scalable. The few attempts to integrate a realistic simulation to aid the engineering process [26,41,48,49,50,51] have also relied of trying different solutions, only shifted at the computational level. The potential exists, however, to leverage expertise in physical modeling and high-performance computing to enable high fidelity computer simulations of fluid filled FREEs and utilize them in an automated inverse design process (evolutionary optimization) to accelerate and establish a rational design process [52,53]. Moreover, realistic simulations can also be used in conjunction with learning-based algorithms for testing control algorithms and discovering control policies to be translated in agroforestry practices.

In a soft robotics context, minimal mathematical modeling might provide qualitative insights for simple settings and asymptotic limits [54,55], but it is of little use from a quantitative, dynamic perspective. On the other hand, high-fidelity 3D FEM simulations also present severe limitations: prohibitive computational costs, numerical instabilities and loss of accuracy due to discretization elements’ distortion, ad-hoc (re-)meshing, and difficulty in implementing complex boundary conditions. To overcome these limitations and retain accuracy, robustness, and versatility, novel modeling approaches based on Cosserat rod assemblies have been developed [56,57]. These complement and extend previous soft-robotics models [26,41,49,50,51,53] by enabling the simulation of complex, composite architectures in a variety of context and environments [58]. This representation entails a number of major advantages:

• Captures 3D dynamics accounting for bend, twist, shear and stretch, fundamental effects for elastomeric or biological materials
• Continuum actuation, interface, and environmental effects can be directly combined with the body dynamics, rendering the inclusion of (self-)contact, friction, muscular activity, hydrodynamics and adhesion (Figure 5)
• It dramatically reduces computational costs, especially when compared with FEM solvers.

Thus, the Elastica solver has been developed [56,58,59] to facilitate and disseminate the use of these models in practical engineering applications. The solver has been validated on a large number of benchmark problems with known analytic solutions and against experimental investigations involving (self-)contact, anisotropic surface friction, highly viscous [58] and inertial fluids (Figure 5). It has been verified experimentally for artificial muscles [60], linking topological changes to mechanical work and extended to simulate arbitrary complex musculoskeletal layouts (Figure 5). In the context of this concept paper, the Elastica solver becomes a valuable tool for the modeling, design and simulation of new soft robots targeted to activities related to agroforestry systems’ management.

5. Agbot Autonomous Rovers for Maneuvering Field Sites

In order to deploy soft arm robotic devices in the agroforestry field setting, a robust rover is needed. The rigorously field-tested TerraSentia (Figure 6) [60,61] is an ultra-compact (14 inches wide), ultra-light (14.5 lbs), low-cost, autonomous agbot that can navigate in a variety of field conditions autonomously. TerraSentia is (1) ultra-light-weight compared to even small robotic rovers (e.g., the 37 lb. Clear Path Husky); (2) ultra-compact, capable of turning 3600 on the spot in 30 inch rows; (3) fully autonomous under dense crop canopy where GPS does not work; and (4) uniquely designed (patent pending) to not damage young plants even if it drives over them. Finally, and most importantly, for lowering barriers to entry, TerraSentia can be manufactured at an extremely low cost due to the simple design forms and lack of mechanical complexity.

5.1. Mechanical Design

The TerraSentia agbot can be customized with an entire suite of sensors attached to assist in perceiving the environment (see Figure 6). The ultra-light and ultra-compact design enables the robot to have an 8.5+ h battery life with 12V LIPO batteries. TerraSentia is built using additive manufacturing (3-D printing), enabling researchers to rapidly iterate more effective and robust designs, as opposed to lengthier iteration processes with metal fabrication. The same iterative mechanism can be used for integrating the continuum arms discussed previously. TerraSentia traverses rough field terrains successfully due to fine-tuned ground clearance, independent suspension on all four wheels, and independently controlled powerful motors with adaptive traction [62]. The wheels of the robot were also selected to minimize point-pressure while maximizing motion control through an extensive field testing of several different wheel designs. The wheel shape also allows sufficient gaps so that plants do not get sucked under the wheel, as would happen with a more traditional tracked robot. The robot has sufficient interior space to house advanced computers, including GP-GPUs, Intel i7 processors, multiple motor controllers, and inertial sensors.

5.2. Motion Control and Autonomous Navigation with LIDAR

For agroforestry applications, the robot must have the ability to move autonomously through a rough field environment. Researchers have designed an algorithm that estimates and accounts for robot wheel slip, and this approach has been validated in several field trials [62,63]. Field trials indicate that the autonomy system is highly accurate in navigating within rows when the reception for real-time kinematic (RTK) satellite based-positioning systems (GPS) is good. However, GPS does not work well under mature canopy or near taller berry plants due to multi-path attenuation error. To address this challenge, a Light Detection and Ranging (LIDAR) based navigation system has been developed and field tested (Figure 7).

5.3. Onboard Machine Vision and Analytics

Sophisticated machine vision systems for have been developed for TerraSentia, and their effectiveness has been demonstrated for quantified phenotyping with plant counting and plant width estimation. In an example of field corn-counting operation [60], TerraSentia drives through a row with a side-facing camera. A real-time machine vision algorithm using a customized compact deep learning model recognizes corn stalks at any time during the season. A vision-based motion estimation technique is used to determine the relative motion between the sensor on the moving robot and the corn stalks in view to minimize double-counting. For stem-width operation, a foreground extraction algorithm recognizes cylindrical plant stems from a high degree of visual clutter. A structure from motion algorithm uses wheel encoder velocity (the best estimate under canopy, since GPS does not work) and optical flow of the side-facing camera to estimate relative depth. The stem width is computed using this depth estimated using deep flow and the identified stem isolated using machine vision filters [64]. These results demonstrate the types of sophisticated level of machine vision capabilities that can be embedded onto compact robots such as the TerraSentia by leveraging recent advances in deep learning. Similar tools can be designed for productive agroforestry to estimate the density and structure of woody plants, count nuts, berries, or fruit, detect diseases, and monitor plant growth.

6. Developing Agbots for Agroforestry Practice

The integration of small, low-cost, and autonomous agricultural robots (agbots) with “soft” arms and manipulators described previously could be the dexterous and multi-purpose co-robotic tools that can address the labor issues that are a primary barrier to adoption of agroforestry. In order to meet the research goal of making these soft-armed multipurpose and dexterous agbots a reality in agriculture, more work is needed. With the successful completion of this ongoing research project, the integration of soft continuum arms on small ag-bots will make the management of intricate agricultural tasks with robots practical. Arms with better reach, dexterity and load capacity are necessary for realistic ag-manipulation and management tasks, especially in productive agroforestry including polycultures and in commodity farms where ubiquitous robots with dexterous arms can collaborate to accomplish intricate tasks.

The research efforts required to realize such arms would provide multiple opportunities for crosscutting research in robotics, mechanical engineering, agricultural engineering, computational modeling, machine learning, and agricultural sustainability. This novel interaction across disciplines and viewpoints can induce highly innovative workflow in research teams. Such projects have the potential to significantly expand the equipment available to address the agricultural labor shortage while advancing the science of soft-robotics through a meaningful and challenging application.

6.1. Design and Integration of Soft Continuum Arms

In order to realize the potential of robotics for agroforestry, a model-based design of soft continuum arms will be developed to explore fundamental design principles in robotics. High-fidelity computational models of soft continuum arms have the potential to lead to the discovery of design principles for small Agbots with continuum arms. The research team seeks to design an optimal continuum multipurpose manipulator for agricultural applications with a focus on berry plants found in productive agroforestry systems. While several continuum manipulators exist, none have addressed the trade-off between workspace, load capacity, dexterity, and energy consumption. The fundamental principles behind the design of continuum manipulators will be discovered by a combination of a dynamic model-based evaluation and experimental testing.

Different architectures of the manipulator will be evaluated for dexterity, load capacity, ability to perform whole arm manipulation, and energy consumption—all important characteristics for performing key activities for agroforestry management in the field. A potential architecture of the manipulator on the Agbot is shown in Figure 8a. We envision the continuum arm to be encased in a rigid cylindrical shell (Link II), which is mounted on a rigid link robot (Link I) with up to tow links to greatly increase the reach. This architecture utilizes a hybrid approach by leveraging the advantages of both the rigid and soft embodiments: rigid when the continuum arm is completely retracted into the casing and soft when deployed out. The arm itself is conceptualized as a series combination of up to two modules (A and B, in Figure 8b). Each module may contain up to four FREE actuators, where some of the FREEs may be used as programmable stiffness members by vacuum assisted granular jamming. The end of the continuum manipulator will house a camera and a suction gripper or a mechanical gripper. The parameters for design are thus, (i) the number of rigid links (0,1 or 2), (ii) the number of series sections (1 or 2), (iii) relative lengths of the series sections (LA, and LB), (iv) the number and type of FREEs per module including their fiber angles (3 or 4), (v) the diameter and stiffness of each FREE (diA, and diB). Each of these design variables influence the trade-offs in arm performance.

Our preliminary results point towards a way of meeting the design requirements using optimally designed FREE actuators. Previous research [23] has demonstrated that multiple serially actuated sections increase dexterity and load capacity. However, this also increases complexity in design, controls, increases overall inertia and energy consumption. To increase dexterity without many serial modules, the inclusion of asymmetric FREEs (a combination of rotating and bending FREEs) could be used. Such an asymmetric combination inherently supports spiral bending, which can lead to whole arm gripping. Thus, each continuum section will constitute at least one bending FREE and two rotating FREEs.

In order to conduct the intricate manipulations required in the management of productive agroforestry systems, a gripping action is needed. Most continuum manipulators utilize a suction gripper at its tip to grip small objects [40,42]. If robot-mounted vacuum pumps are employed for vacuum-assisted stiffening, this may be a viable option. Otherwise the additional vacuum generating element may further add on to the energy requirements of the robot. To circumvent this, a novel mechanical monolithic compliant gripper has been designed that can simultaneously grip and sever fruit by twisting. The design consists of an adaptive gripper that sits on a flexure base. The compliant flexures are so oriented that just one actuation (pulling on the string) will both grip and twist the base and sever the fruit from the branch (Figure 9a,b). The gripper has been tested on large fruits such as apples, mangoes, and tomatoes. The gripper will be scaled down, designed to work with bunches or individual berries, and mounted on the tip of the manipulator for berry harvesting in productive agroforestry systems, as shown in Figure 9c.

A dynamic model can be used to computationally explore the functioning of the continuum arm. Particularly important is the ability to estimate the gripping forces when external loads induced by the field environment are at play. The design of FREE based robots’ need to explicitly take into account the environmental forces at the design stage (prior to fabrication)—when interweaving fiber patterns, geometrical and material properties are determined.

6.2. Autonomous Control with Learning

The robot itself can drive reliably using GPS-based autonomy only when the view of the sky is unobstructed. In the presence of clutter, such as under a forest canopy, GPS performance severely degrades due to multi-path errors. These errors are a result of the signal from the satellites bouncing off the canopy, which changes the time it takes for a particular time-stamped signal to reach the rover. These small changes in time add up quickly in position errors due to the very high speed of light. In addition, there is also the likelihood of complete signal obstruction in very dense canopies. In corn and sorghum fields, TerraSentia mitigates this issue by utilizing LIDAR-based navigation [61]. LIDAR, or Light Detection and Ranging-based units are similar to RADARs in their operation, except for the fact that they use infrared laser lights for ranging. The LIDAR returns a point cloud with which the robot can control itself to stay away from elements of the field such as plants. However, range data alone are not sufficient to disambiguate elements in the field into those that are surmountable and those that are not. An important future direction of research is to combine visual and LIDAR data.

For Agbots to be useful for performing agroforestry tasks, they must work autonomously and have the ability to “learn” as they go. Human demonstration of movement in these agroforestry tasks can be documented and used to provide initial guidance for enabling diverse tasks in complex field environments. It is challenging to design optimal controllers for continuum arms with traditional model-based control methods [42,51,65] due to the large numbers of degrees of freedom of continuum arms and the large number of pressure valves. These factors also make them notoriously difficult to control by humans. Inspired by recent successes of Deep Reinforcement Learning (DRL), the UIUC research team will explore the effectiveness of direct vision-based reinforcement learning for finding optimal control policies for controlling continuum arms.

The “learning” will be the result of leveraging demonstrations from expert operators of continuum arms to reverse engineer the human operators reward function to be utilized. Learning DRL policies directly from simple rewards (e.g., a reward when the arm picks a berry) will lead to an enormous exploration problem, yet, it is hard to code by hand the specifics of every task. To resolve this conundrum, the UIUC research team will build on exciting results in inverse reinforcement learning from human demonstrations for dynamical systems with large state-action spaces that were developed by Chowdhary et al. [66,67]. This concept can be generalized to enable learning of efficient and scalable generative models of the actions generating the trajectories of continuum arms. Previous work has depended on motions that depend on discrete changes in the end-effector motion of the arm [66], but continuum arms are designed to move in a smooth continuous fashion. To model fluid evolutions in motion, the research team will model transitions in arm states using a framework developed by Chowdhary [68,69]. Models of arm motion will be created to demonstrate the execution of particular tasks required for agroforestry management. The second stage of training will be in the real-world, beginning first with the green-house and then progressing into the field.

Leveraging the reward functions learned from experts, the robot can be taught optimal policies for executing multiple complex tasks. Given the complexity of continuum arm models and the very sparse literature that exists on the control of continuum arms [42,51,65], Reinforcement Learning emerges as a good approach to begin with, at least until the advances in this project lead to the discovery of new ways of controlling such arms. DRL policies will be applied for the control of continuum arms that are robust to real-world uncertainty using imperfect simulations. The success in the agroforestry applications will have tremendous implications in making DRL practical for robotics, autonomy, and control of other complex systems. While Deep Reinforcement Learning (DRL) has been shown to find optimal policies for challenging continuous-space robotic applications [70,71,72], real-world physical system applications of DRL in uncertain field environments still pose several difficulties. One of the issues is that DRL requires a large amount of data for training. For computer games and other software-based domains, training can be accomplished using computationally generated data [73,74,75]. However, for physical robots, one has to resort to simulations. The high fidelity simulation models will certainly provide us a great place to begin with, however, even this simulation model will not be able to replicate exactly the variations and uncertainty of the field, hence leading to non-robust policies in the real world.

The reinforcement loop will can be completed with sensor data from an arm-facing camera. The approach is inspired by how DRL was used in playing ATARI [73] with Convolutional Neural Networks (CNNs), and it builds significantly on our own prior work in using CNNs on agbots. However, there are many challenges with real-world sensing that were not present in the ATARI setting. In particular, scenes in the real world can be partially occluded, which requires algorithms capable of predicting and handling motion. Comparative experiments will be conducted to find the best way to train the neural network to extract the most relevant feature vector. The arm-facing camera will be emulated in the simulation so that training in simulation matches with the real world for robust training with our adversarial RL method.

6.3. Example Results with Deep Reinforcement Learning for Soft Robots

The $B{R}^2$ soft arm investigated for this application undergoes large nonlinear spatial deformations due to both inherent actuation and external loading [76]. The physics underlying these deformations is complex and often requires intricate analytical and numerical models. The complexity of these models may render traditional model-based control difficult and unsuitable. Model-free methods offer an alternative for analyzing the behavior of such complex systems without the need for elaborate modeling techniques. In our recent work [77] we presented preliminary, yet promising results on the use of Reinforcement Learning (RL) for position control of the $B{R}^2$ soft arm. The main benefit of RL over other neuro-adaptive control strategies [78,79,80] is that RL directly learns an optimal policy from experience. This precludes the need for a separate control strategy to choose optimal actions while transitioning between states. Control policies obtained using RL techniques are also more robust to external disturbances, making them ideal for the $B{R}^2$ manipulator, whose workspace is dependent on external loads.

The soft manipulator system is abstracted as a Markov Decision Process (MDP) and a Deep Q-Learning Network (DQN) is used to learn the optimum policy. The MDP is characterized by three quantities: (a) State, which is the 3 × 1 vector of the end position coordinates, (b) Action, which corresponds to 12 discrete pressurization and depressurization events that lead to different modes of manipulator bending, and (c) Reward function, which quantifies an effect that an action has on the manipulator states. The reward function is usually the Euclidian distance between the current and the required end position. It penalizes every transition and specifically those that push the end effector position outside the workspace of the manipulator. The optimal action-selection policy is learnt using Q-learning, which is a model-free reinforcement learning technique. This technique defines a quality function Q and updates its value using the Bellman equation, which, in turn, is based on the reward function. To account of the large dimensionality, neural networks are used to approximate the Q-functions and such a framework is called Deep Q-Learning network (DQN). The optimal policy is given by the maximum value of the Q function over the action space.

We trained a BR² manipulator with a length of 0.31 m and maximum operating pressures of 172.36 kPa in bending and 193.05 kPa in rotation. We used a DQN with three hidden layers having 512 nodes and tanh activation functions. To expedite the training, we used the model developed in Uppalapati et al. (2018) [76] to train the system on a simulation. The simulation was based on a Cosserat rod formulation [37], with elasticity and precurvature parameters fitting with experimental results. A similar training scheme could be directly implemented on a prototype in an automated manner. An Adam stochastic gradient optimizer (learning rate $\alpha =0.002$) with mean squared error as the objective function was used to train the NN. In each episode, a tuple of points was randomly selected from the training dataset and the system was trained to transition between the two points using the $ϵ$-greedy policy. The episode was terminated in one of three ways: (1) the position error is below the threshold value, (2) number of steps exceed allowable limit and (3) the manipulator pressure exceeds the maximum rated pressure. The system was trained for 5000 episodes before evaluation. During evaluation, a greedy policy was used to select the best actions from the learned Q function to transition between states.

The learnt policy was implemented under open loop conditions on the $B{R}^2$ prototype as shown in Figure 10a. The prototype was asked to reach five waypoints on the workspace, as shown in Figure 10a. Some of the points are in the extremes of the workspace. The trajectory that resulted from implementing the learnt policy reveals the following: (a) when transitioning between waypoints, the system takes larger strides initially and switches to smaller stride as it gets closer to the target, and (b) when a target is located at the edges of the reachable workspace, the system learns to avoid taking actions that would exceed the prescribed pressure limits, thereby preventing it from getting stuck at a local minimum. Furthermore, to illustrate the robustness of the control policy to tip loads, 10,000 random trajectories were generated and the average errors for the tip positions were evaluated for different tip loads and tabulated in Figure 10c. It is seen that for a maximum tip load of 18 gms, only 164 out of the 10,000 trajectories exhibited an average error of 3 cm, which is the diameter of the manipulator.

Future work will involve incorporating the system with sensors to create a framework with real-time feedback of the BR²’s tip position. Closed loop control, coupled with online policy adaptation based on the feedback, could significantly improve the tracking performance. In addition, we also plan to extend the system for more complex manipulators with greater degrees of freedom.

6.4. Evaluation of Agbots in Agroforestry Field Environments

The effectiveness of soft continuum arms will be evaluated in real productive agroforestry systems with berry shrubs, testing tasks that require dexterity of reach, load capacity for manipulation, and strength of grasp. Berries are chosen as the primary focus for evaluation since management activities with these plants are challenging and intricate, and hence, most labor-intensive. This focus on specialty crops is not limiting, since the arms will also have immediate applications in commodity crops.

The performance attributes of soft continuum arms relevant to the agricultural application are dexterity, load capacity, the ability to manipulate using the whole arm, and energy consumption. Dexterity is associated with the workspace of the manipulator and its ability to move in all directions, starting from a given pose [27]. Load capacity is the ability of the arm to retain its workspace and dexterity under the application of loads, either due to payload or gravity. It also relates to the ability of the arm to impart sufficient forces on its target. Whole arm manipulation involves the use of the entire length of the arm in gripping or manipulating the object (see Figure 3). The last attribute is the energy consumption of the manipulator over time.

Berry crops that are integrated into the productive agroforestry systems are located at an existing site established in 2015. The berry crops include aronia berry (Aronia sp.), elderberry (Sambucus canadensis), and black currant (Ribes nigrum)—all planted in the rows between nut-producing trees. In the greenhouse, ripe berries from the Miracleberry shrub (Synsepalum dulcificum) are available. These berries are ideal candidates for evaluating a transformative robotic system designed to help humans, because the berries of these species are positioned from top to bottom of the plant, mostly located in the range of 2 to 4 feet in height, which is an uncomfortable height for human harvesting. The arm must be able to reach in and around the plant to access all ripe berries. Different species also have different harvest requirements and mechanics. For example, with elderberry, the entire cluster of berries is harvested as one item, whereas with aronia, individual berries are plucked from the plant.

6.5. Agbot Evaluation

Agbots will be designed to augment and collaborate with human labor for agroforestry management. To this effect, a taxonomy of tasks performed by humans in berry plantations is required. The research team will document activities/tasks performed by humans on the plants in form of a video library. The motions will be ranked on level of hardness using a metric depending on joint movements, heart rate (calorie expenditure), difficulty of reach, ergonomic impact, and time for task. This annotated library will be used in the evaluation. Below are specific examples of evaluation metrics for consideration.

• Evaluation of dexterity of reach: Are the berries ready for harvest? The primary metric of success in this evaluation activity is defined by the ability of a camera at the end of the soft arm to reach a particular point on the plant in different poses. These data will provide indication on whether the berries are ripe, which is important in timing the berry harvest. The dexterity of agbots will be compared to two control treatments—off-the-shelf hard manipulators and humans.
• Evaluation of manipulability of force: Can the berries be harvested without damage? The primary metric of success in this task is the number of berries that were harvested without permanent or visible damage to the berry. We test the hypothesis that continuum arms on agbots can harvest more berries per plant without damaging the plant or the berries. In this case, our control is the human harvester from a position that is reachable to the robot; we want to harvest at least as many berries as the human, within a reasonable amount of time.
• Evaluation of strength of grasp: Can the agbot remove weeds? The primary metric of success here is the aggregated length of the roots times the hardness of the soil (between 0–1) versus the number of robots required to pull out the weed. The ability of soft armed agbots to remove wees will be compared with humans’ ability without any tools. Multiple robots could potentially collaborate to pull (or break) weeds that would have required tools.

This project could lead to a collective and autonomous instrument for measuring spatiotemporally varying complex systems such as agriculture. It will hence also serve as a testbed for autonomy, machine learning, security, edge computing, and other technologies.

7. Agbot Deployment for Agroforestry Research

In addition to performing a maintenance task, Agbots could also play a key role in assisting data collection for agroforestry research. An integrated instrument could be developed consisting of multiple connected stationary and moving sensors with different sensing modalities to enable collective measurement of plant response, soil environment, and the surrounding microclimate (Figure 11). The distributed instrument would autonomously reconfigure itself by controlling the trajectories of the robots and optimizing distributed sensor placement to ensure high-quality and consistent data. We refer to this conceptual instrument here as CROPS: Collective RObotic Observation and Phenotyping System. It is integrative, since removing some of the sensing modalities will lead to an inability to make statistically meaningful inferences in the complex field environments. At the same time, CROPS is scalable and field adaptive, able to autonomously meet the tradeoff between the area covered vs. the frequency of full-field collections. In addition, CROPS’ capability to autonomously reconfigure itself will make it transportable between fields and experiments quickly. An integrated, scalable, reconfigurable instrument for collectively monitoring variables in complex field environment such as productive agroforestry is currently not available from any vendor. CROPS would provide the necessary new capability of auto-reconfiguration in the field to adapt the instrument to the objectives of the experiment. This capability cannot be created by just connecting a number of sensors together.

To build this type of new technology, expertise is needed in robotics, sensors, algorithms, wireless architecture, and design of field experiments. Many of the necessary pieces exist. Robots have been successfully used for under-canopy monitoring, but new robots are needed to take soil and plant tissue samples. Farm-based sensors have been deployed in field experiments, but the data collection is still manual. Machine learning algorithms have been developed using a large network of sensors, but not in an agricultural setting. The proposed development effort will enable our team to integrate these different technologies and design new systems for enabling CROPS.

Agroforestry field experiments require collective measurement of spatiotemporally varying quantities. Furthermore, since many agroforestry sites are dispersed across the region, CROPS will need to be able to be deployed on different field sites quickly. There is a significant risk in not being able to manually reconfigure the system quickly and accurately due to the excessive amount of labor required and uncertainty in data. This risk can be mitigated by developing an autonomous sensor network reconfiguration algorithm that automatically adapt the paths of the robots and recommend ground sensor locations. Such “closing-the-loop” coupled with reinforcement from the users, will lead to a robust, scalable, reconfigurable, and continuously improving with data field experimentation system.

CROPS could enable multiple research activities, impacting various fields of research for agroforestry. Some potential applications include:

• Genetics and tree crop breeding, where researchers pursue in-field high-intensity measurements for phenotyping and identifying genes that control specific traits.
• Agricultural production systems, where researchers are pursuing sustainable new agricultural paradigms such as Polycultures, or creating quantified guidelines for input management in precision agriculture.
• Cyber Physical Systems (CPS) are distributed systems that close the loop between sensing and planning. CPS researchers can evaluate their algorithms on CROPS and utilize CROPS as a testbed in new CPS and robotics proposals.
• Sustainability, Environment, and Agricultural Economics, where researchers are measuring the impact of different treatments, practices, and quantifying interactions of plant–soil–microbes–environment through collective measurements.

As an integrated sensor system, CROPS would be capable of reconfiguring itself to adapt the positioning of sensors and robots to optimally serve the objectives of the experiment and the conditions in the farm in an autonomous manner. Plant phenotyping could include tree/shrub stem counts using side facing RGB cameras, plant height using vertically oriented LiDAR, plant width using RGB and side-facing LiDAR, and Leaf-Area index using vertically facing fish-eye cameras. A machine learning pipeline that could correlate visual data with data from other sensors would be developed. Over-the-canopy multispectral measurements could be performed using drones, and under-canopy hyperspectral measurements with artificial light will be utilized to estimate transpiration and compared with gas sensors. This type of technology would allow for the evaluation and comparison of different crops or genotypes based on a variety of performance metrics for sustainable production, including carbon sequestration, water-use efficiency, and plant adaptation.

Sub-surface properties and above-ground microclimate could also be supported by CROPS. Data from connected soil sensors spread across the field and from robots with electromagnetic conductance devices could be utilized to create a complete picture of soil moisture using Bayesian inference. TerraSentia robots could be modified to extract soil samples using drills to get subsurface soil samples (up to 90 cm deep). Soil moisture measurements would dictate where and when robots could dig. For microclimate and under-canopy gas sensing, LICOR CO₂ gas-flux sensors could be deployed and augmented by low-cost mobile CO₂ sensors on the robots to infer local gas flux. Stationary temperature, moisture, and light probes at different heights would enable sensing of changing microclimate conditions.

Collectively measuring different aspects of the plant, soil, and the surrounding microclimate is the key challenge facing scientists working to improve the yield and sustainability of productive agroforestry systems. Field experiments are necessary to advance both plant breeding and agricultural production systems research because it is well known that plants grow very differently inside greenhouses. However, scientists must account for the variability in the field environment to ensure that the results from their field experiments are statistically significant. The inability to collectively measure data that can capture this variability has led to the so-called phenotyping bottleneck: the inability to accurately understand the relationship between crop performance and crop genetics as a function of the plant’s local environment. This critical gap is significantly hampering agroforestry productivity by limiting yield potential, delaying detection of stressors, and precluding accurate predictions. The development of a CROPS instrument would directly address this critical gap.

8. Conclusions

A major barrier to the adoption of agroforestry and sustainable agriculture is the shortage of labor needed to conduct intricate and skilled tasks for maintaining and researching the systems. The labor situation is further stressed with increasing minimum wages and restrictions on immigration. Small robot teams are a promising technology for these complex systems, with future developments leading to fundamental theoretical advances directly impacting soft, compliant, and continuum robots. Furthermore, integrated instruments designed to collect data on the woody plants and their environment could be deployed by robotic networks. These technological advancements have the potential to make agroforestry more feasible for individual growers, while also enabling practicing engineers in the industry to get involved.

There are several open questions on control and sensing for soft robots. Significant advances in closed-loop control of these continuum systems are needed before the technology is ready for wide-scale adoption. When the control and sensing problems are solved, however, the inherent low-cost of these types of robots should aid in wide adoption. At the same time, speed of operation and precision are elements that would need optimization. If autonomous low-cost robotic systems can be designed to work around the clock, the questions about speed may not be relevant and ubiquitous adoption may be possible.

The involvement of engineers and computer scientists in the practical applications of agroforestry could further support the fusion of these fields to engaged a new generation of scientists working to improve the sustainability and resilience of our food system. Learning opportunities for graduate and undergraduate students can include projects that connect computer-based modeling and robotic engineering with real-world applications that help feed a growing population. Even before the post-secondary level, younger students could be engaged through robotic gaming demonstrations, simulations, and other activities. The development of a multi-disciplinary approach that fully engages a new generation of scientists could contribute to innovative solutions to the problems that most threaten our future.