**1. Introduction**

To date, a variety of robots have been used in automated production lines for object manipulation tasks in factories, and these robotic technologies have contributed significantly to the development of modern industry. Recently, with the fourth industrial revolution, robots have been introduced to provide various services not only in factories but also in human living environments. Robots used in factories perform simple repetitive operations such as pick-and-place using specially designed grippers; however, in an environment such as cafes and restaurants, human-level object manipulation ability is required. Therefore, to provide a wider range of services in a human living environment, robots must have a higher level of object manipulation similar to that of humans.

In addition to manipulating objects using grasping, humans can manipulate objects freely by appropriately utilizing nonprehensile manipulations [1] without grasping, such as throwing, batting, rolling, pushing, and sliding. Most robots are still limited to prehensile manipulation using grasping. A robot that performs nonprehensile manipulation is rare owing to the difficulty of control using nonprehensile manipulations. When nonprehensile manipulation is performed, the object moves during manipulation. Since the moving object is not fixed to the robot, it is necessary to plan and control the future behavior

**Citation:** Joe, H.-M.; Lee, J.; Oh, J.-H. Dynamic Nonprehensile Manipulation of a Moving Object Using a Batting Primitive. *Appl. Sci.* **2021**, *11*, 3920. https://doi.org/ 10.3390/app11093920

Academic Editor: Luis Gracia

Received: 18 January 2021 Accepted: 20 April 2021 Published: 26 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

of the moving object based on the state of the robot and the moving object. Despite these difficulties, nonprehensile manipulation has advantages such as increased object manipulation methods, expansion of the robot workspace, and increased manipulation dexterity [2]. To improve the ability of robots to manipulate objects, the robots must have nonprehensile manipulation capabilities with these advantages.

Fabio et al. conducted a survey on nonprehensile manipulation of robots to address the trends and open issues of nonprehensile manipulation studies [3]. According to [3], the nonprehensile manipulation task is complex and difficult. Because of the complexity and difficulty of the task, most studies divide the nonprehensile manipulation task into simpler subtasks called nonprehensile manipulation primitives. The representative nonprehensile manipulation primitives include throwing [4], catching [5], batting [6], pushing [7], sliding [8], and rolling [9]. Each of these nonprehensile manipulation primitives is selectively operated by a high-level supervisor depending on the task [10]. Among the nonprehensile manipulation primitives, batting is challenging because it requires precise and fast manipulation of the moving objects, and studies on this topic are lacking. In sporting events such as table tennis, tennis, and baseball, players have a batting ability to send the moving objects (balls) to the desired position with sophisticated and quick manipulation, while the batting performance of robots is insufficient. Therefore, this study aims to contribute to the improvement of the dexterity of robots by conducting research on batting, which requires more precise and quicker nonprehensile manipulation of moving objects than do the above-mentioned primitives.

For a robot to perform a batting task, at least four methods are required. First, image processing is required to recognize a moving ball. Second, estimation of the future trajectory of the ball is required. Third, the motion of the robot arm must be controlled to affect the ball direction. Lastly, calibration is required to convert the coordinates between the robot arm's coordinate system and the vision sensor's coordinate system. This implies that the performance of each method affects the performance of the batting primitive; studies that can improve the performance of each method are discussed.

Chen et al. [11] developed a vision module for humanoid robotic table tennis. The vision module contains two stereo vision sensors with a 200 fps and an algorithm for predicting the rebound trajectory of a table tennis ball. Nakabo et al. [12] developed a highspeed vision system capable of image acquisition and image processing at 1 kHz for moving ball recognition. A parallel computation architecture was used to reduce image transfer and processing time, and an active vision system with moving cameras was developed to track the moving objects. Tesheng et al. [13] used an aerodynamic model of a ball to account for the trajectory of the ball before and after collision to improve the performance of ball direction control. Serra et al. conducted the study of hitting a table tennis ball to the desired position [14]. To accurately control the arrival position of the ball after hitting, a more accurate aerodynamic model that that in [13] was applied for the trajectory estimation of the ball. Although the algorithms were tested via simulations, the implementation of the algorithms on an actual hardware system was left to be covered under future work. For accurate ball direction control, a batting algorithm considering impact dynamics was proposed [15,16]; however, the problem of extending the 2D algorithm to the 3D algorithm remains due to the computation time required for impact dynamics in 3D.

Schüthe et al. [17] introduced the optimal control with state and soft constraints for a simulated ball batting task. By utilizing the soft constraints, a motion utilizing the redundant degree of freedom (DOF) is automatically generated, but there is a limitation in that a motion exceeding the range of motion of the joint is generated. Pekarovskiy et al. [18] proposed a motion generation method that can adapt to rapidly changing target points in consideration of the feasibility and computation time of the motion trajectory. This method was applied to 2D planar volleyball batting. Kober et al. [19] proposed a method to generate the trajectory of the robot arm through learning. If the system is changed, the process of collecting and processing data is required again, and the ball direction control is not considered. Mori et al. [20] developed a fast and lightweight robotic arm for badminton. The use of pneumatic actuators made the robotic arm lighter, enabling high-speed batting. The position of the shuttlecock was measured using a high-speed motion capture system of 240 Hz, but there was a limitation in that control of the direction of the ball was generated by a pre-learned feedforward motion. In the realization of a three-dimensional ball direction control system, the amount of computation in object recognition and trajectory estimation and the realizable three-dimensional motion generation are important.

Senoo et al. [21] developed a high-speed robot batting algorithm using a high-speed active vision system developed by Nakabo et al. [12]. The algorithm was extended to control the direction of the baseball in three-dimensional space [22]. For fast batting and ball direction control, the authors proposed a hybrid trajectory generator comprising a part that generates a high-speed batting motion and a part that modifies the motion through visual feedback. A high-speed image processing system (1 kHz) specially developed for object recognition was used, and a simple least square method was applied to estimate the object trajectory because the sampling rate was fast. Without this specially designed sensor system, it is difficult for other researchers to implement a batting system using this algorithm.

In this study, we propose a batting framework capable of controlling the three dimensional direction of a moving ball by using an off-the-shelf vision sensor with a relatively low fps (50 Hz). The proposed batting framework includes object recognition, object trajectory estimation, robot arm motion control, and calibration. In a condition where the image sampling rate is low, since the importance of one data point is relatively large, the performance of algorithms for object recognition, trajectory estimation, and motion control becomes more important. Therefore, we propose ways to improve the performance of each algorithm at a low sampling rate. Since the proposed methods were developed based on an off-the-shelf vision sensor, other researchers can easily implement these algorithms.

The specific contributions of this study are as follows. First, in terms of ball recognition, this study proposes an image processing method for improving ball recognition accuracy in a more general environment. Previous studies covered noise filtering after applying a difference image or color segmentation. In the real world, however, various lights affect the ball recognition accuracy. Further, we applied the method using the geometrical properties of the ball to improve the recognition accuracy under these lighting conditions. Second, by applying weighted squares regression considering the positional accuracy according to the distance of the stereo vision sensor, we improve the trajectory estimation accuracy of the target object even with a low sampling rate. Third, we analyze the factors that affect the performance of ball-direction control and propose additional constant posture control methods to reduce the influence of those factors. Finally, in actual implementation, calibration between the camera and the robot coordinate system is essential. A simple but accurate calibration method is introduced.
