Learning by Demonstration of a Robot Using One-Shot Learning and Cross-Validation Regression with Z-Score

Abstract

We introduce a One-Shot Learning system where a robot effectively learns how to manipulate objects by relying solely on the object’s name, a single image, and a visual example of a person picking it up. Once the robot has mastered picking up a new object, an audio command is all that is needed to prompt it to perform the action. Our approach heavily depends on synthetic data generation, which is crucial for training various detection and regression models. Additionally, we introduce a novel combined regression model called Cross-Validation Regression with Z-Score (CVR-ZS), which improves the robot’s grasp accuracy. The system also features a classifier that uses a cutting-edge text-encoding technique, allowing for flexible user prompts for object retrieval. The complete system includes a text encoder and classifier, an object detector, and the CVR-ZS regressor. This setup has been validated with a Niryo Ned robot.

1. Introduction

In collaborative robotics, replicating human-like object manipulation presents a significant challenge, especially when relying on just a single observation of human handling. The difficulty is amplified by the diversity in object shapes, sizes, and materials, requiring robots to exhibit a level of adaptability and dexterity comparable to that of humans.

Traditional robot programming methods often involve extensive manual coding and task-specific programming, which are cumbersome for applications needing quick adaptation and deployment. While manually guiding the robot’s claw to the correct grasping position is a possible approach, it becomes impractical when objects can appear in various positions. Learning by imitation offers a promising alternative, where robots learn by observing and replicating human or expert demonstrations. This method can accelerate skill acquisition but challenges remain due to the requirement for large datasets for each new task.

Few-Shot Learning (FSL) is a machine learning approach aimed at identifying or classifying new instances using only a few examples. In the most restrictive cases, such as One-Shot Learning (OSL), only one sample per category is available [1]. This is particularly valuable in scenarios where collecting data is challenging or costly, making it impractical to compile large datasets. OSL minimizes the need for extensive datasets by focusing on learning efficient representations, using prior knowledge, employing metric-based methods, and applying data augmentation and regularization techniques to generalize from just a few samples. The integration of One-Shot Learning (OSL) addresses the limitations associated with traditional demonstration learning, thereby minimizing the reliance on extensive datasets. In the application of OSL, the user is only required to provide the object’s name and a sample demonstrating how it is manipulated. The robot, leveraging this singular example, learns to handle the object in a human-like manner. To initiate the pick-up action, a plain text prompt suffices, such as “robot, please pick up the screwdriver”. In our case, we consider that each sample is formed by the image of the object and its name since it is necessary to have both visual and textual information. Using only the image would require another prior model of object classification in order to obtain its textual information.

To transform our system into an OSL framework, we introduce an integrated approach wherein synthetic data are generated for each image depicting how a person grasps an object. These synthetic data are then used to train a cascade model, incorporating a classical YOLO v8 (You Only Look Once) object detector [2] and an innovative combined regression model named CVR-ZS (Cross-Validation Regression with Z-score). Unlike a single regression model, this method achieves more accurate localization of the object pick-up position. Additionally, our approach is seamlessly integrated with a CLIP (Contrastive Language–Image Pre-training)-based text encoder [3], enabling the identification of user prompts that specify the object the user desires the robot to pick up. The efficacy of our method is substantiated through validation on a cost-effective robot platform.

The main contribution of our method is the presentation of the new CVR-ZS-based regression system and how it has been applied for object grasping under an OSL paradigm by augmenting data learned by human demonstration.

In Section 2, we review the main contributions to imitation learning. In Section 3, we present the developed method, explaining the data augmentation process, the training of the object detection model, and the classifier integrated with CLIP, as well as the operation of CVR-ZS. In Section 4, we discuss the most significant experiments and results obtained. Finally, the conclusions of our study are presented in Section 5.

2. Overview of Related Work

In the field of collaborative robotics, a significant amount of research has focused on advancing understanding of the environment and object manipulation, trying to emulate human dexterity. Studies have highlighted the use of advanced perception techniques, such as computer vision, to enable robots to acquire a detailed understanding of their environment [4]. In addition, work has been conducted on developing algorithms and systems that mimic the human ability to grasp and manipulate various objects, addressing challenges such as adaptability to variable shapes [5,6]. Demonstration learning [7,8] and reinforced learning [9,10] have also been applied to enable more natural interaction. Taken together [11], these advances bring robots closer to the ability to understand the world and manipulate objects in efficient and versatile ways, mirroring human abilities.

One approach for robotics involves reinforcement learning (RL), where the robot autonomously develops control policies through iterative experimentation. Lobbezoo et al. [12] combine traditional and reinforcement learning control for simulated and real environments to validate the RL approach for standard industrial tasks such as reach, grasp and pick-and-place. They want to bring intelligence to robotic control so that robotic operations can be completed without precisely defining the environment, constraints, and the action plan. In contrast to conventional methods in which robots choose a grasping point and execute the action, Kalashnikov et al.’s method [13] uses a closed-loop vision-based control approach. In this approach, the robot continuously updates its grasping strategy based on the most recent observations, thus optimizing grasping success over the long term. The framework used is called QT-Opt, and is a reinforcement learning method based on self-supervised vision. It uses a large number of real-world grasping attempts to train a deep neural network with more than 1.2 million parameters. This approach has reported 96% success in performing closed-loop real-world captures, even on unseen objects. In addition to achieving a high success rate, the method exhibits advanced behaviors. Using only RGB vision-based perceptual information from an over-the-shoulder camera, the system automatically learns regrasp strategies, probes objects to find the most effective regrasps, learns to reposition objects, and performs other inapprehensible manipulations prior to regrasping. It is also able to respond dynamically to perturbations and disturbances in its environment.

To solve the problem of where to grasp objects, Mahler et al. [14] utilize a synthetic dataset of 6.7 million point clouds, grasps, and analytic grasp metrics to train a model predicting the success probability of grasps from depth images. The model is trained using 3D models from [15] placed in randomized poses on a table. Their framework has been extended to vacuum suction grippers [16] and dual-arm robots [17]. Along this line, the authors of [18] create a dataset which focuses on real-world manipulable objects for robotics, providing category-level pose estimation and affordance prediction annotations. Annotation is streamlined, requiring only a single off-the-shelf camera and semi-automated processing, yielding high-quality 3D annotations without crowdsourcing. A practical multi-stage grasp detection method for a Kinova robot in stacked environments [19] uses a multi-stage network for multi-object grasp detection. The algorithm extracts an initial region of interest and continuously refines it to achieve accurate object detection and grasping results, outperforming existing algorithms on the VMRD dataset [20].

Within the learning by demonstration in robotics, some works have focused on the recognition of gestures using human skeletal coordinates and the use of neural networks and Markov models [21]. Other research explores the problem of learning by human demonstrations in inconsistent contexts [22], presenting an imitation learning framework for robots to reproduce behavior observed in human demonstrations with variations in viewpoints, operators, backgrounds, and positions. Another approach involves robot eye–hand coordination learning [23], where a robot learns a task function from human videos and uses it in real-time to guide its motions in the execution phase.

Jin et al.’s research [24] focuses on a vision-based learning system for robotic arms, aims to detect human actions in real-time, and proposes an inductive object trajectory method for motion planning to replicate tasks demonstrated by a human user.

Other researchers combine learning from human demonstration with reinforcement learning. San et al. [25] propose a method in which the robot continuously interacts with the environment to master the skill. Firstly, learning from demonstrations initializes the control policies, and then the robot begins to practice the demonstrated skill, receiving a reward from the environment after each practice round. Kamali et al. [26] use a virtual reality system to control the robot tool position and orientation with hand motions, while monitoring its movements in a 3D virtual reality environment, to give reference trajectories to a deep neural policy network for controlling the robot’s joint movements. Then, they leverage the Proximal Policy Optimization algorithm for deep reinforcement learning to train the policy network with the robot joint values and the reference trajectory observed at each iteration. Cabi et al. [27] control policies for a diverse set of manipulation tasks by generating observation–action pairs through teleoperation, scripted policies, or trained agents. Human preferences are then introduced to construct a reward function for a new task, which is used to annotate historical data for different tasks. This annotated dataset is employed for batch reinforcement learning to train manipulation policies from visual input.

The advances achieved in our work not only effectively address the inherent challenge of reinforcement learning, which often requires numerous simulations to achieve satisfactory results but also present significant advantages in object manipulation. Our methodology allows learning to manipulate objects without relying on specific three-dimensional representations of objects using point clouds, which facilitates generalization to a wide variety of objects in an efficient manner.

The independence of teleoperation is another distinctive aspect of our research. The ease in data collection is highlighted, as our approach is based on the generation of synthetic data. This translates into a One-Shot Learning system, where the ability to acquire knowledge with a minimum amount of data is a key characteristic. This innovation not only simplifies the data acquisition process but also paves the way for faster and more effective learning.

Finally, although ensemble models have been used to improve the results of regression problems [28,29,30], our method uses a novel method that integrates several regressors under the CVR-ZS paradigm.

3. Analysis of the System

Figure 1 illustrates the schematic representation of the proposed system. By employing an automatic speech recognition (ASR) system based on a pre-trained OpenAI Whisper model [31], the text order (“take the screwdriver” in the figure example) is extracted to invoke a pre-trained CLIP model. CLIP [3] is a neural network trained on diverse (image and text) pairs, capable of understanding natural language instructions to predict the most relevant text snippet for a given image. This CLIP model outputs a 512-dimensional embedding vector, which is then fed into our custom classifier model. The classifier’s objective is to convert the CLIP output embedding vector into a specific class, with the number of classes corresponding to the objects we aim to recognize. To train the classifier, we have generated multiple prompts for each object, such as “take the pen”, “grab the pen”, “pen”, and so on.

Simultaneously, we employ a YOLO model trained with synthetic data generated by our custom generator for recognizing the desired objects. The output class from YOLO aligns with the output class of our CLIP classification model. This approach enables us to isolate the target object and more effectively pinpoint the grasping points. Not utilizing YOLO at this stage results in inferior detection outcomes for grasping points compared to using a single regression model because the image is much larger and there are many small, scattered objects, which confound the model. After identifying the object window, we invoke a specific regression model for grasping point detection, with a distinct model for each added object. The training of these regression models involves learning grasping points from human demonstrations and augmenting the data using techniques discussed later.

The regression model returns the grip points that can be used directly to obtain the robot’s grip points, as well as the orientation from a previous calibration of the robot. One of the highlights of our research is the use of the CVR-ZS method, which significantly improves the regression problem to obtain the grip position.

3.1. Synthetic Data Generation

Figure 2 shows what the synthetic data generation process looks like. From a single image of what an object looks like and how it is grasped, we extract the grasping coordinates using the Mediapipe hand model [32]. We choose the coordinates of the index finger and thumb since they are the ones that directly touch the object. The data generator randomly chooses from all registered objects and places them at positions in the robot’s workspace. We use a subtraction model to subtract the object image from the background and apply scale, rotation, flip, and translation transformations. In our study, we focus on nine common objects, generating a dataset of 30,000 images through data augmentation.

The subtraction model incorporates the Structural Similarity Index [33]. Unlike the mean squared error (MSE), which may not accurately reflect perceived similarity, the structural similarity metric considers texture, enhancing the assessment of image similarity. From the similarity index, we apply a common Otsu threshold [34] and extract the segmentation mask (although, of course, any other segmentation technique can be used).

The data augmentation method randomly chooses different objects to be positioned on the workspace. For each selected object, we perform various random transformations on the isolated object image, including rotations, translations, reduced scaling, or flips both horizontally and vertically. The transformations are also performed on the grip points, calculating the points in their new position. The selection of a random workspace position takes into account the object’s mask within the workspace and ensures it does not overlap with other objects.

In addition to the grip coordinates, we create a file in YOLO format where we record the bounding box of each object. This allows us to train the detection model prior to regression. For the nine objects used, we generate 30,000 training images and 3000 evaluation images.

3.2. Object Detection with Text Prompts

Utilizing a CLIP pre-trained model, we transform a text prompt into a 512-element encoded vector. This output undergoes processing through our custom classification network, featuring two Fully Connected (FC) layers: 512 to 256 and 256 to n outputs, where n corresponds to the number of objects.

We generate a random set of hundreds of sentences by integrating the name of the object given by the user when adding a new one. So, for example, for screwdriver, we train our model with sentences like “Take the screwdriver”, “Please take the screwdriver”, etc. All sentences related to an object are classified with that object, e.g., class 0.

From the synthetic data, which include recorded bounding box coordinates for each object, we train a YOLO v8 model [2]. The output from our classification model, integrated with CLIP, categorizes text prompts into n distinct objects, aligning with the number of objects detected by YOLO. When the confidence of the CLIP classification is over a threshold, we select the object returned by YOLO for that class. Additionally, for YOLO, we establish another threshold for confidence. If the confidence falls below this threshold, we consider the object to not be present in the workspace.

3.3. Cross-Validation Regression Integrated with Z-Score

Once the objects are isolated using YOLO, we use a regression model that starts from the object image and returns four values: center point of object grasp ($X,Y$), modulus of distance between the two grasp points (M), and vector angle (A). Instead of directly using the coordinates of the two grasping points, our regression model works with the central grasping point, the distance modulus of the two grasping points and the angle of the vector joining them. Based on our experiments, this conversion allows us to improve the robot’s grip by avoiding problems when the regression returns a slightly displaced grip point.

We train a regression model for each type of object, which allows for improved grasping results. However, in some cases, the output may not be accurate due to erroneous predictions of the model itself due to the existence of less known situations, such as objects in proximity. To improve accuracy, we propose to use an ensemble model integrated under a novel technique that we present, which we call CVR-ZS (Cross-Validation Regression integrated with Z-score). This technique allows to improve the regression performance of diverse types of models and is based on the training of several models of an analogous nature with disjoint validation sets (see Figure 3). Based on the Cross-Validation Voting method (CVV) used for a classification problem [35], we spread the training set over k datasets and train k models. CVV was successfully used in its classification-oriented CVV-CP version using Siamese networks in the One-Shot Learning problem [36]. Unlike CVV and CVV-CP, which are focused on classification problems, in regression, we could choose to obtain the mean value of the regressors, which in itself improves the result of the individual regressor, or to perform an extraction operation on the outputs prone to be outliers.

The data are previously randomized, and k different validation slots are selected. The rest of the data from each slot are used for training. Let $\tau$ be the set that includes all the samples of the complete training dataset. Let $T_i$ and $V_i$ be the training and validation sets corresponding to the slot i. These sets must verify Equations (1)–(6):

$$\tau =\underset{i=1}{\overset{k}{\uplus }}{T}_i$$

(1)

$$\bigcap _{i=1}^k{T}_i=\varnothing$$

(2)

$$\tau =\bigcup _{i=1}^k{V}_i$$

(3)

$$\bigcap _{i=1}^k{V}_i=\varnothing$$

(4)

$$\left[\tau =T_i\cup V_i\right]\forall i\in k$$

(5)

$$\left[T_i\cap V_i=\varnothing \right]\forall i\in k$$

(6)

Then, k models of the same nature are trained with each of the training and validation slots. The use of an ensemble model allows us to obtain a more robust an accurate model. In addition, integrating it with Z-score allows us to discard outliers in the output of each test. During training, k models are trained separately. During inference, Z-scores are computed for each column of data, X, Y, M, and A, as defined in Equation (7).

$$\left\{\begin{array}{c}{X}_{{ZS}_i}=\left[X_i-{\mu }_X\right]/{\sigma }_X\hfill \\ Y_{{ZS}_i}=\left[Y_i-{\mu }_Y\right]/{\sigma }_Y\hfill \\ M_{{ZS}_i}=\left[M_i-{\mu }_M\right]/{\sigma }_M\hfill \\ A_{{ZS}_i}=\left[A_i-{\mu }_A\right]/{\sigma }_A\hfill \end{array}\right.$$

(7)

For each model, we compute the quadratic value of its Z-score values as depicted in Equation (8). This calculation enables us to filter out outliers and values that closely align with the consensus among most models:

$$z_i=X_i^2+Y_i^2+M_i^2+A_i^2$$

(8)

In CVR-ZS, only the entire half plus one (s) of the models involved in CVV (k) are selected to calculate the regression based on the lowest Z-score (see Equation (9)). This allows most of the outliers to be rejected. Averaging the remaining s models, the test results are significantly improved:

$$s=⌊{\displaystyle \frac{k+1}{2}}⌋$$

(9)

For a $k=5$, the $s=3$ models with the lowest calculated $z_i$ are used. The output of the ensemble model during inference is the mean value for each value for the selected models as seen in Equation (10):

$$\left\{\begin{array}{c}X={\displaystyle \frac{{\sum }_{i\in selected\_models}{X}_i}{len(selected\_models)}}\hfill \\ Y={\displaystyle \frac{{\sum }_{i\in selected\_models}{Y}_i}{len(selected\_models)}}\hfill \\ M={\displaystyle \frac{{\sum }_{i\in selected\_models}{M}_i}{len(selected\_models)}}\hfill \\ A={\displaystyle \frac{{\sum }_{i\in selected\_models}{A}_i}{len(selected\_models)}}\hfill \end{array}\right.$$

(10)

There is one main reason for the selection of five estimators. In [35], a comparison of the CVV method for classification using different number of estimators is carried out. It is concluded that the accuracy is improved up to five estimators but could stabilize and even worsen after that value. In our experiments, we also observe such behavior in the preliminary results. Furthermore, operating with a larger number of estimators would make the method computationally very expensive.

We use a ResNet-34 convolution backbone [37] for each regression model, pre-trained with ImageNet. The classification layers are replaced by a ReLu activation, an average pooling, and a normalization. Finally, a dense layer connects the 512 outputs of the normalization with the four outputs of the model.

An advantage of this ResNet-34 backbone is that it is lightweight and allows us to reduce the computational cost of training. Although other models could improve the regression accuracy, this model is sufficient for the robot to grasp the objects correctly. In any case, the CVR-ZS approach can also be used with backbones of a different nature, such as ResNeXt [38], EfficientNet [39], RegNet [40], or ConvNeXt [41]. Each classifier of a different nature is trained with the similar slots used with the others, taking advantage of the goodness that each model offers against the same data partitioning. Finally, all regressors are combined as detailed earlier.

4. Experiments and Results

The experiments were conducted using a Niryo Ned robot [42], which is sufficiently small and manageable to be used in a human environment. It is 3D printed and has 6 DoFs. An experimental scenario was chosen, where we marked the robot workspace and deployed a tripod with RGB camera. The robot was calibrated using the method of Zhang et al. [43]. The robot was connected to a computer with RTX3090 GPU for faster operation and training.

At first, data acquisition was performed, consisting of the name of each object, an image of the isolated object on the workspace, and an image of how a person picks up the object. For nine evaluated objects (nut, pen, rubber, screw, screwdriver, spoon, glue stick, usb stick, and lip corrector), we applied our data augmentation method to generate 30,000 training images, where we randomly mixed different objects. In addition, we generated 3000 test images. For each of the images, we generated the grasping coordinates of each object from the person’s thumb and index finger, as well as the bounding boxes of the objects.

Next, we performed the training of the models. We trained a YOLO v8 model, a classifier based on the CLIP text encodings for different sentences generated from the object names, and finally five regression models per object using CVR-ZS. The whole synthetic data generation process required 24 h to be completed.

For experimentation, we evaluated the results of CVR-ZS with respect to the test set, as well as the results of object collection by the real robot. It should be noted that, in this last case, the results are visually quantified.

Regarding the object detection, the results obtained with new synthetic data when evaluating the metrics of YOLOv8 are satisfactory, consolidating the efficiency and robustness of this architecture in object detection (see Figure 4). The loss plots charted a path of steady improvement, evidencing the model’s ability to adjust to intricate image details and learn meaningful representations of the objects in question.

Regarding the CVR-ZS method, each of the regressors is based on a ResNet-34 backbone connected to an FC regression layer that links the 512 features to four outputs (coordinates of the central gripping point, modulus, and direction). We used the L1 loss, which takes the mean element-wise absolute value difference between the actual and the predicted value. For each object, we trained five regressors by partitioning the training dataset as explained above (five disjoint validation sets). Each of the models was trained with fine-tuning and an Adam optimizer, where we reduced the learning factor from 0.006 to 0.001 and then to 0.0001. Each fine-tuning block was trained for 100 epochs using a patience of 20 epochs to cut the training if the validation was not reduced.

The integration of the models was performed using Z-scores. In

Table 1. Regressor comparison (L1 loss).

Model	L1 Loss
Normal Regressor without CVR-ZS (average 5 runs)	0.0578
CVR with output as average of k = 5 models	0.0505
CVR with K-Means clustering	0.0490
CVR-ZS	0.0482

, we show the results comparing the average result of a model trained without CVR-ZS (with five runs), a model where we simply average the output values of the five regressors trained with CVR, another model where we apply clustering to the output of the models by distributing in two classes and selecting the class with the largest number of samples and another model with CVR-ZS.

In the case of clustering, a grid search was carried out with different methods and parameters, including methods such as classic K-Means, Feature Agglomeration, Bisecting K-Means, MiniBatch K-Means, Spectral Clustering or DBSCAN. The best results were obtained with classic K-Means and specifying two output classes. The clustering algorithm partitions the outputs of the k regressors into two classes. The main class is the one with the largest number of samples, and its centroid is the regression output. Using clustering, we sought to isolate the regressors with a result farther away from the centroid of the main class.

We obtained the errors for each object as shown graphically in Figure 5 and

Table 2. Regression per object (L1 loss).

Object	L1 Loss (without CVR-ZS)	L1 Loss (with CVR-ZS)
nut	0.0788	0.0694
pen	0.0476	0.0370
rubber	0.0700	0.0611
screw	0.0592	0.0522
screwdriver	0.0655	0.0372
spoon	0.0397	0.0328
glue stick	0.0523	0.0473
usb stick	0.0517	0.0475
lip corrector	0.0552	0.0499

. The results show that the model improves the regression of all objects. Even objects like the nut, characterized by symmetry and challenges in determining the correct grip angle, show a reduction in error.

Finally, we evaluated the success of this method with the Niryo Ned robot. We performed 20 pick and place movements for each object.

Table 3. Pick-and-place success rate.

Objects	Success
Nut	0.90
Pen	0.90
Rubber	0.70
Screw	0.85
Screwdriver	0.90
Spoon	0.75
Glue stick	0.80
USB stick	0.65
Lip corrector	0.80

shows the results of this experiment, which are mostly satisfactory. However, there are some objects that seem more difficult to pick up by the robot.

Despite the positive achievements, some objects posed notable difficulties due to the limitations of the gripper and the precision of the robot’s hand–eye calibration performed. One prominent cause of error identified was the narrowness of the gripper. Some objects, particularly the USB stick and the rubber, posed difficulties, as they fit tightly within the gripper. This limitation led to the need for a higher level of accuracy during the pick-up operations. Another crucial aspect contributing to the challenges was the hand–eye calibration of the robot. The complexity in the hand–eye calibration of the robot was accentuated by the position and angle of the external camera, introducing a degree of inaccuracy in commanding the robot to an accurate image position. Consequently, this issue led to occasional misplacements and hindered the overall success rate of the picking-up motions. The optimal location of the external camera would be one perpendicular to the working plane of the robot, which would correct possible errors due to calibration and even reduce the possibility of occlusions when taking the photo with which the network is trained, but this was not contemplated in our setup due to the unavailability of a tripod of these characteristics. This makes the results obtained even more satisfactory.

The setup used to this experiment is illustrated in Figure 6 and Figure 7, which show the results of instructing the robot to take a pen and a spoon, respectively. More details about the setup and results can be found in the Supplementary Materials Video S1.

5. Conclusions

Imitation learning via One-Shot Learning has emerged as one of the most compelling challenges in collaborative robotics in recent years. Our architecture tackles this challenge effectively by utilizing synthetic data generation and training a series of interconnected models. This system includes a model for classifying objects for pick up using speech-to-text and text prompts through the CLIP encoder, an object detector based on YOLO v8, and a novel combined regression model called CVR-ZS. The CVR-ZS model enhances the robot’s accuracy and holds promise for application in various domains, significantly reducing errors compared to individual regression models.

We conducted experiments with a Niryo Ned robot, which is compact and well suited for human environments. Notably, the methods we used are adaptable to different types of robots. Despite some technical limitations, such as the gripper size and minor precision issues, the gripping results were highly promising.

For future research, we propose investigating the system’s ability to learn new objects in real-time through continuous learning techniques. We also aim to extend the use of CVR-ZS to other regression problems. Real-time learning could improve our model’s adaptability when introducing new objects, potentially eliminating the need for retraining. Expanding CVR-ZS to other problem domains will help gauge the extent of improvement it can offer and broaden its applications beyond robotics.