Active Prior Tactile Knowledge Transfer for Learning Tactual Properties of New Objects

Abstract

Reusing the tactile knowledge of some previously-explored objects (prior objects) helps us to easily recognize the tactual properties of new objects. In this paper, we enable a robotic arm equipped with multi-modal artificial skin, like humans, to actively transfer the prior tactile exploratory action experiences when it learns the detailed physical properties of new objects. These experiences, or prior tactile knowledge, are built by the feature observations that the robot perceives from multiple sensory modalities, when it applies the pressing, sliding, and static contact movements on objects with different action parameters. We call our method Active Prior Tactile Knowledge Transfer (APTKT), and systematically evaluated its performance by several experiments. Results show that the robot improved the discrimination accuracy by around $10\%$ when it used only one training sample with the feature observations of prior objects. By further incorporating the predictions from the observation models of prior objects as auxiliary features, our method improved the discrimination accuracy by over $20\%$. The results also show that the proposed method is robust against transferring irrelevant prior tactile knowledge (negative knowledge transfer).

1. Introduction

1.1. Motivation

We humans perceive tactual properties of an object (e.g., stiffness, texture, temperature, weight) by applying exploratory actions (e.g., pressing, sliding, static contact, lifting) [1]. After applying different exploratory actions on an object, we can obtain its different tactile information. Conversely, making the same exploratory action on different objects produces different tactile observations. Therefore, when we learn about an object, we always link its physical properties with the exploratory actions that we apply on it.

Besides different kinds of exploratory actions, the tactile information we perceive from an object is also dependent on how we apply an action. Consider an example of pressing on two objects. Object 1 is made of soft sponge, and object 2 is made by covering a solid metal with a soft sponge surface. When pressing our fingertips on both objects with a small normal force, we can recognize similar object deformations. However, if we press with a larger normal force, object 1 deforms much more than object 2, since we have reached the metal part of object 2. A similar situation occurs when we apply the sliding movement on object surfaces with different forces and velocities. As a result, by applying different exploratory actions in different ways, we can build a detailed knowledge of the object’s tactual properties which we call tactile exploratory action experiences.

We humans learn about new objects in an active and incremental way. We actively select the most informative exploratory actions to interact with them [2,3]. More importantly, we relate these new objects with the experiences of exploring objects that we have previously encountered. By transferring the prior tactile knowledge, or prior tactile exploratory action experiences, we can largely reduce the amount of exploratory actions required to discriminate among new objects. In this way, we humans save a lot of time and energy, and recognize new objects with high accuracy [4,5,6,7,8,9,10].

Can robotic systems with a sense of touch also perform like humans to actively transfer the past tactile exploratory action experiences when learning about new objects (transfer learning)?

1.2. Background

Over the past decades, researchers have developed various tactile sensors and mounted them on robotic systems (e.g., [11,12,13,14,15,16,17]). In this way, the robots with a sense of touch can perceive different objects’ tactual properties by applying exploratory actions. For example, a robot can slide its sensory parts on objects to sense their textural properties [18,19,20,21], establish a static contact to estimate the temperature [22], or lift objects to measure their center of mass [23]. Bhattacharjee et al. [24] developed algorithms to classify objects into four categories: (1) Hard-Unmoved; (2) Hard-Moved; (3) Soft-Unmoved; and (4) Soft-Moved using One Nearest Neighbor Classifier, Hidden Markov Models and Long Short Term Memory networks based on features of time-varying tactile sensor data (maximum force, contact area, and contact motion). Furthermore, several methods have been proposed for the active object exploration problem, in which the robot actively applies multiple exploratory actions to recognize objects (e.g., [25,26,27,28,29,30,31,32]).

However, the problem of transferring the robotic tactile knowledge has been rarely investigated. Even though many transfer learning techniques have been successfully applied to several areas (e.g., Natural Language Processing: [33]; WiFi-based localization: [34]; Computer Vision: [35,36,37,38]; Bio-informatics: [39]), it is our works that introduced tactile transfer learning. Previously, Kaboli et al. [20,21] developed a novel textural descriptor. Using the descriptor, a ShadowHand dexterous robotic hand equipped with BioTac sensors on its fingertips could efficiently discriminate among object surface textures. Later, we designed a transfer learning method [40,41,42] so that the robotic hand could reuse the prior texture models from 12 objects to learn about the surface textures of 10 new objects. However, since only the sliding movement was applied, the robot could only transfer the object textural properties.

In our previous works [43,44], we proposed an active touch learning method in which an UR10 robotic arm with an artificial skin on its end-effector or fingertips could apply sliding, pressing, and static contact movements to learn about objects’ surface texture, stiffness, and thermal conductivity, respectively. Even though our active learning method enables the robot to efficiently learn about objects, the robot still needs to learn from scratch given a new set of objects. In this regard, recently, for the first time in robotics and tactile domains, we proposed an algorithm called Active Tactile Transfer Learning (ATTL) [45] to actively transfer multiple physical properties of prior objects. Using ATTL, the UR10 robotic arm could actively select prior knowledge to transfer (surface texture, stiffness, and thermal conductivity by applying sliding, pressing, and static contact movements). As a result, the robot could use fewer training samples (even one sample) to achieve higher recognition rate, when it learns about new objects.

The robotic systems in the above-mentioned works only applied exploratory actions with fixed action parameters, e.g., sliding with a fixed velocity to perceive surface textures. In order to learn their detailed physical properties (e.g., the vibro-tactile feedbacks by sliding at different speeds) so as to better discriminate among them, the robots should be able to apply exploratory actions with different action parameters.

1.3. Contribution

In this paper, we focus on actively transferring the prior tactile exploratory action experiences to learn more details about the physical properties of new objects (see Figure 1). Our contributions are two-fold:

• We enable a robot to apply exploratory actions with multiple action parameters. In this way, the robot gains more detailed tactile information.
• We propose an active tactile transfer learning algorithm so that the robot leverages the previously obtained detailed tactile knowledge (prior tactile exploratory action experiences) while learning about a new set of objects.

In the sequel, we first introduce the robotic system (Section 2). Then, we illustrate how the robot applies exploratory actions and obtains the physical properties of objects (Section 3). Afterwards, we illustrate our proposed tactile transfer learning in detail (Section 4), followed by a systematic evaluation of the method (Section 5). We finalize this paper with a conclusion and a discussion about future works (Section 6).

2. System Description

2.1. Multi-modal Artificial Skin

To enable the robot to perform more human-like behaviours with multiple tactile sensing modalities, we designed and manufactured multi-modal artificial skin (Figure 2a made by seven active tactile modules (Figure 2b [12]. Each module is a small hexagonal printed circuit board equipped with off-the-shelf sensors (one temperature sensor, one accelerometer, three normal force sensors, and one proximity sensor). In this way, robots are equipped with such an artificial skin that contains seven temperature sensors, seven accelerometers, 21 normal force sensors, and seven proximity sensors. They can emulate the human tactile sensing about temperature, vibrations, force, and light touch. Their technical information is summarized in

Table 1. Technical information of sensors in the artificial skin ([12]).

Type	Sensor	Range	Accuracy	Resolution
Proximity	VCNL4010	$200\mathrm{mm}$	N.A.	$0.25$ lx
Acceleration	BMA250	$\pm 2\mathrm{g}$	$256\mathrm{LSB}/\mathrm{g}$	$3.91$ mg
Temperature	LM71	−40–150 °C	$\pm 1.5{}^{\circ }\mathrm{C}$	$31.25{\mathrm{m}}^{\circ }\mathrm{C}$
Normal force	customized	>$10\mathrm{N}$	$0.05\mathrm{N}$	N.A.

.

2.2. UR10 Robotic Arm

We mounted the multi-modal artificial skin on the end-effector of an Universal Robotic Arm (UR10) with six DoFs (Figure 2a). The UR10 was controlled in collaboration with the aritificial skin in order to apply different exploratory actions on objects.

3. Exploratory Actions and Perception

3.1. Exploratory Actions Definition

By applying exploratory actions on objects with different action parameters, the robot can attain different feature observations. In this work, we consider three types of exploratory actions: pressing (denoted as P), sliding (denoted as S), and static contact (denoted as C). Formally, we define $N_{\mathbf{\alpha }}$ number of exploratory actions as $A={\{{\mathbf{\alpha }}_n^{{\mathbf{\theta }}_n}\}}_{n=1}^{N_{\mathbf{\alpha }}}$, where ${\mathbf{\theta }}_n$ is the action parameters that define "how" the robot can apply the exploratory action. We further define ${\mathbf{\theta }}_n\in \{{\mathbf{\theta }}_P,{\mathbf{\theta }}_S,{\mathbf{\theta }}_C\}$, where ${\mathbf{\theta }}_P,{\mathbf{\theta }}_S$, and ${\mathbf{\theta }}_C$ represent the action parameters for the pressing, sliding, and static contact movements respectively.

3.1.1. Pressing

The robotic system presses its sensory part on the object surfaces in order to perceive its stiffness (see Figure 3a). The pressing movement consists of pressing until a depth of $d_P$ and holding the artificial skin for $t_P$ seconds, i.e., ${\mathbf{\theta }}_P=[d_P,t_P]$. During the pressing, the multi-modal artificial skin can record the normal force feedbacks from each normal force sensor: ${\mathbf{F}}_{n_f,n_s}={\{F_{n_f,n_s}^m\}}_{m=1}^{t_P·f_s}$ in order to measure the object stiffness. $n_f$ is the index of a normal force sensor in one skincell ($n_f=1,\dots ,N_f$, in our case $N_f=3$), and $n_s$ is the index of skincells in the artificial skin ($n_s=1,\dots ,N_s$, in our case $N_s=7$). $f_s$ is the sampling rate of the artificial skin, and m the sampling time step. In addition to the normal force feedbacks, the robot can also record the temperature feedbacks from each temperature sensor in order to measure the object thermal conductivity: ${\mathbf{T}}_{n_t,n_s}={\{T_{n_t,n_s}^m\}}_{m=1}^{t_P·f_s},n_t=1,\dots ,N_t$, with $N_t$ being the number of temperature sensors in one skincell (in our case $N_t=1$).

3.1.2. Sliding

The robot slides the artificial skin on the object surface and perceives its textural properties [18,21] (see Figure 3b). To do this, the robot first builds a contact with objects with the normal force of $F_S$, then it linearly slides on the objects with a speed of $v_S$ for $t_S$ seconds, ${\mathbf{\theta }}_S=[F_S,v_S,t_S]$. During sliding, the robot collects the outputs of accelerometers (in three axes: $x,y,z$): ${\mathbf{a}}_{n_a,n_s}^{\left(x\right)}={\{a_{n_a,n_s}^{\left(x\right),m}\}}_{m=1}^{t_S·f_s}$, ${\mathbf{a}}_{n_a,n_s}^{\left(y\right)}={\{a_{n_a,n_s}^{\left(y\right),m}\}}_{m=1}^{t_S·f_s}$, ${\mathbf{a}}_{n_a,n_s}^{\left(z\right)}={\{a_{n_a,n_s}^{\left(z\right),m}\}}_{m=1}^{t_S·f_s}$. Then the robot combines these signals together: $\mathbf{a}={\{{\mathbf{a}}_{n_a,n_s}\}}_{n_a=1,n_s=1}^{N_a,N_s}$; ${\mathbf{a}}_{n_a,n_s}=[{\mathbf{a}}_{n_a,n_s}^{\left(x\right)},{\mathbf{a}}_{n_a,n_s}^{\left(y\right)},{\mathbf{a}}_{n_a,n_s}^{\left(z\right)}],n_a=1,\dots ,N_a$, where $N_a$ is the number of accelerometers in one skincell (in our case $N_a=1$). Besides, the change of temperature during sliding is also collected as an extra information ${\mathbf{T}}_{n_t,n_s}={\{T_{n_t,n_s}^m\}}_{m=1}^{t_S·f_s}$.

3.1.3. Static Contact

The object thermal cues can be attained by the robotic system by applying static contact movement: the robot presses its sensory part against the object surface until a depth of $d_C$ and maintains the contact for $t_C$ seconds, i.e., ${\mathbf{\theta }}_{\mathit{C}}=[d_C,t_C]$ (see Figure 3c). The normal force feedbacks and temperature feedbacks are recorded: ${\mathbf{F}}_{n_f,n_s}={\{F_{n_f,n_s}^m\}}_{m=1}^{t_C·f_s}$, ${\mathbf{T}}_{n_t,n_s}={\{T_{n_t,n_s}^m\}}_{m=1}^{t_C·f_s}$.

3.2. Object Physical Properties Perception

3.2.1. Stiffness

We use the normal force averaged over all normal force sensors and time steps as an indicator for the object stiffness. For the pressing movement with pressing time steps $t_P·f_s$, object stiffness can be estimated by $\overline{F}=\frac{1}{t_P·f_s}\frac{1}{N_f}\frac{1}{N_s}{\sum }_{m=1}^{t_P·f_s}{\sum }_{n_f=1}^{N_f}{\sum }_{n_s=1}^{N_s}{F}_{n_f,n_s}^m$.

3.2.2. Textural Property

In this work, we use the same textural feature extraction method in [43]: The vibration signals $\mathbf{a}$ in the artificial skin are used to calculate the activity, mobility and complexity features, denoted as $A\left(\mathbf{a}\right)$, $M\left(\mathbf{a}\right)$, $C\left(\mathbf{a}\right)$. These features represent the object tactile properties in the time domains. We also computed the linear correlation of accelerometer signals between different directions ($xy$,$yz$,$xz$) denoted as $L\left(\mathbf{a}\right)$, as these accelerometer components are correlated with each other during the sliding movement. The final descriptor of textural features combines activity, mobility, complexity and linear correlation together [43]: $TD=\left[A\right(\mathbf{a}),M(\mathbf{a}),C(\mathbf{a}),L(\mathbf{a}\left)\right]$.

3.2.3. Thermal Conductivity

To extract the features that describe the object thermal cues, we first calculate the average temperature sequence from all the temperature sensors: $\overline{\mathbf{T}}={\sum }_{n_t=1}^{N_t}{\sum }_{n_s=1}^{N_s}\frac{{\mathbf{T}}_{n_t,n_s}}{N_t·N_s}$. We then calculate its gradient at each time step as: $\nabla \overline{\mathbf{T}}$, and combine it with the average temperature sequence: $[\overline{\mathbf{T}},\nabla \overline{\mathbf{T}}]$. To avoid the curse of dimensionality, we further reduce this combination to 10 dimensions via Principle Component Analysis (PCA) method and use it as the final feature to describe the object thermal conductivity.

Table 2. Exploratory actions and perception.

Exploratory actions	Action Parameters ($\mathbf{\theta }$)	Sensory feedbacks	Features
Pressing	$d_P$, $t_P$	$\mathbf{F}$, $\mathbf{T}$	$\overline{F}$, $[\overline{\mathbf{T}},\nabla \overline{\mathbf{T}}]$
Sliding	$F_S$, $t_S$, $v_S$	$\mathbf{a}$, $\mathbf{T}$	$TD$, $[\overline{\mathbf{T}},\nabla \overline{\mathbf{T}}]$
Static contact	$d_C$, $t_C$	$\mathbf{F}$, $\mathbf{T}$	$\overline{F}$, $[\overline{\mathbf{T}},\nabla \overline{\mathbf{T}}]$

summarizes the exploratory actions, the sensory feedbacks and the corresponding tactile features.

4. Transferring Prior Tactile Exploratory Action Experiences

This section describes our proposed active prior tactile knowledge transfer algorithm (APTKT) in detail. First, we formulate our problem in Section 4.1. Then, we illustrate our transfer learning method, including its process (Section 4.3) and the problems of what to transfer (Section 4.4), how to transfer (Section 4.5), from where to transfer, and how much to transfer (Section 4.6). The motivation of our method is demonstrated in Figure 1.

4.1. Problem Formulation

Assume that a robotic system has gained prior tactile knowledge of some old objects, on which the robot has previously applied different exploratory actions with different action parameters. These prior exploratory action experiences consist of the feature observations perceived by the multiple sensors and observation models from the old objects. Now, the robot is tasked to learn about a set of new objects. Since the old objects might share some similar physical properties with the new objects, by leveraging the related tactile exploratory action experiences, the robot can learn about new objects more efficiently.

We define $N_{\mathrm{new}}$ number of new objects ($C^{\mathrm{new}}={\{c_j^{\mathrm{new}}\}}_{j=1}^{N_{\mathrm{new}}}$) the robot is tasked to learn about through different exploratory actions $A={\{{\mathbf{\alpha }}_n^{{\mathbf{\theta }}_n}\}}_{n=1}^{N_{\mathbf{\alpha }}}$ (For simplicity, we will denote $\mathbf{\alpha }$ as an exploratory action in the rest of the paper). In other words, the robot should actively attain object feature observations ($V_{\mathbf{\alpha }}^{\mathrm{new}}=\{V_{c_1}^{\mathrm{new}},V_{c_2}^{\mathrm{new}},\dots ,V_{c_{N_{\mathrm{new}}}}^{\mathrm{new}}\}$) for each exploratory action $\mathbf{\alpha }$ and construct reliable observation models $V_{\mathbf{\alpha }}^{\mathrm{new}}\stackrel{{\mathbf{f}}_{\mathbf{\alpha }}^{\mathrm{new}}}{\to }{C}^{\mathrm{new}}$. We further define the robot prior tactile experience for an exploratory action $\mathbf{\alpha }$ for $N_{\mathrm{old}}$ number of prior objects ($C^{\mathrm{old}}={\{c_i^{\mathrm{old}}\}}_{i=1}^{N_{\mathrm{old}}}$) as the prior object feature observations ($V_{\mathbf{\alpha }}^{\mathrm{old}}=\{v_{c_1}^{\mathrm{old}},V_{c_2}^{\mathrm{old}},\dots ,V_{c_{N_{\mathrm{old}}}}^{\mathrm{old}}\}$) and the observation models of old objects $V_{\mathbf{\alpha }}^{\mathrm{old}}\stackrel{{\mathbf{f}}_{\mathbf{\alpha }}^{\mathrm{old}}}{\to }{C}^{\mathrm{old}}$. These feature observations are collected by the multiple tactile sensors from the artificial robotic skin.

We formulate our problem as the transfer learning in the Gaussian Process Classification (GPC) framework [46], where each object is regarded as a class, and for each exploratory action, a GPC model is built as the observation model. The robot iteratively applies the exploratory actions and leverages prior tactile knowledge to improve the GPC observation models of new objects.

4.2. Gaussian Process Classification

The Gaussian Process Classification (GPC) model describes the mapping between the observation set X and the output set Y by: $X\stackrel{f}{\to }Y$. The latent function $g\left(\mathbf{x}\right)$ in the GPC model is assumed to be sampled from a high-dimensional gaussian distribution called GP prior [46]: $g\left(\mathbf{x}\right)\sim \mathcal{GP}(m\left(\mathbf{x}\right),K(\mathbf{x},{\mathbf{x}}^{\prime }))$, where each sample $g\left(\mathbf{x}\right)$ is a random variable. In this work, we use one-vs-all multi-class classification. For each object class, a binary GPC whose output label is converted to $\{-1,+1\}$ is trained for each of the N labels: $f_n(·)$. Given a new sample ${\mathbf{x}}^{\ast }$, each binary classifier predicts the observation probability of its label $p\left(y_n\right|{\mathbf{x}}^{\ast })$. The sample is assigned to the class with the largest prediction probability $y^{\ast }=\arg {\max }_{y_n\in Y}p\left(y_n\right|{\mathbf{x}}^{\ast })$.

4.3. Process

The robot following our proposed method first applies each exploratory action one time on each new object, in order to collect a small number of feature observations $V^{\mathrm{new}}={\{V_{{\mathbf{\alpha }}_n}^{\mathrm{new}}\}}_{n=1}^{N_{\mathbf{\alpha }}}$ (Initial data collection). Then, the robot reuses its prior tactile exploratory action experiences to improve the observation models for each new object (Initial prior knowledge transfer). During this process, the robot compares the relatedness between its prior tactile exploratory action experiences and the new objects (Section 4.6), and chooses the most related one to transfer the old object feature observations $V^{\mathrm{old}}$ (Section 4.5). Afterwards, the robot begins to iteratively collect and combine the feature observations and update the prior tactile knowledge in order to improve the observation models. At each iteration of prior tactile knowledge updating, the robot (1) actively selects the next object and the next exploratory action in order to attain a new feature observation; and (2) updates the prior tactile knowledge for the selected exploratory action. The iteration terminates when there is no improvement in the observation models of new objects. Our algorithm is demonstrated by Figure 4.

4.4. What to Transfer

When the robotic system applies an exploratory action on objects, it perceives multiple feature observations (e.g., by the pressing movement, the robot can perceive the object stiffness and thermal conductivity). The prior tactile exploratory action experiences are built using the feature observations of prior objects from multiple sensory modalities that are combined together and the corresponding GPC observation models of prior objects.

In order to combine the observations perceived from different tactile sensors, we first define ${\mathbf{v}}_{\mathbf{\alpha }}$ as the feature observation of an exploratory action $\mathbf{\alpha }$. It is comprised of multiple observations: ${\mathbf{v}}_{\mathbf{\alpha }}=[{\mathbf{v}}_{\mathbf{\alpha }}^{\left(1\right)},\dots ,{\mathbf{v}}_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)},\dots ,{\mathbf{v}}_{\mathbf{\alpha }}^{\left(M_{\mathbf{\alpha }}\right)}]$, where ${\mathbf{v}}_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)}$ is an observation from the sensor modality $m_{\mathbf{\alpha }}$, $M_{\mathbf{\alpha }}$ is the number of sensing modalities. For the pressing and static contact movements, we use the normal force and temperature sensing, for the sliding movement the accelerometer and temperature sensing (Table 2). Then, we assume that for a sensor modality $m_{\mathbf{\alpha }}$, a kernel function $K^{\left(m_{\mathbf{\alpha }}\right)}$ is given. To combine multiple feature observations so as to exploit the information from all sensors after applying the exploratory action $\mathbf{\alpha }$, we linearly combine the kernels:

$$K_{\mathbf{\alpha }}^{\prime }={\gamma }_{\mathbf{\alpha }}^{\left(1\right)}{K}^{\left(1\right)}+\dots +{\gamma }_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)}{K}^{\left(m_{\mathbf{\alpha }}\right)}+\dots +{\gamma }_{\mathbf{\alpha }}^{\left(M_{\mathbf{\alpha }}\right)}{K}^{\left(M_{\mathbf{\alpha }}\right)},$$

(1)

where ${\gamma }_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)}\ge 0$. This hyper-parameter controls how much the robot can rely on the sensor modality $m_{\mathbf{\alpha }}$. It ranges between 0 and 1, with ${\gamma }_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)}=0$ indicating that the sensor feedback is not informative, and ${\gamma }_{\mathbf{\alpha }}^{\left(m_{\mathbf{\alpha }}\right)}=1$ highly useful. We further constrain these hyper-parameters with $L_1$ norm: $|{\sum }_{m_{\mathbf{\alpha }=1}}^{M_{\mathbf{\alpha }}}{\gamma }_{\mathbf{\alpha }}^{(m_{\mathbf{\alpha }})}|=1.$ For each exploratory action, a GPC observation model is built using $K^{\prime }$. The hyper-parameters of $\gamma$ and kernels are selected by maximizing the log marginal likelihood [46]. Figure 5 illustrates our multiple feature observations combination method. It is also demonstrated by Algorithm 1.

Algorithm 1 Multiple Feature Observations Combination

4.5. How to Transfer

Taking advantage of our previously proposed method [45], the robotic system transfers the feature observations of a prior object $c_i^{\mathrm{old}}$ to learn the GPC observation model of a new object $c_j^{\mathrm{new}}$, based on an exploratory action $\mathbf{\alpha }$. For simplicity, we hereby refer to i and j as $c_i^{\mathrm{old}}$ and $c_j^{\mathrm{new}}$, respectively. We define ${\mathbf{g}}_i^{\mathrm{old}}$ as the Gaussian Process latent function values [46] for the old object $c_i^{\mathrm{old}}$ and ${\mathbf{g}}_j^{\mathrm{new}}$ for the new object $c_j^{\mathrm{new}}$. We assume that these two function values are not independent of each other, but are sampled together over a dependent Gaussian Prior (GP). This dependent GP is then used to construct the GPC observation model of the new object. The latent function can be modified accordingly: ${\mathbf{g}}_j^{{\mathrm{new}}^{\prime }}\leftarrow [{\mathbf{g}}_i^{\mathrm{old}},{\mathbf{g}}_j^{\mathrm{new}}]$ [45]. We further incorporate the relatedness between prior object and new object into the dependent GP model by introducing the following dependent kernel function:

$$\begin{array}{c}\hfill K^{\prime }=\left[\begin{array}{cc}K(V_i^{\mathrm{old}},V_i^{\mathrm{old}})& \lambda K(V_i^{\mathrm{old}},V_j^{\mathrm{new}})\\ \lambda K(V_j^{\mathrm{new}},V_i^{\mathrm{old}})& K(V_j^{\mathrm{new}},V_j^{\mathrm{new}})\end{array}\right].\end{array}$$

(2)

$K(V_i^{\mathrm{old}},V_i^{\mathrm{old}})$ and $K(V_j^{\mathrm{new}},V_j^{\mathrm{new}})$ serve as the kernel matrix that measures the similarity among all feature observations of the old object and the new object, respectively. Each element in the kernel matrix measures the similarity between two feature observations, which is calculated by the radial basis function (RBF). $\lambda K(V_j^{\mathrm{new}},V_i^{\mathrm{old}})$ and $\lambda K(V_i^{\mathrm{old}},V_j^{\mathrm{new}})$ are the kernel matrix between the old object and the new object. $\lambda$ controls the relatedness, or similarity, between $c_i^{\mathrm{old}}$ and $c_j^{\mathrm{new}}$. We constrain its range within $[0,1]$. As Chai et al. [47] evaluated, $\lambda =0$ indicates that the old object and the new object are totally different, while $\lambda =1$ indicates that the two objects are the same.

4.6. From Where and How Much to Transfer

Section 4.5 describes how to transfer the prior tactile knowledge to learn about new objects. This section illustrates how the robotic system selects the most related old object (from where to transfer) and how to determine the relatedness ($\lambda$) between two objects (how much to transfer).

To do this, we use our previously proposed method [45]. Let $p(c_i^{\mathrm{old}}|{\mathbf{v}}_j^{\mathrm{new}})$ be the prediction probability that a feature observation from the new object ${\mathbf{v}}_j^{\mathrm{new}}$ is assigned to the old object $c_i^{\mathrm{old}}$. We measure the average prediction to all the observations ${\mathbf{v}}_j^{\mathrm{new}}\in V_j^{\mathrm{new}}$ that belong to the new object: $\overline{p}(c_i^{\mathrm{old}}|V_j^{\mathrm{new}})=\frac{1}{N_j^{\mathrm{new}}}\sum p(c_i^{\mathrm{old}}|{\mathbf{v}}_j^{\mathrm{new}})$, with $N_j^{\mathrm{new}}$ being the number of new object feature observations. This average prediction value indicates the similarity between the old object $c_i^{\mathrm{old}}$ and the new object $c_j^{\mathrm{new}}$. A larger value indicates that these two objects are highly similar. Therefore, we can use it to select the most related old object (denoted as $c^{{\mathrm{old}}^{\ast }}$) for a new object regarding the exploratory action $\mathbf{\alpha }$. Furthermore, to avoid transferring irrelevant tactile information, we add a threshold ${ϵ}_{neg}$ which prevents the robot from selecting any old object when the prediction value is smaller than ${ϵ}_{neg}$. The final old object selection criterion is:

$$\begin{array}{c}\hfill c^{{\mathrm{old}}^{\ast }}=\left\{\begin{array}{cc}\underset{c_i^{\mathrm{old}}\in C^{\mathrm{old}}}{\arg \max }\overline{p}(c_i^{\mathrm{old}}|V_j^{\mathrm{new}}),\hfill & \mathrm{if}\overline{p}(c^{{\mathrm{old}}^{\ast }}|V_j^{\mathrm{new}})\ge {ϵ}_{neg}\hfill \\ \mathrm{None},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(3)

Once we select $c^{{\mathrm{old}}^{\ast }}$, we further use the predictions from the observation model of old objects to determine the object relatedness ${\lambda }^{\ast }$: ${\lambda }^{\ast }=\overline{p}(c^{{\mathrm{old}}^{\ast }}|V^{\mathrm{new}})$.

4.7. Prior Exploratory Action Experiences Update

When the robot updates its prior exploratory action experiences, it needs to iteratively collect a new feature observation by applying an exploratory action on an object. We use our previously proposed active tactile learning algorithm [43] called Active Touch for Learning Physical Properties (AT-LPP). Using our AT-LPP approach, the robot actively decides which new feature on the object to explore next (denoted as $c^{{\mathrm{new}}^{\ast }}$) and which physical property to learn next (which exploratory action to apply next). It is denoted as ${\mathbf{\alpha }}^{\ast })$. In the following, we briefly summarize the AT-LPP algorithm (Algorithm 2) [43].

The robot first calculates the Shannon entropy of the object posterior for a new feature observation ${\mathbf{v}}^{\mathrm{new}}$ with the equation: $H(c^{\mathrm{new}}|{\mathbf{v}}^{\mathrm{new}})=-{\sum }_{c_j^{\mathrm{new}}\in C^{\mathrm{new}}}p(c_j^{\mathrm{new}}|{\mathbf{v}}^{\mathrm{new}})\log (p(c_j^{\mathrm{new}}|{\mathbf{v}}^{\mathrm{new}}))$. Then the robot estimates the uncertainty in the GPC model with regard to each exploratory action and new object by the mean value of the Shannon entropy: $\mathrm{UNC}({\mathbf{\alpha }}_n,c_j)={\displaystyle \frac{1}{N_{{\mathbf{\alpha }}_n,j}^{\mathrm{new}}}}{\sum }_{{\mathbf{v}}_{{\mathbf{\alpha }}_n,j}^{\mathrm{new}}\in V_{{\mathbf{\alpha }}_n,c_j^{\mathrm{new}}}^{\mathrm{new}}}H(c_j^{\mathrm{new}}|{\mathbf{v}}_{{\mathbf{\alpha }}_n,j}^{\mathrm{new}})$, where ${\mathbf{v}}_{{\mathbf{\alpha }}_n,j}^{\mathrm{new}}$ refers to the a feature observation the robot has collected for the new object $c_j^{\mathrm{new}}$ and exploratory action ${\mathbf{\alpha }}_n$; $N_{{\mathbf{\alpha }}_n,j}^{\mathrm{new}}$ is the number of feature observations. A large $\mathrm{UNC}({\mathbf{\alpha }}_n,c_j)$ indicates that the robot is uncertain about the object feature observations from the exploratory action ${\mathbf{\alpha }}_n$. As discussed in [43], an efficient next object and the next action selection process should be considered to greedily reduce such uncertainty while at the same time allowing the robot to explore (exploration-exploitation trade-off). In this regard, the next exploratory action ${\mathbf{\alpha }}^{\ast }$ and the next object $c^{{\mathrm{new}}^{\ast }}$ are determined by:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{c}^{{\mathrm{new}}^{\ast }},{\mathbf{\alpha }}^{\ast }=\underset{{\mathbf{\alpha }}_n\in A;c_j^{\mathrm{new}}\in C^{\mathrm{new}}}{\arg \max }\mathrm{UNC}({\mathbf{\alpha }}_n,c_j^{\mathrm{new}})\hfill & \mathrm{if}{p}_{\mathrm{rand}}\ge {ϵ}_{\mathrm{explor}}\hfill \\ c^{{\mathrm{new}}^{\ast }}=\mathcal{U}\{c_1^{\mathrm{new}},c_2^{\mathrm{new}},\dots ,c_{N_{\mathrm{new}}}^{\mathrm{new}}\},{\mathbf{\alpha }}^{\ast }=\mathcal{U}\{{\mathbf{\alpha }}_1,{\mathbf{\alpha }}_2,\dots {\mathbf{\alpha }}_{N_{\mathbf{\alpha }}}\}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(4)

where ${ϵ}_{\mathrm{explor}}$ is the exploration rate, and $p_{\mathrm{rand}}$ is randomly generated following the uniform distribution $\mathcal{U}(0,1)$.

Algorithm 2 Active Touch for Learning Physical Properties

Once the robot collects a new feature observation, it updates the prior tactile exploratory action experiences only from action ${\mathbf{\alpha }}^{\ast }$. This process includes updating the feature observation combination, updating the object relatedness $\lambda$, and transferring these prior feature observations to the observation models of new objects.

5. Experimental Results

5.1. Experimental Objects

In order to evaluate the performance of the proposed active prior tactile knowledge transfer algorithm (APTKT), we deliberately selected 10 daily objects with different physical properties which served to build the robotic prior to tactile exploratory action experiences (see Figure 1 Prior objects). Furthermore, we selected five new objects about which the robotic system was tasked to learn (Figure 1 New objects). For each new object, there existed one or more old objects that shared similar physical properties. For example, both rough sponge and smooth sponge are soft; paper box and hard box have similar surface textures; metal toolbox and biscuit box have high thermal conductivity. In this way, when learning about new objects based on their physical properties, the robot can leverage the related prior tactile knowledge.

5.2. Exploratory Action Determination and Test Data Collection

In our experiment, we defined seven exploratory actions from the pressing, sliding, and static contact movements with various action parameters (Pressing: $P1$, $d_P=1$ mm, $t_P=3$ s; $P2$, $d_P=2$ mm, $t_P=3$ s. Sliding: $S1$, $F_S=0.1$ N, $t_S=5$ s $v_S=1$ cm/s; $S2$, $F_S=0.1$ N, $t_S=1$ s, $v_S=5$ cm/s; $S3$, $F_S=0.2$ N, $t_S=5$ s, $v_S=1$ cm/s; $S4$, $F_S=0.2$ N, $t_S=1$ s, $v_S=5$ cm/s. Static Contact: $C1$, $d_C=2$ mm, $t_C=15$ s). Before applying any of the seven exploratory actions, the robot established light contact with the objects which were detected once the total normal force on the artificial skin increased above $0.05$ N. Furthermore, after applying an exploratory action, the robot was controlled to raise its end-effector for 30 s such that the temperature sensors could be restored to the ambient temperature.

We evaluated the performance of our proposed method based on a test dataset built by the robot by applying each actions 20 times on each object. During this process, objects were manually shifted and rotated so that the data was robust against the variations in the object contact locations with the artificial skin.

5.3. Evaluation of Multiple Feature Observations Combination Method

We first evaluated the performance of our proposed robotic multiple feature observation combination algorithm. To do this, the robot selected 10 groups of objects (shown in Figure 1) to construct the GPC observation models for each of the seven exploratory actions. Each group contained five objects that were selected randomly both from the old and new object lists, following a uniform distribution. The algorithm performance was evaluated by the discrimination accuracy of the test dataset predicted by the GPC models with the growing number of feature observations. We compared our method with the baseline methods that built the GPC models using only a single sensor modality.

The experiments were conducted 10 times for each object group. For a fair comparison, we used RBF kernel [46] for each sensor modality. Results are plotted in Figure 6. For all seven exploratory actions, our proposed algorithm either took advantage of combining different sensor modalities to reach the best discrimination accuracy ($P1$, $P2$, $C1$, $S4$ in Figure 6), or performed the same as the best single-sensor result ($S1$, $S2$, $S3$ in Figure 6), indicating that the robot actively selected the most informative sensory feedback to learn about objects.

5.4. Evaluation of the Transfer Learning Method with Different Groups of Prior Objects

In this experiment, we evaluated the performance of our proposed transfer learning method (APTKT) to learn the five new objects (see new objects in Figure 1) with different groups of prior objects (see prior objects in Figure 1). To start the learning process, the robot applied each of the seven actions once on each new object. When the robot iteratively learned the new objects’ physical properties, it updated both the multiple feature observations combination and the prior tactile knowledge built by the dependent GPC models with all the feature observations collected so far. At each learning iteration, we measured the object discrimination accuracy of the test dataset. The transfer learning performance was compared with the baseline learning method that combined multiple feature observations without transferring any prior tactile knowledge.

We randomly shuffled the prior objects into ten groups following a uniform distribution. Each group consisted of the feature observations and the observation models from three prior objects. We conducted the experiment with five trials for each group. In each trial, the robot followed the transfer learning approach and no-transfer approach to collect 40 feature observations in total, allowing a fair comparison between different learning strategies to be made. Figure 7 illustrates that with the help of prior knowledge, the robot consistently outperformed the learning process without prior knowledge with a discrimination accuracy of $10\%$.

In order to further evaluate the robustness of APTKT, the robot was then tasked to learn about objects via applying only one of the exploratory actions. The experimental procedure was the same as the one described above. As the results in Figure 8 show, The robot had a larger improvement by actions P1, P2 and C1 than actions S1, S2, S3 and S4. For example, the robot increased the discrimination accuracy by $25\%$, when it reused the prior tactile instance knowledge from the movement P2. However, when learning about objects by actions S1 and S4, little improvement was seen. This was due to the fact that different exploratory actions produced different object feature observations. For action P2, there existed higher related prior tactile knowledge than S1 and S4, and the robot could benefit more on it.

In all scenarios, using our proposed transfer learning algorithm, the robot could achieve a higher discrimination accuracy than the baseline method with the same number of feature observations. Therefore, we can conclude that APTKT helps the robot build reliable observation models of new objects with fewer training samples, even when only one kind of exploratory action is applied.

5.5. Increasing the Number of Prior Objects

We further evaluated the performance of our proposed method with an increasing number of prior tactile experiences. Intuitively, as the number of old objects grows, it is more likely that the robot can find highly-related prior tactile knowledge, so that the learning performance can continue to be improved. In this regard, the robot was asked to learn about new objects via all seven exploratory actions, with the number of old objects increasing from 5, 7 to 10. We followed the same experimental procedure described above, and conducted each experiment with five trials. Unexpectedly, as Figure 9b–d show, the growing number of prior tactile knowledge reduced the transfer learning improvement. This was because as the number of prior objects grow, it was more difficult for the robot to classify them. As a result, the object relatedness $\lambda$ predicted by the old object GPC models was lower than the threshold ${ϵ}_{\mathrm{neg}}$, making the robot stop transferring prior knowledge.

To compensate for this, we use our previously proposed feature augmentation trick [45]. We defined $p(c_i^{\mathrm{old}}|\mathbf{v})$ as the prediction probability that a feature observation from the new object $\mathbf{v}$ is assigned to the old object $c_i^{\mathrm{old}}$. Then we augmented a feature observation $\mathbf{v}$ from a new object as:

$$\begin{array}{c}\hfill {\mathbf{v}}^{\prime }=[\underset{\mathrm{original}\mathrm{features}}{\underbrace{\mathbf{v},}}\underset{\mathrm{predictions}\mathrm{from}\mathrm{old}\mathrm{objects}{}^{\prime }\mathrm{observation}\mathrm{models}}{\underbrace{p(c_1^{\mathrm{old}}|\mathbf{v}),\dots ,p(c_i^{\mathrm{old}}|\mathbf{v}),\dots ,p(c_{N_{\mathrm{old}}}^{\mathrm{old}}|\mathbf{x})}}].\end{array}$$

(5)

The auxiliary features $[p(c_1^{\mathrm{old}}|\mathbf{v}),\dots ,p(c_{N_{\mathrm{old}}}^{\mathrm{old}}|\mathbf{v})]$ encode the knowledge of all prior objects. They represent the relatedness between prior objects and the new object, and thus can help the robotic system to distinguish among new objects. Furthermore, since the auxiliary features can be regarded to be perceived from an auxiliary sensor, we directly employed our proposed multiple feature observation combination method to the augmented feature observations by casting a weight $\gamma$ to its kernel. The augmented feature observations were then used to build the new object dependent GPC models.

We tested our proposed feature augmentation technique when the robot leveraged the tactile knowledge of 3, 5, 7, and 10 prior objects to learn about new objects via all seven actions. The learning performance is shown by the green curves in Figure 9a–d. Clearly, by introducing the probability predictions as auxiliary features, the robot was able to reuse the prior tactile knowledge again, and it achieved similar improvement of discrimination accuracy for 3 prior objects, and higher improvement for 5, 7, and 10 prior objects compared to the other methods. Specifically, when resuing 10 prior objects, the robot achieved $20\%$ higher discrimination accuracy than the baseline method, when only one new feature observation was collected, showing the one-shot learning behaviour. This experiment also indicates that with a further growing number of prior objects, a further improvement of discrimination accuracy is achievable.

5.6. Negative Prior Tactile Knowledge Transfer Testing

When the constructed prior tactile exploratory action experiences are not relevant to the new objects, a brutal-force transfer may degrade the learning performance, resulting in the negative knowledge transfer phenomena. In this case, the transfer learning algorithm should stop leveraging irrelevant prior knowledge.

In order to evaluate our proposed transfer learning method (APTKT) against the negative tactile knowledge transfer, we deliberately selected irrelevant prior objects and compared the transfer learning performance with the baseline method, following the same experimental process described in Section 5.4. When finding which objects were relevant (or irrelevant) to each other, we built object confusion matrices to roughly evaluate the object similarity. For each of the seven exploratory actions, we trained a Gaussian Mixture Model (GMM) and calculated the object confusion matrix. To do this, we first used GMM to cluster all the samples from the dataset with the hyper-parameters optimized by the Expectation-Maximization (EM) algorithm. The number of clusters was set to be the same as the number of objects (in our case, 15), and each cluster centroid was initialized as the mean value of all data samples that belonged to an object. The maximum EM iterations was set to be 100, with convergence threshold being $0.001$. We further calculated the confusion matrix averaged over all exploratory actions. These matrices indicated the averaged similarity between objects. We rescaled their values to be within 0-1, with 0 meaning that two objects are totally dissimilar, and 1 the same. The objects which had low similarity values with target objects were selected as irrelevant objects. The results are shown in Figure 10. According to Figure 10, prior objects {1, 5, 7} (objects {1–10}) were dissimilar to the new objects (objects {11–15}) regarding the exploratory movement P1, objects {1, 4, 7} for P2, objects {4, 7, 10} for C1, objects {1, 6, 9} for S1, objects {1, 7, 10} for S2, objects {1, 3, 9} for S3, and objects {1, 3, 8} for S4. We thus used these objects as prior objects to test the transfer learning performance via the single exploratory action. We further selected objects {1, 5, 10} to test the learning process via all exploratory actions, since these three objects shared relative small similarity to the new objects.

The results in Figure 11 illustrate that the discrimination accuracy achieved by APTKT was similar to the baseline method, when the robot applied either one or all seven exploratory actions. The results indicate that our proposed algorithm stopped transferring negative prior tactile instance knowledge.

6. Conclusions

In this work, we proposed a transfer learning method for a robot equipped with multi-modal artificial skin to actively reuse the prior tactile exploratory action experiences when learning about the detailed physical properties of new objects. These prior action experiences are built by the feature observations, when the robotic arm applies the pressing, sliding and static contact movements with different action parameters on the previous-explored objects (prior objects). The feature observations are perceived from multiple sensory modalities. Using our proposed tactile transfer learning method, the robot has a "warm start" of the learning process. It applies fewer exploratory actions and gains a detailed tactile knowledge of new objects (e.g., normal force feedback at different pressing depths).

One limitation of our work is that performing static contact movement took 15 s, which prevented the rapid transfer learning. Furthermore, due to the limitations of our artificial skin, the robot can only interact with objects with flat surfaces. In the future, we will extend our method to more exploratory actions (such as tapping and lifting), so that the robot can transfer more exploratory action experiences to learn more physical properties of an object, such as auditory feedback and center of mass. Furthermore, an interesting topic would be how to transfer the prior tactile knowledge across different exploratory actions, e.g., transferring the tactile knowledge from pressing to static contact movement.