*3.2. Data Set Acquisition*

In order to train the machine learning algorithms, a representative data set needs to be acquired. For this purpose, the error recovery sequence is used to acquire the force/torque signals on every defined error position.

To safely perform constraint movements with the dual arm robot, considering the control strategy for a discrete event system, two primitive movement functions are defined to avoid damage to the system. This can be described as follows:

$$move\_{\max}(f) = \begin{cases} 1 & f < f\_{\max} \\ 0 & f \ge f\_{\max} \end{cases} \tag{4}$$

$$movc\_{min}(f) = \begin{cases} 1 & f > f\_{min} \\ 0 & f \le f\_{min} \end{cases} \tag{5}$$

$$f = \sqrt{f\_{\mathfrak{X}}^2 + f\_{\mathfrak{Y}}^2 + f\_{\bar{z}}^2} \tag{6}$$

where *movemax*(*f*) is a function that starts the robot movement whenever the force *f* is less than the maximum limit *fmax* and the movement stops whenever the force *f* is greater than the limit *fmax*. For the case of the *movemin*(*f*), the function starts or stops the robot movement according to the conditions based on the limit *fmin*. These primitive movements are used for every step of the sequence in Figure 3 according to logic constraints.

**Figure 3.** Petri-Net representation of the sequence.

Step 1 moves the robot to the error position as shown in Figure 1a. The error position is selected randomly. On step 2, the orientation is selected according to the error position as shown in Figure 1b. In step 3, the peg is moved along the X-Z plane in the -Z direction until a lateral contact force is reached Figure 1c, step 4 sets the orientation to be aligned with the hole Figure 1d and, finally, step 5 moves the peg towards insertion Figure 1e.

The sequence is repeated for each error position, gathering contact state information to make the data set for training, testing, and to make a validation data set.

#### *3.3. Data Set Analysis*

In order to analyze whether the data set can be used for learning the contact states, the t-distributed stochastic neighbor embedding (t-SNE) method is performed. As can be seen in the results, this graphical representation helps find out whether there are separable classes. The t-SNE [35] is a nonlinear dimensionality reduction technique that helps in visualization of data in a low dimensional space.

### *3.4. Standard Automation Logic Rules*

In the case of programming a standard automated program, logic rules are used to trigger transitions among sequence steps. Variable limits or thresholds to classify F/T data are presented in [7,23]. To find whether this method can be used for this research, the data set is analyzed using statistical measures. Thresholds are used as logic conditions to classify the F/T input values in accordance to the next equation:

$$Th\_m = \begin{cases} \lim\_{\substack{\sup}} & \mu\_M + \sigma\_M \\ \lim\_{\inf} & \mu\_M - \sigma\_M \end{cases} \tag{7}$$

where *μ<sup>M</sup>* corresponds to the mean of the momentum in any axis and *σ<sup>M</sup>* corresponds to the standard deviation of the momentum in any axis. This defines the value of the superior and inferior limits: *limsup* and *liminf* .

#### *3.5. Deep Neural Network Definition*

Deep Neural Networks can map the force/torque signals to the defined error position class. These classes can also map the corresponding robot actions. Contact states are defined as quasi-static motion, which means that velocity of the motion is not taken into account.

For this investigation, the contact states that are under analysis are those that occur during the first assembly stage: the initial approach between peg and hole. Although the error position during this stage is unknown, a set of error positions can be defined to convert the case into a classification problem.

The sensors deliver x, y, z components for the force and momentum. Instead of using separate sensor signals, a single F/T vector of 12 features is proposed as shown next:

$$FT = \begin{bmatrix} f\_{x1} \ f\_{y1} \ f\_{z1} \ m\_{x1} \ m\_{y1} \ m\_{z1} \ f\_{x2} \ f\_{y2} \ f\_{z2} \ m\_{x2} \ m\_{y2} \ m\_{z2} \end{bmatrix} \tag{8}$$

where *FT* represents the force/torque vector, and the sub-indexes *x*1, *x*2 and so on, represent the component from sensor 1 or sensor 2.

Then, a DNN model as expressed in the next equations is defined experimentally by changing parameters and hyper-parameters to reach an acceptable accuracy during the training and testing.

$$y = \sigma(\sum\_{i=0}^{n} W\_i X\_i) \tag{9}$$

$$
\sigma = \frac{e^z - e^{-z}}{e^z + e^{-z}} = \tanh
\tag{10}
$$

$$\text{Loss}(\mathcal{Y}, \mathcal{Y}) = -\sum\_{i=1}^{2} y\_i \log \left( \mathcal{Y}\_i \right) \tag{11}$$

$$S(y)\_i = \frac{\exp\left(y\_i\right)}{\sum\_{j=1}^n \exp\left(y\_i\right)}\tag{12}$$

where *Xi* corresponds to the number of features or inputs, *y* correspond to the number of outputs or classes, *h* correspond to the number of hidden layers, and *n* to the number of hidden units.

In order to pre-process the data set for training the DNN, a standardization process corresponding to the following equation is executed.

$$
\hat{X} = \frac{X - \mu}{\sigma} \tag{13}
$$

where *X*ˆ is the normalized sensor features, *X* is the input feature, *μ* the mean of each feature in the data set, and *σ* the standard deviation of each feature in the data set.

A second DNN model is designed to classify basic contact states such as orthogonal contacts in all axes *X*, *Y*, *Z*. That means that the second DNN model will have the same 12 input features and 5 outputs corresponding to the actions: move to +*X*, move to −*X*, move to +*Z*, move to −*Z*, and move to the insertion position −*Y*. This model classifies the contact states during the insertion step. In any case, a small deviation causes a contact state force *f* > *flimit* that commands the robot to stop insertion and align according to the second DNN model output.

The trained DNN model helps the system classify the contact states caused by the error position. This classification selects the relative position to which the robot should move next. Other algorithms are also trained and tested with the same data set to compare the performance.

#### *3.6. Dnn Evaluation and Implementation on the Robot Controller*

For the purpose of evaluating the DNN performance, different data sets were generated running the same sequence on different dates. The trained model is packed and loaded to run an offline validation test.

The DNN model is then packed as a computational library to be integrated into the robot controller. Thus the robot controller is programmed using the robot operating system (ROS). The libraries were programmed in python and packed as a Service-Node. The force/torque vector is received and processed running the DNN model. The result is sent back to the main control program that processes the data in the sequence logic.

#### *3.7. Error Recovery Sequence*

Since the DNN models present misclassifications in a real scenario, the assembly sequence is strengthened by adding logic constraints in every transition between steps. Two repetition cycles are added to the sequence. The first is represented as step S7, which repeats an orientation selection in case that the first DNN model outputs a wrong class, causing different contact states conditions. This can be identified and then triggers transition T6 to start this correction cycle.

For the case of the final insertion step S5, in case the insertion forces exceed the defined limits, the logic conditions will trigger T8 to start the adjustment cycle S8, where the second DNN model outputs a robot action that moves the peg to a center position.

This is one of the strengths of a discrete event system, that any step can be adjusted without modifying the whole control strategy. This new sequence program would lead the system to increase the rate of successful assemblies. Figure 4 shows the petri-net that represents the recovery sequence.

**Figure 4.** Modified sequence considering recovery steps.

The rule for trying another orientation after the first contact step is defined as follows:

$$o\_{n+1} = \begin{cases} o\_{\text{lt}} + 1 & f > f\_{\text{min}}, o\_{\text{axis}} < o\_{\text{max}}, try = 1 \\ o\_{\text{lt}} - 1 & f > f\_{\text{min}}, o\_{\text{axis}} < o\_{\text{max}}, try = 2 \end{cases} \tag{14}$$

The rule for aligning the peg during the insertion step uses a DNN trained with basic contact states in order to output a reaction movement according to the predicted F/T pattern. The logic rule is described in the Table 1.


**Table 1.** Centering Rule for the correction position step.

The first four patterns consider *X* and *Z* axis movements for error correction and for the pattern # 5; no error correction is commanded in the X-Z plane but there is a movement in the Y axis to continue insertion.

#### *3.8. Validation Process of the Methodology*

A validation process of different data sets is executed and the results are compared with the training set. A comparison of threshold methodology of single sensor data with DNN single sensor data, and then DNN with dual sensor data, is also executed.

A counter variable is set to limit the number of iterations or attempts in the assembly sequence.

In order to evaluate the performance of the machine learning algorithms, the Exact Match Ratio (MR) [36] as presented in the next equation is used to report experimental results.

$$MR(\mathcal{Y}) = \frac{1}{n} \sum\_{i=1}^{n} I\_n \tag{15}$$

$$I\_{\mathbb{N}} = \begin{cases} 1 & \text{if } & \text{\$\mathfrak{y} = \$y\$} \\ 0 & \text{otherwise} \end{cases} \tag{16}$$

where *n* is the sample number and *In* is a logic function of the correct classification of the sample.

Once the DNN models are trained and integrated into the assembly sequence, Success Ratio (SR) is used to evaluate the effects of using deep neural networks with one sensor and a double sensor in the automated cycle. *SR* is defined as follows:

$$SR = \frac{1}{n} \sum\_{i=1}^{n} SA\_{\pi} \tag{17}$$

$$SA\_{\mathcal{U}} = \begin{cases} 1 & \text{if } \quad p\_{\text{pos}} \ge p\_{i\text{-pos}} \\ 0 & \text{otherwise} \end{cases} \tag{18}$$

where *SR* is the Success Ratio, *SAn* is the successful assembly that is true if the peg position *ppos* at the end of the cycle is greater or equal than the defined insertion position *pi*\_*pos* and *n* is the sample number.

#### **4. Experimental Results**

#### *4.1. TestBed*

The robotic testbed is composed of an industrial robot assembly cell that includes a dual arm robot SDA20D with two 7 DOF articulated arms and 1 rotating base giving a total of 15 DOF (degrees of freedom) as a whole system Figure 5. The robot controller is a DX100 with Ethernet port.

The maximum payload is 20 kg per arm, and repetitive positioning accuracy ±0.1 mm. Mounted in each arm are the force/torque sensors in a range with ±300 N and Torque

±30 Nm with 3 Cartesian components for each magnitude with a maximum data stream of 100 Hz for all component signals. Noise around 1.2 N and 0.02 Nm. Recommended threshold 5 N and 0.12 Nm maximum.

**Figure 5.** Motoman SDA20D Dual Arm Robot.

Both sensors are connected to an RS-485 to USB converter that sends the data to a Raspberry pi 3 running Robot Operating System (ROS). The parts to be assembled are a circular peg and hole in stainless steel with diameter of 40 mm and 39.9 mm for the peg with 50mm length. Both are fixed to the robot arms.

The main PC controller runs on Linux Ubuntu 18, and ROS Melodic is connected to the robot by the Ethernet 100 MB. A second PC controller runs Linux Ubuntu 20 and ROS Noetic, which runs the Deep Neural Network model as a service node. The robot has installed the MotoROS library that allows transfer of data via TCP/IP to the PC controllers and the Motodriver for ROS. Figure 6 shows a representation of the setup.

**Figure 6.** Experimental Setup.
