Kinematic and Joint Compliance Modeling Method to Improve Position Accuracy of a Robotic Vision System

Abstract

In the field of robotic automation, achieving high position accuracy in robotic vision systems (RVSs) is a pivotal challenge that directly impacts the efficiency and effectiveness of industrial applications. This study introduces a comprehensive modeling approach that integrates kinematic and joint compliance factors to significantly enhance the position accuracy of a system. In the first place, we develop a unified kinematic model that effectively reduces the complexity and error accumulation associated with the calibration of robotic systems. At the heart of our approach is the formulation of a joint compliance model that meticulously accounts for the intricacies of the joint connector, the external load, and the self-weight of robotic links. By employing a novel 3D rotary laser sensor for precise error measurement and model calibration, our method offers a streamlined and efficient solution for the accurate integration of vision systems into robotic operations. The efficacy of our proposed models is validated through experiments conducted on a FANUC LR Mate 200iD robot, showcasing notable improvements in the position accuracy of robotic vision system. Our findings contribute a framework for the calibration and error compensation of RVS, holding significant potential for advancements in automated tasks requiring high precision.

1. Introduction

The advent of robotic vision systems (RVSs) has ushered in a new era of robotic capabilities, fundamentally transforming the scope and efficiency of automated tasks across various industries [1,2]. The intrinsic value of such systems lies in their ability to perform complex visual tasks with remarkable accuracy, from intricate assembly operations in manufacturing to delicate surgical procedures in medicine. High-precision RVSs enable robots to detect, recognize, and manipulate objects with a level of detail and accuracy previously unattainable, bridging the gap between robotic automation and tasks requiring human-like dexterity and visual acuity.

The position accuracy of an RVS is defined by the system’s capability to precisely locate a target within its field of view [3]. This accuracy is gauged by the system’s effectiveness in pinpointing an object’s position relative to its true location in the physical world. Enhancing this position accuracy necessitates robust modeling and calibration processes. These processes entail the acquisition of part point clouds via the vision system, followed by the transformation of these data points into a consistent representation of the target features within the Cartesian coordinate system. Therefore, meticulous modeling is paramount in improving the position accuracy of an RVS. Current methodologies for RVS modeling are bifurcated into kinematic and compliance models, each addressing different aspects of system behavior and contributing to the overall precision of the system [4].

The kinematic model primarily involves the identification of structural parameters of basic robotic movements [5] and the “hand eye” parameters [6,7] between the robot and camera coordinate systems. The D-H model proposed by Denavit and Hartenberg has become widely applied in kinematic modeling within the industrial robotics field, serving as a standard for an extended period [8,9]. Subsequent improvements to this model, such as those introduced by Hayati, incorporate angles between adjacent parallel joints [10]. Du Guanglong presents a novel approach for online robotic kinematic calibration, which, by integrating an Unscented Kalman Filter and an Iterative Particle Filter, achieves precise identification of robotic kinematic parameters without necessitating operational halts [11]. Joubair introduces a kinematic calibration approach utilizing distance and sphere constraints, significantly enhancing the position accuracy of a six-axis serial industrial robot within a specific target workspace [12]. The most common method for hand–eye calibration is based on estimating transformation matrices. Through reprojection error minimization, Koide directly utilizes calibration pattern images, without the need for explicit camera pose estimation [13]. Hua Jiang successfully demonstrates a robot hand–eye calibration approach utilizing an optimized neural network to accurately model the complex, nonlinear relationship between camera and robotic coordinates [14].

The robot compliance model focuses on the elasticity of the materials constituting the robot, addressing deformation errors under applied forces [15,16]. Due to its own weight and external loads, the robot experiences structural deformation in its links and joints [17]. Current research on the joint stiffness modeling of six-degree-of-freedom robots mainly analyzes deformations in joints 2 and 3, establishing simplified linear torsion spring models [18]. Abele utilized a linear torsion spring model and the Jacobian matrix to create a static compliance model for robots, enhancing tool path accuracy [19]. Dumas et al. developed a stiffness model for serial robots based on translational and rotational errors, experimentally validated on the Kuka KR240 robot, identifying compliance coefficients for its six joints [20]. P Kozlov used a CAD virtual experimental environment and finite element numerical analysis to derive the compliance matrices describing robot stiffness, achieving a comprehensive robot compliance model [21]. However, this model, which includes many redundant parameters, may not suit calibration involving numerous redundant parameters. Further, Klimchik et al. investigated the static calibration issue of heavily loaded industrial robots, employing the Virtual Joint Modeling method to establish a robot compliance model [22]. Du Liang developed an approach for calibrating compliance errors in robots by statistically analyzing the effects of gravity and elastostatic forces on individual joints. Utilizing single joint rotations and laser tracking measurements, the study identifies significant compliance errors and compensated for them, markedly improving robot accuracy and operational efficiency [23]. Tepper proposed a cost- and time-efficient approach for setting up a compliance model for industrial robots, utilizing an optimal design of experiments for variance-minimal Bayesian inference of gear stiffness parameters [24].

Overall, kinematic models aim to enhance RVS geometric accuracy with various solutions already available, while compliance models lack a unified consensus among researchers. There are four approaches to the compliance analysis of robots, incorporating Finite Element Analysis [25], the Virtual Joint Model [26], the nonlinear transmission model (NTM) [27], and the Rigid–Flexible Coupling Model (RFCM) [28]. Firstly, Finite Element Analysis simulates system parts to estimate deformation errors under different configurations but struggles with predicting errors from real shapes, material properties, and manufacturing assembly. Secondly, the Virtual Joint Model introduces excessive redundant parameters through adding 6-DOF springs at the joints, complicating identification. Thirdly, NTM corrects nonlinear errors of connectors in joint spaces, typically modeling end-effector spaces with high-order harmonic functions, which cannot solve the issues of interpretability and model under-fitting. Ultimately, RFCM considers the mechanical impact of robot self-weight and loads, but cannot indicate the mass and centroid errors of robot parts effectively.

As stated above, model calibration is indispensable for a newly installed or worn RVS. This study introduces a model-based kinematic and joint compliance approach, enhancing calibration and error compensation for precise positioning upon integrating vision systems into robots. Our contributions are as follows:

(1)

We design a 3D rotary laser sensor mountable on robot grippers to create a representative RVS, propose an error measurement method based on vision measurement, and prove the ability to improve the position accuracy of the RVS.

(2)

Existing RVS modeling methods separate the robot body and the hand–eye system calibration, leading to internal error accumulation. We proposed a unified kinematic model by trimming redundant structural parameters.

(3)

We introduced a joint compliance model by combining NTM and RFCM, comprehensively considering the joint friction of connectors, external loads, and link self-weight on joint compliance. The specific optimization is as follows:

(i)

We proposed an extended NTM, using second-order Fourier functions to fit spatial errors of the terminal three joints based on the Pieper Criterion in robotics. And the under-fitting issue in NTM is addressed.

(ii)

To resolve the issue of model hyper-parameters caused by unknown external load configurations, we approximated the load’s position and direction by using the hand–eye transformation matrix of the RVS.

(iii)

Through mechanical analysis, we simplified the link model for self-weight.

To address these issues and propose kinematic and joint compliance modeling and calibration methods, this paper focuses on the establishment and parameter identification of RVS models. The rest of the paper is organized as follows: Section 2 introduces a 3D vision sensor to build the RVS, and establishes a unified kinematic model. Section 3 establishes the joint compliance model for the RVS, including an extended nonlinear transmission model and compliance models for the loads’ and links’ self-weights. Section 4 proposes a parameter identification process for kinematic and joint compliance models. Section 5 completes two accuracy verification experiments on the FANUC LR Mate 200iD robot. Section 6 concludes the paper.

2. Method Framework and Unified Kinematic Model

2.1. Overview of Kinematic and Joint Compliance Model

Figure 1 presents an overview of the robotic vision system in our study. A 3D rotary laser sensor is designed in Figure 1a, which is integrated into the gripper of a robot in Figure 1b. Moreover, Figure 2 provides a visual exposition of the methodological architecture underpinning the proposed kinematic and joint compliance model. The flowchart presents a structured sequence of operations beginning with a unified kinematic model that integrates both robot body modeling and hand–eye system calibration. This integration is critical to reducing internal error accumulation and serves as the basis for further error compensation strategies, ensuring a strong foundation for the precision of robotic movements and vision system coordination.

The subsequent step in the process involves the joint compliance model which addresses the elasticity of robot materials and corrects for deformation errors. This model accounts for joint connector elasticity, external load compliance, and the self-weight of robotic links, which are imperative for the robot’s operational accuracy under varying loads and conditions. The framework illustrates how these compliance factors are systematically incorporated into the model, providing a method to compensate for the various forces and torques affecting the robot’s joints. The symbols used in this paper are presented below.

2.2. Geometric Model of a 3D Rotary Laser Sensor

The calibration of industrial robot models requires reliable error observation methods. One mainstream approach is measuring the error of robot motion in configuration space (C-space) using 3D stereo vision sensors. Depending on the installation position of the vision sensor, there are two types: eye-in-hand and eye-to-hand. The choice between these two depends on the practical application requirements, with little difference in the underlying mathematical principles.

First, a geometric model of the rotary laser sensor is established. The camera imaging process includes both a distortion model and a geometric model. The geometric model of the camera is based on the pinhole imaging principle, mathematically expressed in Equation (1).

$$\begin{array}{l}{f}_x{\mathrm{X}}_c=\left(u-u_0\right){\mathrm{Z}}_c\\ f_y{\mathrm{Y}}_c=\left(v-v_0\right){\mathrm{Z}}_c\end{array}$$

(1)

where $f_x$ and $f_y$ represent the camera’s focal lengths on the image plane, and $u_0$ and $v_0$ are the pixel coordinates of the intersection point between the camera’s optical axis and the image plane. $\left(u,v\right)$ and $\left({\mathrm{X}}_c,{\mathrm{Y}}_c,{\mathrm{Z}}_c\right)$, respectively, express the spatial point in the image plane coordinate system and the camera coordinate system. The distortion model can be expressed as Equation (2).

$$\left\{\begin{array}{l}\stackrel{⌢}{u}=u\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6+2p_{1}v\right)+p_2\left(r^2+2u^2\right)\\ \stackrel{⌢}{v}=v\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6+2p_{2}u\right)+p_1\left(r^2+2v^2\right)\\ r=\sqrt{{\left(u-u_0\right)}^2+{\left(v-v_0\right)}^2}\end{array}\right.$$

(2)

where $\left(k_1,k_2,k_3\right)$ represents the radial distortion coefficient and the tangential distortion coefficient is $\left(p_1,p_2\right)$. The radial distortion, influenced by Snell’s Law, addresses deviations from the optical axis, where light refracts to different positions, causing magnification discrepancies between actual and ideal imaging. And the tangential distortion coefficient compensates for errors caused by the nonparallel alignment of the lens and the imaging plane.

Subsequently, the structure of the 3D rotary laser sensor is designed in Figure 1a, and its schematic representation is illustrated in Figure 3. A line laser hits the mirror and the object surface in turn. Next, the light is captured by a fixed camera. Through the rotation of the mirror, multiple laser stripes can be illustrated in the image. The reflected light planes can be expressed as Equation (3).

$${\overrightarrow{n}}^i{\left(\begin{array}{ccc}{\mathrm{X}}_c& {\mathrm{Y}}_c& {\mathrm{Z}}_c\end{array}\right)}^{\mathrm{T}}+{\left(\begin{array}{ccc}0& 0& b^i\end{array}\right)}^{\mathrm{T}}=\mathbf{0}$$

(3)

where ${\overrightarrow{n}}^i$ is the normal plane to the light plane, and $b^i$ is the bias. Then, Equations (1) and (3) are expressed in matrix form, as shown in Equation (4).

$$\left(\begin{array}{ccc}{f}_x& 0& u_0-u\\ 0& f_y& v_0-v\\ {n_x}^i& {n_y}^i& {n_z}^i\end{array}\right)\left(\begin{array}{c}{\mathrm{X}}_c\\ {\mathrm{Y}}_c\\ {\mathrm{Z}}_c\end{array}\right)+\left(\begin{array}{c}0\\ 0\\ b^i\end{array}\right)=\mathbf{0}$$

(4)

Finally, the point cloud in camera frame {Cam} can be further calculated by 3D reconstruction Equation (5).

$$\left(\begin{array}{c}{\mathrm{X}}_c\\ {\mathrm{Y}}_c\\ {\mathrm{Z}}_c\end{array}\right)={\left(\begin{array}{ccc}{f}_x& 0& u_0-u\\ 0& f_y& v_0-v\\ {n_x}^i& {n_y}^i& {n_z}^i\end{array}\right)}^{-1}\left(\begin{array}{c}0\\ 0\\ -b^i\end{array}\right)$$

(5)

2.3. Unified Kinematic Model

Methods of robot body modeling and hand–eye calibration have already been extensively studied individually. However, this fragmented approach introduces redundant kinematic parameters, leading to the accumulation of system errors.

This section builds upon the serial robot MDH (Modified Denavit–Hartenberg) model by introducing camera and user frames at the beginning and end of the robot, respectively. It further establishes a unified kinematic model for RVS that conforms to a physically expressive model with low redundancy. Then, a full parameter optimization solution is performed on the system’s kinematic model.

The initial model of the measurement system is illustrated in Figure 1b. The transformation from the camera frame {Cam} to the user frame {User} is defined as ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{C}}$, which satisfies Equation (6).

$$\mathrm{q}={}^{\mathrm{U}}\mathrm{T}_{\mathrm{C}}\mathrm{p}$$

(6)

where $\mathrm{p}, \mathrm{q}$ are the same targets described in {User} and {Cam}.

In order to separate the robotic model from the system, ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{C}}$ can be divided into three parts. In Equation (7), ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$ is the transformation matrix from the robot base frame {Base} to the {User}. ${}^{\mathrm{B}}\mathrm{T}_{\mathrm{G}}$ is the transformation matrix from the gripper frame {Gripper} to the {Base}. ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ is the transformation matrix from the {Cam} to the {Gripper}.

$${}^{\mathrm{U}}\mathrm{T}_{\mathrm{C}}={}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}{}^{\mathrm{B}}\mathrm{T}_{\mathrm{G}}{}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}={}^{\mathrm{U}}\mathrm{T}_1{\displaystyle \prod _{\mathrm{i}=1}^{\mathrm{Joints}}{}^{\mathrm{i}}\mathrm{T}_{\mathrm{i}+1}}{}^{\mathrm{Joints}+1}\mathrm{T}_{\mathrm{C}}$$

(7)

where $\mathrm{Joints}$ is the number of robotic joints.

To reduce degrees of freedom, ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$ and ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ are modeled based on the Euler angle, and ${}^{\mathrm{B}}\mathrm{T}_{\mathrm{G}}$ is constructed by the MDH model of the robot. The rotation and translation relationship is shown in Equations (8)–(10). In addition, parameters of the end joint are deleted because the 3D laser scanner is rigidly connected with the robot. Similarly, two redundant parameters near the robotic base are deleted, including ${\mathrm{\theta }}_1,{\mathrm{d}}_1$.

$${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}=\mathrm{Trans}\left({\mathrm{x}}_0,{\mathrm{y}}_0,{\mathrm{z}}_0\right){\mathrm{Rot}}_{\mathrm{z}}\left({\mathrm{r}}_0\right){\mathrm{Rot}}_{\mathrm{y}}\left({\mathrm{w}}_0\right){\mathrm{Rot}}_{\mathrm{x}}\left({\mathrm{p}}_0\right)$$

(8)

$${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}=\mathrm{Trans}\left({\mathrm{x}}_{\mathrm{e}},{\mathrm{y}}_{\mathrm{e}},{\mathrm{z}}_{\mathrm{e}}\right){\mathrm{Rot}}_{\mathrm{z}}\left({\mathrm{r}}_{\mathrm{e}}\right){\mathrm{Rot}}_{\mathrm{y}}\left({\mathrm{w}}_{\mathrm{e}}\right){\mathrm{Rot}}_{\mathrm{x}}\left({\mathrm{p}}_{\mathrm{e}}\right)$$

(9)

$${}^{\mathrm{B}}\mathrm{T}_{\mathrm{G}}=\mathrm{Trans}\left({\mathrm{a}}_1,0,0\right){\mathrm{Rot}}_{\mathrm{x}}\left({\mathrm{\alpha }}_1\right){\displaystyle {\prod }_{\mathrm{i}=2}^{\mathrm{Joints}-1}{\mathrm{Rot}}_{\mathrm{z}}\left({\mathrm{\theta }}_{\mathrm{i}}\right)\mathrm{Trans}\left({\mathrm{a}}_{\mathrm{i}},0,{\mathrm{d}}_{\mathrm{i}}\right){\mathrm{Rot}}_{\mathrm{x}}\left({\mathrm{\alpha }}_{\mathrm{i}}\right)}$$

(10)

where $\mathrm{\theta },\mathrm{d},\mathrm{a},\mathrm{\alpha }$ represent the joint angle, link offset, link distance, and twist angle, respectively. $\mathrm{x},\mathrm{y},\mathrm{z},\mathrm{r},\mathrm{w},\mathrm{p}$ denote three translations and three rotations of different directions. In summary, the established unified kinematic model includes 30 parameters $\left({\mathrm{\theta }}_1={\mathrm{d}}_1=0\right)$: ${\mathit{\eta }}_{\mathrm{Ki}}{=(\mathrm{x}}_0,{\mathrm{y}}_0,{\mathrm{z}}_0,{\mathrm{r}}_0,{\mathrm{w}}_0,{\mathrm{p}}_0\cdots {\mathrm{\theta }}_{\mathrm{i}},{\mathrm{d}}_{\mathrm{i}},{\mathrm{a}}_{\mathrm{i}},{\mathrm{\alpha }}_{\mathrm{i}},\cdots {\mathrm{x}}_{\mathrm{e}},{\mathrm{y}}_{\mathrm{e}},{\mathrm{z}}_{\mathrm{e}},{\mathrm{r}}_{\mathrm{e}},{\mathrm{w}}_{\mathrm{e}},{\mathrm{p}}_{\mathrm{e}}{)}^{\mathrm{T}},\left(\mathrm{i}=1,2,\dots ,5\right)$.

Based on ${\mathit{\eta }}_{\mathrm{Ki}}$, the kinematic coordinates of the robot’s position (defined as the origin of the frame {Cam}) can be obtained. Then, the unified kinematic model can be expressed as ${\mathrm{q}}_{\mathrm{K}\mathrm{i}}=\mathrm{q}\left({\mathit{\eta }}_{\mathrm{Ki}}\right)={}^{\mathrm{U}}\mathrm{T}_{\mathrm{C}}$.

3. Joint Compliance Model

The unified kinematic model merely describes the geometric error of RVS, but is unable to cope with the elastic deformation. In general, global compliance models for various parts of the robot are established, based on elasticity theory used for nonrigid models. However, numerous redundant parameters will be introduced, causing under-fitting in calibration. According to [28], 90% of the elastic deformation in serial robotic arms concentrates at the joint connections. Thereby, a simplified form based on Hooke’s law of joint torsional elasticity is proposed in Equation (11).

$$\mathrm{\delta }\mathbf{\theta }={\lambda }_{\mathrm{\theta }}\mathbf{\tau }$$

(11)

where ${\lambda }_{\mathrm{\theta }}$ is the compliant matrix of joint torsion, $\mathbf{\theta }$ is a vector concatenated joint angle $\mathrm{\theta }$, and $\mathbf{\tau }$ is a vector concatenated joint torque $\mathrm{\tau }$.

Analyzing the impact of each component from the perspective of joint compliance is of great significance. In quasi-static conditions, joint deformation is primarily influenced by three types of forces: frictional forces, externally applied loads, and the self-weight of the robot links. Based on the principle of superposition in mechanics, the effects originating from the torque output by each of the robot’s drive motors can be decomposed into joint frictional forces caused by connectors ${\tau }_{\mathrm{C}}$, end-effector load forces ${\tau }_{\mathrm{F}}$, and link gravitational forces ${\tau }_{\mathrm{Link}}$, as indicated in Equation (12).

$$\tau ={\displaystyle \sum _{i\in \mathrm{Joints}}{\tau }_{\mathrm{C}}^i}+{\tau }_{\mathrm{F}}+{\displaystyle \sum _{i\in \mathrm{Joints}}{\tau }_{\mathrm{Link}}^i}$$

(12)

By combining (10) and (11), the joint compliance error can be derived as Equation (13).

$$\mathrm{\delta }\mathrm{\theta }={\displaystyle \sum _{i\in \mathrm{Joints}}{\mathrm{\delta }\mathrm{\theta }}_{\mathrm{C}}^i}+{\mathrm{\delta }\mathrm{\theta }}_{\mathrm{F}}+{\displaystyle \sum _{i\in \mathrm{Joints}}{\mathrm{\delta }\mathrm{\theta }}_{\mathrm{Link}}^i}$$

(13)

The joint frictional forces, originating from the relative rotation at joint connections, are influenced by several factors including motor encoders, gear reducers, and couplings. These forces are complex and challenging to explain through linear models. Consequently, this paper introduces a nonlinear transmission model to address the angle errors caused by frictional forces. Moreover, joint angle errors resulting from end-effector load forces and the gravitational forces of the robot links can be predicted using the theory of robot statics. Based on these predictions, a linear compliance model is established.

3.1. Extended Nonlinear Transmission Model

An integrated joint module encompasses a motor, a magnetic encoder, and a harmonic reducer, where the encoder, attached to the motor’s shaft end, gauges the motor angle, and the motor shaft’s linkage to the harmonic reducer’s waveform generator facilitates output with reduced rotational speed through its flexible wheel. This setup introduces an error chain from the encoder through the harmonic reducer (as shown in Figure 4), affecting the joint module’s angular measurement precision due to both components. Manufacturing considerations for magnetic encoders include addressing the positioning inaccuracies in the magnetic plate’s pole distribution and the relative positioning of the Hall sensor and the magnetic plate, which are critical for the encoder’s signal accuracy. Imperfections such as tilt and eccentricity introduce periodic, low-frequency errors, necessitating harmonic analysis for error correction.

In addition, the harmonic reducer operates on elastic deformation principles, with a setup comprising rigid and flexible gears and a wave generator [27]. The wave generator, by driving the flexible gear and inducing deformation waves, achieves dynamic transmission through periodic elastic deformation, engaging with the fixed rigid gear. Transmission errors, including radial and run-out errors from the wave generator, geometric and motion eccentricity from gear machining and installation, and backlash between gears, manifest as periodic errors with distinct low- and high-frequency components. These characteristics justify employing a second-order Fourier analysis for detailed error assessment and mitigation in the manufacturing process [29].

Since the angular error of a joint module is caused by the elastic deformation of its components and has continuous and integrable characteristics over a cycle, the angular error of the joint module can be represented as a signal with a 2π period, approximated by the Fourier series $\mathrm{\phi }\left(\mathrm{\theta }\right)$. As shown in Equation (14), the angular error of the connector $\mathrm{\delta }{\mathrm{\theta }}_{\mathrm{C}}$, caused by the periodic change in angle, is transmitted to the robot’s tool center point (TCP) through the robot’s motion Jacobian matrix $\mathit{J}\left(\mathrm{\theta }\right)$, and can ultimately be approximated as the positioning error $\delta {\mathrm{q}}_{\mathrm{C}}$ induced by frictional forces.

$$\delta {\mathrm{q}}_{\mathrm{C}}=\mathit{J}\left(\mathrm{\theta }\right)\mathrm{\delta }{\mathrm{\theta }}_{\mathrm{C}}=\mathit{J}\left(\mathrm{\theta }\right)\mathrm{\phi }\left(\mathrm{\theta }\right)$$

(14)

In the equation, $\mathrm{\theta }$ refers to the angle value measured by the robot’s magnetic encoder. Next, following the decomposition into a Fourier series, $\mathrm{\phi }\left(\mathrm{\theta }\right)$ conforms to Equation (15).

$$\mathrm{\phi }\left(\widehat{\mathbf{\theta }}\right)={\left(\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_1\right)\hspace{1em}\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_2\right)\hspace{1em}\cdots \hspace{1em}\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_{\mathrm{Joints}}\right)\right)}^{\mathrm{T}}$$

(15)

where

$$\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_{\mathrm{j}}\right)={\mathrm{k}}_0^{\left(\mathrm{j}\right)}+{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{k}}_{\mathrm{ai}}^{\left(\mathrm{j}\right)}\cos \left(\mathrm{i}w{\widehat{\mathrm{\theta }}}_{\mathrm{j}}\right)}+{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{k}}_{\mathrm{bi}}^{\left(\mathrm{j}\right)}\sin \left(\mathrm{i}w{\widehat{\mathrm{\theta }}}_{\mathrm{j}}\right)}$$

(16)

In the equation, N represents the order of the Fourier series, k is the amplitude of the i-th order component signal, and j denotes the joint index. For simplicity, $w$ will be forced to 1.

To solve for k in Equation (16), taking a six-degree-of-freedom industrial robot as an example, it is necessary to calculate the error $\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_{\mathrm{j}}\right)$ for each joint of the robot through inverse kinematics. Since inverse kinematics involves mapping from Cartesian space to joint space, to ensure the inverse kinematics solving process adheres to ${ℝ}^3\to {ℝ}^3$, this paper restricts the three degrees of freedom in the joint space based on the Pieper Criterion (the axes of the last three joints of a six-degree-of-freedom robot always intersect at a single point, which has a minor impact on positional errors.), denoted as $\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_4\right)=\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_5\right)=\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_6\right)=0, \mathrm{N}=2$.

After simplification, $\mathrm{\phi }\left(\mathrm{\theta }\right)$ can be expressed as Equation (17). And the parameters of the extended NTM are expressed as ${\mathit{\eta }}_{\mathit{C}}=\left({\mathrm{k}}_0^{\left(1:3\right)},{\mathrm{k}}_{\mathrm{a}1}^{\left(1:3\right)},{\mathrm{k}}_{\mathrm{b}1}^{\left(1:3\right)},{\mathrm{k}}_{\mathrm{a}2}^{\left(1:3\right)},{\mathrm{k}}_{\mathrm{b}2}^{\left(1:3\right)}\right)$.

$$\begin{array}{l}\mathrm{\phi }\left({\widehat{\mathrm{\theta }}}_{1:3}\right)={\mathrm{k}}_0^{\left(1:3\right)}+{\mathrm{k}}_{\mathrm{a}1}^{\left(1:3\right)}\cos \left(\widehat{\mathrm{\theta }}\right)+{\mathrm{k}}_{\mathrm{b}1}^{\left(1:3\right)}\sin \left(\widehat{\mathrm{\theta }}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\mathrm{k}}_{\mathrm{a}2}^{\left(1:3\right)}\cos \left(2\widehat{\mathrm{\theta }}\right)+{\mathrm{k}}_{\mathrm{b}2}^{\left(1:3\right)}\sin \left(2\widehat{\mathrm{\theta }}\right)\end{array}$$

(17)

3.2. Compliance Error Caused by the External Load

After modeling the nonlinearities of the transmission system, this section focuses on modeling the linear part of joint deformation. Based on the theory of linear torsional springs, a fixed stiffness coefficient is assigned to each joint of the robot. To establish the mathematical relationship between the forces and deformations experienced by the robot’s end effector in various configurations of the robot and the external load, it is first necessary to understand the forces and deformations at each of the robot’s joints. Figure 5 shows the mechanical relationship.

In the robot’s base frame {Base}, assume that the force applied at the origin of the camera frame {Cam} is ${\left({}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}} {}^{\mathrm{B}}\mathrm{M}_{\mathrm{C}}\right)}^{\mathrm{T}}$, and the force applied at the origin of the end actuated joint frame {Gripper} is ${\left({}^{\mathrm{B}}\mathrm{F}_{\mathrm{G}} {}^{\mathrm{B}}\mathrm{M}_{\mathrm{G}}\right)}^{\mathrm{T}}$. For the six-degree-of-freedom (6-DOF) serial industrial robot in this paper, we set G = 6. The relationship between them can be solved by force-and-torque transformation ${}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}$, as shown in Equation (18).

$$\left(\begin{array}{l}{}^{\mathrm{B}}\mathrm{F}_6\\ {}^{\mathrm{B}}\mathrm{M}_6\end{array}\right)={}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}\left(\begin{array}{l}{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}\\ {}^{\mathrm{B}}\mathrm{M}_{\mathrm{C}}\end{array}\right)$$

(18)

The transformation of forces under frame conversion ${}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}$ is seen in Equation (19).

$${}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}=\left(\begin{array}{cc}{\mathbf{E}}_3& \mathbf{0}\\ \left[{}^6\mathbf{t}_{\mathrm{C}}\right]{}^6\mathrm{R}_{\mathrm{C}}+{}^6\Delta _{\mathrm{C}}& {\mathbf{E}}_3\end{array}\right)$$

(19)

where ${}^6\mathbf{t}_{\mathrm{C}}$ and ${}^6\mathrm{R}_{\mathrm{C}}$ are the translation and rotation matrices of ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ in Equation (9); specifically, ${}^6\mathrm{t}_{\mathrm{C}}={\left({\mathbf{x}}_{\mathbf{e}},{\mathbf{y}}_{\mathbf{e}},{\mathbf{z}}_{\mathbf{e}}\right)}^{\mathrm{T}}$ and ${}^6\mathrm{R}_{\mathrm{C}}={\mathrm{Rot}}_{\mathrm{z}}\left({\mathbf{r}}_{\mathbf{e}}\right){\mathrm{Rot}}_{\mathrm{y}}\left({\mathbf{w}}_{\mathbf{e}}\right){\mathrm{Rot}}_{\mathrm{x}}\left({\mathbf{p}}_{\mathbf{e}}\right)$. ${}^6\Delta _{\mathrm{C}}$ is the error matrix from the load center of gravity to the origin of {Cam}. We assume that ${}^6\Delta _{\mathrm{C}}$ is equal to zero, because it mainly affects the torque of the sixth joint with a relatively short lever arm, and has minimal impact on the other joints.

The notation $\left[\mathbf{t}\right]$ in Equation (19) represents the antisymmetric matrix of the three-dimensional vector $\mathbf{t}$, and when $\mathbf{t}={\left(\begin{array}{ccc}{\mathrm{t}}_{\mathrm{x}}& {\mathrm{t}}_{\mathrm{y}}& {\mathrm{t}}_{\mathrm{z}}\end{array}\right)}^{\mathrm{T}}$, [t] can be expressed using Equation (20).

$$\left[\mathbf{t}\right]=\left(\begin{array}{ccc}0& -{\mathrm{t}}_{\mathrm{z}}& {\mathrm{t}}_{\mathrm{y}}\\ {\mathrm{t}}_{\mathrm{z}}& 0& -{\mathrm{t}}_{\mathrm{x}}\\ -{\mathrm{t}}_{\mathrm{y}}& {\mathrm{t}}_{\mathrm{x}}& 0\end{array}\right)$$

(20)

For the high-precision positioning requirements of RVS in noncontact scenarios, the gravity of the externally applied load acts in a vertically downward direction. The force acting on the center of gravity of the externally applied load can be described in the frame {Cam} as ${\left(\begin{array}{cc}{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}& {}^{\mathrm{B}}\mathrm{M}_{\mathrm{C}}\end{array}\right)}^{\mathrm{T}}={\left(\begin{array}{cccccc}0& 0& -{\mathrm{m}}_{\mathrm{L}}\mathrm{g}& 0& 0& 0\end{array}\right)}^{\mathrm{T}}$.

Based on the definition of the robot kinematics Jacobian matrix $\mathit{J}\left(\mathrm{\theta }\right)$, and the relationship between it and the static-force Jacobian matrix ${\mathit{J}}_{\mathrm{F}}\left(\mathrm{\theta }\right)$, the errors caused by the load $\mathrm{\delta }{\mathrm{q}}_{\mathrm{F}}$ and the load torque ${\mathrm{\tau }}_{\mathrm{F}}$ can be derived, as shown in Equation (21).

$$\left\{\begin{array}{l}\mathrm{\delta }{\mathrm{q}}_{\mathrm{F}}=\mathit{J}\left(\mathrm{\theta }\right)\mathrm{\delta }{\mathrm{\theta }}_{\mathrm{F}}\\ {\mathrm{\tau }}_{\mathrm{F}}={\mathit{J}}^{\mathrm{T}}\left(\mathrm{\theta }\right)\left(\begin{array}{c}{}^{\mathrm{B}}\mathrm{F}_6\\ {}^{\mathrm{B}}\mathrm{M}_6\end{array}\right)={\mathit{J}}_{\mathrm{F}}\left(\mathrm{\theta }\right)\left(\begin{array}{c}{}^{\mathrm{B}}\mathrm{F}_6\\ {}^{\mathrm{B}}\mathrm{M}_6\end{array}\right)\end{array}\right.$$

(21)

Based on the linear torsion spring theory, the compliance coefficient corresponding to each joint is denoted as ${\mathrm{\lambda }}_{\mathrm{\theta }}$. According to the linear torque–torsion relationship, the relationship between the torsional torque of each joint and its elastic deformation can be obtained as shown in Equation (22).

$$\mathrm{\delta }\mathbf{\theta }={\mathrm{\lambda }}_{\mathrm{\theta }}\mathbf{\tau }=\left(\begin{array}{ccc}{\mathrm{\lambda }}_1& & \\ & \ddots & \\ & & {\mathrm{\lambda }}_{\mathrm{Joints}}\end{array}\right)\mathbf{\tau }$$

(22)

where $\mathrm{\delta }\mathbf{\theta }={\left(\begin{array}{cccc}\mathrm{\delta }{\mathbf{\theta }}_1& \mathrm{\delta }{\mathbf{\theta }}_2& \cdots & \mathrm{\delta }{\mathbf{\theta }}_{\mathrm{Joints}}\end{array}\right)}^{\mathrm{T}}$ represents the elastic deformation of each joint under torsional torque, ${\mathrm{\lambda }}_{\mathrm{\theta }}$ denotes the compliance coefficient of each joint, and $\mathbf{\tau }={\left(\begin{array}{cccc}{\mathrm{\tau }}_1& {\mathrm{\tau }}_2& \cdots & {\mathrm{\tau }}_{\mathrm{Joint}}\end{array}\right)}^{\mathrm{T}}$ is the torsional torque corresponding to each joint.

By combining Equations (16), (19), and (21), the pose error under the base frame {Base} of the 6-DOF industrial robot studied in this paper can be expressed as Equation (23).

$$\mathrm{\delta }{\mathrm{q}}_{\mathrm{F}}=\mathit{J}\left(\mathrm{\theta }\right){\mathrm{\lambda }}_{\mathrm{\theta }}{\mathit{J}}^{\mathrm{T}}\left(\mathrm{\theta }\right){}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}\left(\begin{array}{l}{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}\\ \mathbf{0}\end{array}\right)$$

(23)

For an RVS without external forces, the direction of the load aligns with the direction of gravity; that is, $\frac{{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}}{‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖}={\left(\begin{array}{ccc}0& 0& -1\end{array}\right)}^{\mathrm{T}}={-\mathbf{e}}_3$. By integrating the load weight $‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖$ with the compliance matrix ${\mathrm{\lambda }}_{\mathrm{\theta }}$ to form ${\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast }=‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖{\mathrm{\lambda }}_{\mathrm{\theta }}$, Equation (24) can be derived. The parameters of the external load compliance model are expressed as ${\mathit{\eta }}_{\mathbf{F}}=\left({\mathrm{\lambda }}_1^{\ast },{\mathrm{\lambda }}_2^{\ast },{\mathrm{\lambda }}_3^{\ast },{\mathrm{\lambda }}_4^{\ast },{\mathrm{\lambda }}_5^{\ast },{\mathrm{\lambda }}_6^{\ast }\right)$.

$$\begin{array}{ll}\mathrm{\delta }{\mathrm{q}}_{\mathrm{F}}& =\mathit{J}\left(\mathrm{\theta }\right)\left(‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖{\mathrm{\lambda }}_{\mathrm{\theta }}\right){\mathit{J}}^{\mathrm{T}}\left(\mathrm{\theta }\right){}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}\left(\begin{array}{c}\frac{{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}}{‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖}\\ \mathbf{0}\end{array}\right)\\ & =\mathit{J}\left(\mathrm{\theta }\right){\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast }{\mathit{J}}^{\mathrm{T}}\left(\mathrm{\theta }\right){}_{\mathrm{C}}{}^6\mathrm{H}_{\mathrm{B}}\left(\begin{array}{c}{-\mathbf{e}}_{\mathbf{3}}\\ \mathbf{0}\end{array}\right)\end{array}$$

(24)

3.3. Compliance Error Caused by Weight of Robot Link

Furthermore, the effect of the self-weight of the links on joint deformation is crucial. As the robot is in any position or posture, the joint torque generated by the self-weight of the links always exists, and the compliance error caused by this factor is always coupled with geometric parameter errors. The body structure of most industrial serial robots is similar to that of the FANUC LR Mate 200iD robot, allowing for a generalized analysis method. The centroid of the link relative to the origin of the robot’s base frame {Base} forms a cantilever structure, generating torque. For a particular joint, its compliance error accumulates unidirectionally, for example, joint 3 is affected by the self-weight of links 3, 4, 5, and 6. The torque that does not change with the robot’s posture can be represented by the kinematic model. Therefore, two scenarios can be disregarded: (1) the lever arm between the link and gravity is zero; (2) the rate of change in the lever arm between the link and gravity is zero.

As shown in Figure 5, the rotation center axis (z₁) of link 1 is parallel to the gravity vector, so the self-weight’s torque effect on the joint can be ignored. The rotation center axes (z₃, z₄) of links 2 and 3 are approximately perpendicular to the gravity vector, and their self-weight effects cannot be ignored. When link 4 rotates around its rotation center axis (z₄), the lever arm does not change, so its effect on joint 4 can be ignored. The frame origins of links 5 and 6 coincide (L₅ = 0 mm), and link 6 is rigidly connected to the load, so the self-weight effects of links 5 and 6 are already included in Equation (22). Therefore, only the deformations produced by the self-weight of links 2, 3, and 4 on joints 2 and 3 need to be considered.

Define the distance from the centroid G₂ of link ₂ to its rotation center axis (z₂) as A₂, and the distance from the centroid to the rotation center axis (z₃) of link ₃ as A₃. Consider links 3 and 4 as a whole, with their corresponding centroid as G₃. The lever arms exerted by each link on the joint are shown in

Table 1. Torque applied by the link.

Torque	Link1	Link2	Link3~Link4	Link5~Link6
${\mathrm{\tau }}_1$	0	0	0	—
${\mathrm{\tau }}_2$	—	G2A2sin(θ2)	G3L2sin(θ2) + G3A3sin(θ2 − θ3)
${\mathrm{\tau }}_3$	—	—	G3A3sin(θ2 − θ3)
${\mathrm{\tau }}_4$	—	—	0
${\mathrm{\tau }}_5$	—	—	—
${\mathrm{\tau }}_6$	—	—	—

.

By summing the torque components experienced by each joint as listed in Table 1, Equation (25) can be derived in matrix form.

$$\begin{array}{l}{\mathrm{\tau }}_{\mathrm{Link}23}=\left(\begin{array}{ccc}\sin \left({\mathrm{\theta }}_2\right)& \sin \left({\mathrm{\theta }}_2-{\mathrm{\theta }}_3\right)& 0\\ 0& 0& \sin \left({\mathrm{\theta }}_2-{\mathrm{\theta }}_3\right)\end{array}\right)\left(\begin{array}{c}{\mathrm{G}}_2{\mathrm{A}}_2+{\mathrm{G}}_3{\mathrm{L}}_2\\ {\mathrm{G}}_3{\mathrm{A}}_3\\ {\mathrm{G}}_3{\mathrm{A}}_3\end{array}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}={\mathrm{N}}_{23}\left(\mathrm{\theta }\right)\left(\begin{array}{c}{\mathrm{s}}_1\\ {\mathrm{s}}_2\\ {\mathrm{s}}_2\end{array}\right)\end{array}$$

(25)

By substituting Equation (21) into Equation (25), the end-effector position error caused by the self-weight of the links can be obtained as Equation (26).

$$\begin{array}{ll}\mathrm{\delta }{\mathrm{q}}_{\mathrm{Link}23}& ={\mathit{J}}_{23}\left(\mathrm{\theta }\right)\left(\begin{array}{cc}{\mathrm{\lambda }}_2& 0\\ 0& {\mathrm{\lambda }}_3\end{array}\right){\mathrm{N}}_{23}\left(\mathrm{\theta }\right)\left(\begin{array}{c}{\mathrm{s}}_1\\ {\mathrm{s}}_2\\ {\mathrm{s}}_2\end{array}\right)\\ & ={\mathit{J}}_{23}\left(\mathrm{\theta }\right){\mathrm{\lambda }}_{23}^{\ast }{\mathrm{N}}_{23}\left(\mathrm{\theta }\right)\left(\begin{array}{c}{{\mathrm{s}}_1}^{\ast }\\ {{\mathrm{s}}_2}^{\ast }\\ {{\mathrm{s}}_2}^{\ast }\end{array}\right)\end{array}$$

(26)

where ${\mathrm{s}}_1^{\ast }={\mathrm{s}}_1{‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖}^{-1}$, ${{\mathrm{s}}_2}^{\ast }={\mathrm{s}}_2{‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖}^{-1}$, and ${\mathrm{\lambda }}_{23}^{\ast }=‖{}^{\mathrm{B}}\mathrm{F}_{\mathrm{C}}‖\mathrm{diag}\left({\mathrm{\lambda }}_2,{\mathrm{\lambda }}_3\right)$. And the parameters of the robot link self-weight compliance model are expressed as ${\mathit{\eta }}_{\mathrm{Link}}=\left({\mathrm{s}}_1^{\ast },{\mathrm{s}}_2^{\ast }\right)$.

In summary, combining Equations (7), (17), (24), and (26), the kinematic and joint compliance model can be expressed as Equation (27) in differential form.

$$\delta \mathrm{q}=\delta {\mathrm{q}}_{\mathrm{K}\mathrm{i}}+\delta {\mathrm{q}}_{\mathrm{C}}+\mathrm{\delta }{\mathrm{q}}_{\mathrm{F}}+\mathrm{\delta }{\mathrm{q}}_{\mathrm{Link}23}$$

(27)

4. Parameter Identification

The models proposed in Section 2 and Section 3 establish the mathematical relationship between the parameters to be identified and the RVS across different poses. However, it is difficult to identify all parameters simultaneously, especially since the extended NTM in Section 3.1 has high nonlinearity. Therefore, parameter identification is required in multiple stages in Figure 6.

The process begins with a measurement dataset to calibrate the unified kinematic model as the foundation for identifying the parameters of the joint compliance model. This step is crucial as it establishes the kinematic parameters that are the bedrock for subsequent procedures. Next, the process splits into two branches. In the first branch, the inverse kinematics are computed, which leads to the determination of the residual error of joint angles. This error is then addressed through Fourier fitting, which helps to fine-tune the extended nonlinear transmission model (NTM). In parallel, the second branch is focused on calibrating the whole kinematic and joint compliance model, which integrates the kinematics of the robot with the mechanical effects of transmission, load, and the robot’s own links. Finally, an accurate model is obtained, which represents a synergy of all the calibrated parameters and models.

The parameter Jacobian matrix $\mathit{J}\left(\mathit{\eta }\right)$ is necessary for identification, based on the differential kinematics model. For a single measurement point, it includes three scalar equations, which can be represented as Equation (28).

$$\left(\begin{array}{c}\Delta {\mathrm{q}}_1\\ \Delta {\mathrm{q}}_2\\ ⋮\\ \Delta {\mathrm{q}}_{\mathrm{n}}\end{array}\right)=\left(\begin{array}{cccc}\frac{\partial {\mathrm{q}}_1}{\partial {\mathrm{\eta }}_1}& \frac{\partial {\mathrm{q}}_1}{\partial {\mathrm{\eta }}_2}& \cdots & \frac{\partial {\mathrm{q}}_1}{\partial {\mathrm{\eta }}_{\mathrm{m}}}\\ \frac{\partial {\mathrm{q}}_2}{\partial {\mathrm{\eta }}_1}& \frac{\partial {\mathrm{q}}_2}{\partial {\mathrm{\eta }}_2}& \cdots & \frac{\partial {\mathrm{q}}_2}{\partial {\mathrm{\eta }}_{\mathrm{m}}}\\ ⋮& ⋮& ⋮& ⋮\\ \frac{\partial {\mathrm{q}}_{\mathrm{n}}}{\partial {\mathrm{\eta }}_1}& \frac{\partial {\mathrm{q}}_{\mathrm{n}}}{\partial {\mathrm{\eta }}_2}& \cdots & \frac{\partial {\mathrm{q}}_{\mathrm{n}}}{\partial {\mathrm{\eta }}_{\mathrm{m}}}\end{array}\right)\left(\begin{array}{c}\Delta {\mathrm{\eta }}_1\\ {\Delta \mathrm{\eta }}_2\\ ⋮\\ {\Delta \mathrm{\eta }}_{\mathrm{m}}\end{array}\right)=\mathit{J}\left(\mathit{\eta }\right)\Delta \mathit{\eta }$$

(28)

where $\Delta \mathrm{q}$ is the position error that can be physically measured in the frame {User}, and it can be substituted by the measurement error $\Delta \mathrm{p}$ in the frame {Cam}. $\Delta \mathit{\eta }$ is the errors associated with the model parameters.

The identification of $\mathit{\eta }$ is a nonlinear estimation problem, which can be calculated by the Levenberg–Marquardt (L-M) algorithm. An iterative gradient descent algorithm is summarized as follows:

(1)

Calculate the parameter Jacobian matrix $\mathit{J}\left(\mathit{\eta }\right)$.

(2)

Calculate the update vector $\Delta {\mathit{\eta }}_{\mathrm{k}}$ of parameter:

$$\Delta {\mathit{\eta }}_{\mathrm{k}}={\left({\mathit{J}}^{\mathrm{T}}\left({\mathit{\eta }}_{\mathrm{k}}\right)\mathit{J}\left({\mathit{\eta }}_{\mathrm{k}}\right)\right)}^{-1}{\mathit{J}}^{\mathrm{T}}\left({\mathit{\eta }}_{\mathrm{k}}\right)\Delta \mathrm{q}$$

(3)

Update: $\Delta {\mathit{\eta }}_{\mathrm{k}+1}={\mathit{\eta }}_{\mathrm{k}}+\zeta \Delta {\mathit{\eta }}_{\mathrm{k}},\mathrm{k}=\mathrm{k}+1$

where $\zeta$ is the descent rate of each iteration and equals 0.005.

According to the steps in Figure 6, combined with Equation (28) and the L-M algorithm, the parameter identification is performed in three stages:

Stage 1: Calibrate the unified kinematic model and identify 30 unified kinematic parameters ${\mathit{\eta }}_{\mathbf{K}}$. After the calibration, the residual errors of each measurement point are recorded as $\left(\Delta {\mathrm{r}}_x,\Delta {\mathrm{r}}_y,\Delta {\mathrm{r}}_z\right)$.

Stage 2: Calibrate the extended NTM. First, the residual errors of the unified kinematic model are projected into the joint space, based on the numerical inverse kinematics in Equation (29). Furthermore, the parameters of the extended NTM ${\mathit{\eta }}_{\mathbf{C}}$ are identified by discrete Fourier transform.

$$\left(\begin{array}{c}\Delta {\mathrm{r}}_x\\ \Delta {\mathrm{r}}_y\\ \Delta {\mathrm{r}}_z\end{array}\right)=\left(\begin{array}{ccc}\frac{\partial {\mathrm{q}}_x}{\partial {\mathrm{\theta }}_1}& \frac{\partial {\mathrm{q}}_x}{\partial {\mathrm{\theta }}_2}& \frac{\partial {\mathrm{q}}_x}{\partial {\mathrm{\theta }}_3}\\ \frac{\partial {\mathrm{q}}_y}{\partial {\mathrm{\theta }}_1}& \frac{\partial {\mathrm{q}}_y}{\partial {\mathrm{\theta }}_2}& \frac{\partial {\mathrm{q}}_y}{\partial {\mathrm{\theta }}_3}\\ \frac{\partial {\mathrm{q}}_z}{\partial {\mathrm{\theta }}_1}& \frac{\partial {\mathrm{q}}_z}{\partial {\mathrm{\theta }}_2}& \frac{\partial {\mathrm{q}}_z}{\partial {\mathrm{\theta }}_3}\end{array}\right)\left(\begin{array}{c}{\Delta \mathrm{\theta }}_1\\ {\Delta \mathrm{\theta }}_2\\ {\Delta \mathrm{\theta }}_3\end{array}\right)={\mathit{J}}^{†}\left(\mathbf{\theta }\right)\Delta \mathbf{\theta }$$

(29)

where $\left(\Delta {\mathrm{\theta }}_1,\Delta {\mathrm{\theta }}_2,\Delta {\mathrm{\theta }}_3\right)$ are the angular residual errors of the point in the joint space (J1–J3). ${\mathit{J}}^{†}\left(\mathbf{\theta }\right)$ is the numerical Jacobian matrix of the joint angles.

Stage 3: All 53 parameters of the kinematic and joint compliance model $\mathit{\eta }={\left({\mathit{\eta }}_{\mathrm{Ki}},{\mathit{\eta }}_{\mathrm{C}},{\mathit{\eta }}_{\mathrm{F}},{\mathit{\eta }}_{\mathrm{Link}}\right)}^{\mathrm{T}}$ are identified to obtain the optimal position accuracy.

5. Calibration Experiment of Kinematic and Joint Compliance Model

To assess the improvement in system position accuracy provided by our proposed method, we first calculated the geometric parameters of the 3D rotary laser sensor based on Zhang’s calibration method, and completed the local accuracy measurement verification of the four ceramic sphere calibrators. Then, we set up two experiments to validate the position accuracy of our model, as shown in Figure 7.

The first experiment was conducted with a calibrator composed of four ceramic balls, in order to assess the overall performance of the RVS. These four balls were measured by a Coordinate Measuring Machine (Hexagon Leitz PMM-Xi) with a precision of 0.5 μm, as shown in

Table 2. Ceramic balls measured by CMM (mm).

Ceramic Ball	x	y	z	r
1	185.328	408.400	2303.870	14.999
2	240.396	377.943	2640.910	15.002
3	−196.154	289.749	2727.350	15.001
4	−246.181	344.340	2379.780	14.998

. The robot, carrying the rotating laser sensor, scanned the point cloud of the standard ceramic balls from different poses and calculated the center of the spheres. And a set of data points under the camera frame {Cam} were produced, which correspond to the four data points under the ceramic ball frame {Ceramic}.

The second experiment was conducted with a laser tracker to eliminate the influence of the 3D rotary laser sensor. We mounted the reflector of the laser tracker on the side panel of the sensor. Then, the robot was controlled to move to 7000 array positions. The array points under the laser tracker’s frame {LT} ere obtained, which correspond to the origin under the reflector’s frame {Reflector}.

5.1. Calibration of 3D Rotary Laser Sensor

In our previous work [30], the principle and feasibility of geometric feature measurement using the 3D rotary laser sensor were already validated and will not be elaborated upon here. Therefore, this section will focus on verifying the accuracy of the sensor in measuring ceramic spheres.

In the experimental setup, the 3D rotary laser sensor employed comprised an industrial camera with model number MV-CA013-20GM, offering a resolution of 1280 × 1024 pixels. The camera was capable of capturing images at a frame rate of 90 Hz, with a 16mm focal length providing a horizontal field of view (HFOV) of 21.7° and a vertical field of view (VFOV) of 17.5°. The sensor’s laser component had an output power of 50 mW at a wavelength of 405 nm. A rotating mirror, integral to the sensor, rotated at an angle of 20°, facilitating the sensor’s ability to create 120 distinct flight planes for data acquisition. The sensor’s optimal working distance was set between 115 and 135 mm, with a baseline distance of 69 mm. The system offered high resolutions, with a horizontal resolution of 0.5 mm and an even finer vertical resolution of 0.045 mm, ensuring detailed and precise data collection for the study.

As shown in Figure 8, a backlight chessboard was used to calibrate the 3D rotary sensor based on Zhang’s algorithm. The intrinsic parameters of the camera utilized in the 3D rotary laser sensor were meticulously calibrated to ensure the accuracy of the experimental data. The focal lengths in the x and y axes were determined to be $f_x$ = 3423.018 and $f_y$ = 3420.949, respectively. The principal point of the camera, which is the point on the sensor where the optical axis intersects, was located at coordinates $\left(u_0,v_0\right)$ = 605.260, 580.506. The radial distortion coefficients $\left(k_1,k_2,k_3\right)$, critical for correcting lens distortions, were found to be −0.163, 0.951, and −0.874. Additionally, the tangential distortion coefficients $\left(p_1,p_2\right)$ were measured as 0.000107 and −0.00119, which are imperative for correcting the decentering distortion in the camera lens system. According to the calibration result of 3D rotary sensor, the re-projection error of the camera is 0.06 pixels, and the average fitting error of 120 light planes is 0.004 mm.

Then, the robot moved the 3D rotary sensor to measure 200 positions around the upper part of the four ceramic balls from different directions, following the layout method in [12]. Additionally, all positions were simulated in WeldPro and distributed as evenly as possible in Figure 9. According to 372 rounds of cycle measurements of the surface, the position of the balls’ center can be optimized using the Spherical Regression Algorithm. Next, the radius error distribution is concluded in Figure 10. The RMS (Root Mean Square) of the shape accuracy is 0.0114 mm and the Std. (standard deviation) can reach 0.0049 mm.

5.2. Accuracy Verification Based on 3D Rotary Laser Sensor

From the first experiment shown in Figure 7a, we validated the position accuracy of the RVS by measuring the four ceramic spheres with the 3D rotary laser sensor in Figure 8. According to the 200 measured points of ceramic spheres from different directions, 100 random positions were used for system calibration, and the remaining 100 for evaluation.

After the parameter identification process in Section 4, the kinematic and joint compliant model calibration results of RVS were calculated, including two frame transformations listed in

Table 3. Calibration results of ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$ and ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ based on the 3D rotary laser sensor.

	Transformation	Trans (x, y, z)/mm	Rot (r, w, p)/deg
${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$	{Ceramic}→{Base}    (User1)	500.580, 298.611, 2638.660	−1.815, −170.153, 98.596
${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$	{Gripper}→{Cam}	7.096, 0.355, −348.800	0.825, −24.750, 179.425

, and the MDH parameters and joint compliance parameters in

Table 4. Calibration results based on 3D rotary laser sensor.

MDH Parameters					Joint Compliance Parameters (×10−3)
Joint	θ/deg	d/mm	a/mm	α/deg	${\mathrm{k}}_0^{\left(1:3\right)}$	0.0690, 0.250, −0.164
1	—	—	49.780	−89.973	${\mathrm{k}}_{\mathrm{a}1}^{\left(1:3\right)}$	−0.245, −0.261, −0.695
2	−90.066	−0.0206	330.341	179.937	${\mathrm{k}}_{\mathrm{a}2}^{\left(1:3\right)}$	−0.177, −0.272, −0.0487
3	0.0372	0.0106	34.969	−90.001	${\mathrm{k}}_{\mathrm{b}1}^{\left(1:3\right)}$	−0.333, −0.196, −0.610
4	0.167	−335.305	−0.0286	90.037	${\mathrm{k}}_{\mathrm{b}2}^{\left(1:3\right)}$	−0.0437, −0.199, −0.00706
5	179.720	−0.601	−0.0766	90.103	${\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast }$ [mm]	diag(−1.159, 0.0640, 0.0582, 0.183, 0.852, −0.158)
6	—	—	—	—	${\mathrm{s}}_1^{\ast },{\mathrm{s}}_2^{\ast }$ [mm]	319.836, 1833.495

.

The results depicted in Figure 11, alongside the data presented in

Table 5. Statistical comparison based on the first experiment.

Method	Model Size	RMS (mm)	Std. (mm)	Avg (mm)	Max (mm)
MDH	30	0.0773	0.0485	0.0602	0.1947
NTM	45	0.0702	0.0428	0.0557	0.1752
RFCM	39	0.0692	0.0438	0.0537	0.1755
Ours	53	0.0574	0.0350	0.0455	0.1699

, provide a quantitative and visual assessment of our calibration method’s performance. Figure 11a shows a scatter plot of position errors before and after calibration at different points, with a noticeable reduction in error after calibration, signifying an improvement in accuracy post calibration. Figure 11b illustrates a histogram of the frequency of position errors, comparing our method with three others: MDH [31], NTM [27], and RFCM [17]. Our method demonstrates fewer occurrences of higher errors, with the majority of errors congregating towards the lower end of the scale. Figure 11c is a 3D bar chart, offering a visual comparison of error frequencies for various calibration methods at different directions. Our method consistently shows lower frequency counts for larger errors, further emphasizing its precision. Figure 11d is a violin plot providing a visual summary of error distribution for each method. It reveals that our method has a tighter distribution of errors, suggesting a higher level of precision and reliability. The majority of the data points are clustered near the lower end of the error scale, and the spread of data points is narrower, indicating fewer outliers and less variation in measurement error.

Table 5 shows that our method has achieved a reduction in the Root Mean Square (RMS) error, outperforming other methods. Specifically, our method exhibits a 25.7% RMS improvement over the MDH method and surpasses the NTM and RFCM methods by 18.2% and 17.1%, respectively. When measuring with the 3D rotary laser sensor, the reductions in standard deviation and the average and maximum errors corroborate the superior calibration performance of our method, suggesting that it is not only more accurate on average but also more consistent and reliable across a range of measurements.

5.3. Accuracy Verification Based on a Laser Tracker

To eliminate the influence of measurement errors from the 3D rotary laser sensor, we used a laser tracker as the accuracy verification device, for which we designed the measurement scheme shown in Figure 7b. We affixed the ball seat of the laser tracker onto the 3D rotary laser sensor to measure positions but not attitude data. Subsequently, the joint angles from the FANUC robot register and the positions of the reflector were collected.

The automated measurement process is illustrated in Figure 12. We utilized an Industrial PC (IPC) to control and collect path data from the FANUC robot and the laser tracker (API T3). Communication with the robot’s string registers via the IPC allows for the control of the robot to execute pre-defined simulation paths and read the six angles from the robot joint encoders. When the robot moves to position Pt., it sends a signal to the IPC. A 6.0 s data-reading pause is reserved to ensure the robot stabilizes. The laser tracker is set to Stable Point Mode, automatically locating the reflector mounted on the 3D rotary laser sensor when the robot stops steadily. The advantage of this scheme is that it allows for direct evaluation of the kinematic and joint compliance model without introducing measurement errors from the 3D rotary laser sensor. Meanwhile, due to the single-point tracking lacking the reflector’s attitude, it is necessary to modify the kinematic model: the ${\mathbf{r}}_{\mathbf{e}},{\mathbf{w}}_{\mathbf{e}},{\mathbf{p}}_{\mathbf{e}}$ of ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ in Equation (9) is set to 0.

Then, the laser tracker is used to measure 7000 points in a spatial array as reference data, as shown in Figure 13. Following the layout design in references [32,33], these 7000 points are arranged in seven robot orientations, with each pose set up in a 10³ configuration. The final distribution of the measurement points across the six joint spaces of the robot is depicted in Figure 14, with the least active joint reaching an approximate range of 50°.

As a result, 6998 effective position points were measured, of which 100 random points were selected for calibrating our model. Following the calibration procedure described in Section 4, ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$ and ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ in the unified kinematic model are identified in

Table 6. Calibration results of ${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$ and ${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$ based on laser tracker.

	Transformation	Trans (x, y, z)/mm	Rot (r, w, p)/deg
${}^{\mathrm{U}}\mathrm{T}_{\mathrm{B}}$	{LT}→{Base}    (User2)	−1105.029, −955.794, −116.344	21.846, −0.521, 0.166
${}^{\mathrm{G}}\mathrm{T}_{\mathrm{C}}$	{Gripper}→{Reflector}	44.200, −60.458, 104.009	—

. Meanwhile, the identification results of the MDH and joint compliant parameters are presented in

Table 7. Calibration results based on laser tracker.

MDH Parameters					Joint Compliance Parameters (×10−3)
Joint	θ/deg	d/mm	a/mm	α/deg	${\mathrm{k}}_0^{\left(1:3\right)}$	0.158, 0.403, −0.0281
1	—	—	50.202	−89.979	${\mathrm{k}}_{\mathrm{a}1}^{\left(1:3\right)}$	−0.120, −0.0430, −0.503
2	−89.876	−0.0121	330.049	179.950	${\mathrm{k}}_{\mathrm{a}2}^{\left(1:3\right)}$	−0.0512, −0.0553, 0.144
3	−0.0293	0.2361	35.343	−89.973	${\mathrm{k}}_{\mathrm{b}1}^{\left(1:3\right)}$	−0.206, 0.0217, −0.418
4	0.720	−335.152	−0.232	90.016	${\mathrm{k}}_{\mathrm{b}2}^{\left(1:3\right)}$	0.0819, 0.0188, 0.183
5	−179.415	−0.0786	0.125	90.012	${\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast }$ [mm]	diag(−1.289, 0.0668, 0.0522, 0.138, 0.732, −0.117)
6	—	—	—	—	${\mathrm{s}}_1^{\ast },{\mathrm{s}}_2^{\ast }$ [mm]	307.383, 1841.103

.

Comparing the results of the two experiments in Table 4 and Table 7, it can be observed that there is a significant difference in the 15 parameters ${\mathrm{k}}^{\left(1:3\right)}$ related to the extended NTM, while the differences in ${\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast },{\mathrm{s}}_1^{\ast },{\mathrm{s}}_2^{\ast }$ are smaller. It is speculated that the reason for this is the smaller data volume in the second experiment, leading to larger errors in ${\mathrm{k}}^{\left(1:3\right)}$ due to Fourier analysis, and part of the transmission model error being transmitted to ${\mathrm{\lambda }}_{\mathrm{\theta }}^{\ast },{\mathrm{s}}_1^{\ast },{\mathrm{s}}_2^{\ast }$.

The accuracy of the kinematic and joint compliance modeling method is evaluated based on the 6998 points. The experimental results are depicted in Figure 15, demonstrating effectiveness in enhancing position accuracy. In Figure 15a, the scatter plot contrasts position errors at various points before and after calibration. It is clear that the position error decreases significantly, demonstrating the effectiveness of the calibration process. Figure 15b is a histogram that compares the frequency of position errors between different calibration methods. The method labeled as our method appears to achieve a higher concentration of lower magnitude errors, indicating superior performance in reducing the position error when compared to MDH, NTM, and RFCM. Figure 15c presents a 3D histogram, further emphasizing the distribution of position errors in different directions, and our method exhibits a distribution skewed towards lower errors, reinforcing its efficacy. Lastly, Figure 15d showcases a violin plot for the error distribution of each method, giving insight into the density distribution of the errors. The narrow and peaked distribution of our method suggests a tighter clustering of data points around a lower median error, while other methods display broader distributions, indicative of a wider range of error magnitudes. Overall, the data across these visuals collectively suggest that our model consistently outperforms the alternative methods in minimizing position errors.

According to

Table 8. Statistical comparison based on the second experiment.

Method	RMS (mm)	Std. (mm)	Avg (mm)	Max (mm)
MDH	0.0473	0.0282	0.0380	0.1319
NTM	0.0446	0.0266	0.0358	0.1132
RFCM	0.0424	0.0255	0.0339	0.1099
Ours	0.0381	0.0255	0.0308	0.0955

, our proposed method achieved a reduction in RMS of 19.5% compared to the MDH method, and reductions of 13.8% and 9.1% compared to the NTM and RFCM, respectively. Furthermore, it exhibits optimal performance in terms of the distribution of errors and extreme values.

6. Discussion and Conclusions

In this paper, it is evident that the kinematic and joint compliance modeling method effectively enhances the position accuracy of robotic vision systems. Through an innovative integration of a 3D rotary laser sensor, unified kinematic modeling, joint compliance modeling, and a three-stage parameter identification process, this research offers a comprehensive solution to the challenges of system position accuracy. The two validation experiments—employing both a laser tracker and a 3D vision measurement system to assess the accuracy of the proposed models—robustly demonstrate the method’s superiority over existing models. Compared to the Modified Denavit–Hartenberg (MDH), nonlinear transmission model (NTM), and Rigid–Flexible Coupling Model (RFCM) methods, our approach proves its efficacy in enhancing the position accuracy of robotic vision system. The following discussion reviews the key aspects of the proposed model, evaluates its complexity related to previous methods, and emphasizes its potential benefits for industrial manufacturing cycles that require the highest precision.

The integration of kinematic and joint compliance factors introduces an increase in model complexity. This complexity is a direct consequence of our comprehensive approach to modeling, which accounts for factors often overlooked in simpler models, such as the elasticity of robotic joints, and the impact of external loads and links. While the proposed model demands more significant computational resources and a complex calibration process, it delivers substantial improvements in position accuracy by at least 9.1% according to the experiment results. These improvements are critical in applications where precision is paramount, outweighing the drawbacks associated with increased model complexity.

The practical implications of our model are particularly significant in high-precision industrial manufacturing cycles. The advanced accuracy offered by our modeling approach can lead to remarkable enhancements in product quality and a notable reduction in waste and rework. In industries such as industrial manufacturing, automotive assembly, and aerospace engineering, where the cost of inaccuracies can be exceptionally high, the potential savings and efficiency gains are substantial. Furthermore, the adaptability of our model to various robotic systems and its scalability across different manufacturing tasks underscore its versatility and broad applicability.

Future research will focus on streamlining the model’s complexity and enhancing its usability, striving to simplify the calibration process and reduce computational requirements without compromising the accuracy benefits. Furthermore, exploring the model’s application across a broader range of industrial scenarios will be crucial in fully realizing its potential to revolutionize precision in robotic automation.