2.1. Problem Statement
The typical robot measurement system is shown in
Figure 1, with a 3D camera installed at the robot end-effector. In this system, several coordinate systems are defined. The robot base coordinate system is denoted as {B}, the robot end coordinate system as {E}, the camera coordinate system as {C}, and the workpiece coordinate system as {W}, which has the same pose as {B}, as shown in
Figure 1. The transformation relationships between these coordinate systems are represented by homogeneous matrices. For example,
represents the pose of {E} in {B}, or the transformation from {B} to {E}. During the measurement procedure, the point cloud
in {C} needs to be transformed into {B} using
. The errors in
,
, and
are the robot, hand–eye matrix, and camera errors mentioned in
Section 1. According to
, there are also error in
, which is referred to as compensative error in this paper, and the accuracy of
determines the accuracy of the measurement system.
This paper focuses on a joint calibration method for the robot, hand–eye matrix, and camera errors. Separate error models and the joint error model are first proposed. Then, the calibration problem is transformed into a stepwise optimization problem, and analytical solutions for each calibration parameter are derived.
2.2. Joint Calibration Method
The principle of the joint calibration method is depicted in
Figure 2. A ceramic standard sphere is utilized as the calibrator and fixed at an appropriate position. The calibrator coordinate system is also donated as {W}, and the sphere center as W. Local point clouds of the sphere are collected from various sampling poses, and then spherical fitting is performed to obtain
, which represents the position of the sphere center W in {C}. As the relative position between the calibrator and robot is fixed,
should always be equal to its ideal value
. However,
occurs due to errors in
,
, and
. The joint calibration method is achieved by calibrating
, and the specific principle is as follows.
Firstly, the robot error is modeled.
can be represented as
using the modified Denavit–Hartenberg (MDH) method [
28], where
is the transformation matrix from the
th joint coordinate system to the
ith joint coordinate system. The transformation matrix
is calibrated by right-multiplying the calibration matrix
.
can be represented by Lie algebra
as
, where
. The symbol ∧ denotes the skew-symmetric matrix, as shown in Equation (
1), and the expression for
is
. Then, the calibration value
of
can be obtained as shown in Equation (
2).
The hand–eye matrix
is likewise calibrated using the calibration matrix
and Lie algebra
, as shown in Equation (
3).
Finally, the camera error is modeled. The measurement principle of the 3D camera is illustrated in
Figure 3. The structured light is projected onto the object, and two cameras simultaneously capture patterns on the object’s surface. Then, x-axis and z-axis coordinates are obtained based on the similar triangle relationship in the XOZ plane, followed by the computation of the y-axis coordinate through the geometric relationship. The coordinate accuracy depends on the accuracy of the baseline
B, tilt angles
, focal lengths
, and image resolutions
. However, these parameters are coupled, leading to a complex error form that is difficult to model and identify. To simplify, we will establish an error model that relates the error to the measurement value. In addition, the error model should consider the following characteristics based on the measurement principle and existing studies [
21,
22,
29,
30]. (1) The error increases as the measurement point moves away from point S, which is the intersection point of the two cameras’ lines of sight. This indicates the presence of a distortion field around S, as illustrated in
Figure 3. (2) The error in the z-axis coordinate is generally larger than those of the x-axis and y-axis coordinates. (3) Different cameras may exhibit various forms of error.
The Surface HD50 camera is used in this study, with a repeatability accuracy of ±0.15 mm, an optimal working distance of 500 ± 250 mm, and an FOV of H55° × V36°. To observe the camera’s error distribution, a ceramic standard sphere with a diameter of 38.1 mm and a diameter deviation of no more than 1 µm is placed within the camera’s 250–400 mm working distance. The fitted diameter is compared with the theoretical diameter to observe the camera’s error. This process is illustrated in
Figure 4.
Figure 5 displays the fitted diameter of the measurement points, where the horizontal coordinate represents the positions of the measurement points and the vertical coordinate represents the fitted diameter. Several patterns can be observed in
Figure 5. (1) The fitted diameter of all measurement points is larger than the theoretical diameter. (2) The fitted diameter decreases obviously as the z-axis coordinate increases. (3) The fitted diameter tends to decrease as the x-axis coordinate increases, followed by a slight increase. (4) The fitted diameter is not related to the y-axis coordinate. These patterns suggest the existence of a pillow error at point S, causing the fitted diameter to increase as the measurement point moves away from S, as illustrated in
Figure 3. Referring to the pillow distortion of color cameras, the error model of the 3D camera is defined as Equation (
4). This expression takes into account the measurement principle of the camera, which states that the x-axis and z-axis coordinates are independent of the y-axis coordinate.
In Equation (
4),
,
, and
represent the calibration value, the measurement value, and the coordinate of point S, respectively.
,
, and
are the calibration parameters, whereas
i,
j,
l, and
m are positive integers. The coordinate of point S for the used camera is theoretically
. Based on the second error characteristic of the camera, the errors in the x-axis and y-axis coordinates can be neglected. Furthermore,
is set to simplify the expression, and thus, the error model can be expressed as Equation (
5), where
.
The calibration value
, as shown in Equation (
6), can be obtained by combining these separate error models. Theoretically,
should be equal to
.
can be obtained from
m sampling poses, and Equation (
7) is obtained by taking pairwise differences between
.
Since the relative position of the calibrator and robot is fixed, the equations in Equation (
7) should theoretically be equal to 0. Equation (
7) is then transformed into an optimization problem, as shown in Equation (
8), where
is to be solved.
,
, and
are independent of each other, allowing them to be solved step by step. The process of solving
is as follows. Let
and
, and define
to update
in each iteration, with the update expression given by
. Then, Equation (
8) can be transformed into Equation (
9).
Define
as
. Since
is small,
can be simplified using the first-order Taylor expansion around
, yielding
. Equation (
10) is obtained by substituting the simplified
into Equation (
9).
Define
,
, and
can be obtained as Equation (
11) by solving the partial derivative
.
and
can be derived using the definition of the derivative, as shown in Equations (
12) and (
13), where the symbol ∧ denotes the skew-symmetric matrix.
R and
t represent the rotation matrix and translation vector of
.
Then, the hand–eye matrix calibration parameter
is derived. By defining
and
, and using
to update
, the optimization problem
can be formulated.
can be obtained following the similar derivation as Equations (
9)–(
13).
Finally,
is derived. Let
and
, and define
to update
, with the expression given by
. Then, Equation (
8) can be transformed into Equation (
14).
can be obtained as Equation (
15) by solving the partial derivative
.
In each iteration, the calibration parameter
x is calculated, and the objective function value
is updated by substituting
x into Equation (
8). The iterative process continues until
no longer decreases or the maximum number of iterations
L has been reached. Algorithm 1 presents the pseudo-code for the iterative process. The initial
is obtained using a calibration method described in Reference [
26].
Algorithm 1: The pseudo-code of calibration process |
|