A Multi-Camera Rig with Non-Overlapping Views for Dynamic Six-Degree-of-Freedom Measurement

Abstract

Large-scale measurement plays an increasingly important role in intelligent manufacturing. However, existing instruments have problems with immersive experiences. In this paper, an immersive positioning and measuring method based on augmented reality is introduced. An inside-out vision measurement approach using a multi-camera rig with non-overlapping views is presented for dynamic six-degree-of-freedom measurement. By using active LED markers, a flexible and robust solution is delivered to deal with complex manufacturing sites. The space resection adjustment principle is addressed and measurement errors are simulated. The improved Nearest Neighbor method is employed for feature correspondence. The proposed tracking method is verified by experiments and results with good performance are obtained.

1. Introduction

In recent years, there has been growing interest in intelligent manufacturing [1] of large-scale equipment, such as airplane assembly [2], shipbuilding [3], and spacecraft inspection [4]. As one of the key technologies in intelligent manufacturing, large-scale measurement [5] plays a crucial part in the improvement of product quality and working efficiency. Large-scale measuring instruments are expected to provide adaptive and flexible services to end-users and enable a highly integrated human-machine manufacturing system. However, popular measuring instruments like laser tracker [6], total station [7], and indoor Global Positioning System (iGPS) [8] have problems with portability and flexibility, especially in a narrow space. In practical operation, it is quite a challenging task to measure complex components with high efficiency and accuracy in a narrow ship or spacecraft cabin using any of the above instruments. Besides, operating personnel have no access to real-time visual measuring results due to the lack of interaction with measuring instruments, which makes it more difficult for them to get involved into measurement environment. Augmented Reality (AR) [9] is a novel human-machine interaction tool that combines virtual objects with real environment in a seamless way, thus offering an effective solution to large-scale measurement.

On this background, an immersive human-machine-environment interactive positioning and measuring method is proposed. By the integration of global positioning and local measuring, three-dimensional coordinates of the measured objects can be obtained in the global coordinate system. Then, based on AR, the measuring results and auxiliary information are accurately overlaid onto the measured object in real time using a projector, which enhances the user’s interactive and immersive experiences. With the immersive positioning and measuring helmet (see Figure 1), operating personnel are able to free their hands to carry out assembly and inspection work. The whole system possesses the features of high integration, excellent portability, and powerful functionality. Therefore, the immersive positioning and measuring method gives a huge boost to working efficiency with AR-assisted guidance, and it also represents the developing trend of large-scale measurement in intelligent manufacturing. In order to obtain accurate measuring results and merge virtual information with the real object perfectly, high-accuracy global positioning and tracking method is required [10]. The major task of tracking is to determine the positions and orientations of the helmet in real time, that is, dynamic six-degree-of-freedom (6-DOF) measurement [11].

A number of alternative technologies have been proposed for indoor positioning [12], such as magnetic [13], inertial [14], ultrasound [15], and vision [16]. However, complex working environment, portability, and accuracy requirement pose challenges to these methods. With small operating range, magnetic measurement is prone to distortion. Inertial measurement is of poor accuracy due to the error accumulation with time. As for ultrasound, it is severely affected by obstacles, so it does not apply to manufacturing sites. Compared with the above methods, vision measurement can realize pixel accuracy and large-scale multi-target tracking with excellent flexibility and convenience, which shows great advantages in industrial manufacturing.

The vision-based 6-DOF measurement methods can be classified into two categories: outside-in measurement [17] and inside-out measurement [18]. As for outside-in measurement, cameras are installed in the working environment and markers are fixed on the moving target. Images of markers are taken by the cameras to calculate positions and orientations of the target. The OptiTrack system [19] developed by NaturalPoint is one of the representative outside-in systems, and it produces positional error less than 0.3 mm and rotational error less than 0.05°. However, it is costly to install multiple cameras in large-scale environments, which also bring difficulties to realize exact synchronization. By contrast, inside-out measurement uses cameras mounted on the tracked object to take images of markers in the working environment, which makes it more flexible and easier to extend. As the research focus of robot autonomous navigation, Simultaneous Localization and Mapping (SLAM) [20] relies on sequences of images to recognize the robot’s location and surrounding environment. But the computational load for image correspondence is particularly high, and this view-based approach can hardly meet the accuracy requirements. In order to improve accuracy of reference points, retro reflective targets [21] are used for indoor positioning. Nonetheless, these systems lack of robustness, especially under conditions with varying illumination. Thus, active markers also have been utilized, and HiBall tracking system using LED panels is one of the most successful systems. The HiBall tracking system [22] achieves 0.5 mm and 0.03° of absolute error in a 4.5 m × 8.5 m room. However, it is quite difficult to install the LED panels at industrial spots, hindering the application of this system.

In order to better meet the needs of dynamic 6-DOF measurement for immersive positioning and measuring in manufacturing sites, this paper presents an inside-out measuring method using a multi-camera rig with non-overlapping views. The multi-camera rig is mounted on the integrated helmet for global tracking, and it is effective to increase the field of view as well as reduce the impacts of vision occlusion. Taking images of the cooperative LED markers that are deployed in the surrounding environment, positions and orientations of the helmet are determined through a collinearity equation based space resection adjustment method. As the LED markers are interchangeable with 38.1 mm spherical targets, their three-dimensional coordinates can be accurately obtained with laser tracker or industrial photogrammetry system. Furthermore, a motion information combined Nearest Neighbor (NN) method [23] is adopted to implement the matching of image points and LED markers under dynamic conditions.

The remaining of this paper is structured as follows: Section 2 describes configuration of the multi-camera rig and design of the cooperative object; Section 3 presents the dynamic 6-DOF measurement principle, including space resection adjustment method and feature correspondence method; measurement error is simulated in Section 4, while experiments are carried out in Section 5; finally, conclusions of the work are provided in Section 6.

2. System Hardware

As depicted in Figure 2a, the multi-camera rig consists of one control circuit board and three compact CMOS cameras, which are mounted on a 3D-printed connector with good rigidity. Each camera is able to provide 1280 pixel × 960 pixel resolution with frame rate up to 54 Hz, and the size of unit pixel is 3.75 µm × 3.75 µm. As the field of view of each camera is 34.5° with a 6 mm lens, the angle between two neighboring cameras is set as 35°. Based on this design, there are none overlapping views between two neighboring cameras, so the multi-camera rig can cover larger visible range to avoid vision occlusion. The control circuit board is programmed to synchronize clocks as well as gather images, and it also transmits data to the computer via Ethernet. Consequently, the multi-camera rig is well-positioned to realize global tracking with light weight and high reliability.

The active LED markers are designed as control points to deal with complicated industrial environment. A red LED with a wavelength of 660 nm is installed at the center of a spherical target (see Figure 2b). The LED marker is aligned accurately using TESA-VISIO 300 video measuring machine. Compared with passive markers, active LED markers are less sensitive to illumination, and they can also provide optimal contrast and sharp edges. Moreover, this active LED marker is interchangeable with the 38.1 mm spherically mounted retroflector (SMR) of laser tracker.

3. Dynamic 6-DOF Measurement Principle

3.1. Camera Model

On the basis of pinhole model, a more complex camera model is introduced for high-accuracy vision measurement, including principal point offset and lens distortion. For convenience, the symmetrical plane of image plane is analyzed. As shown in Figure 3, a spatial object point $P(X_p,Y_p,Z_p)$ is projected at $p(x_p,y_p)$ on the image plane through the perspective center $O_c$.

In the camera coordinate system ($O_c$-$X_c{Y}_c{Z}_c$), $X_c$ axis and $Y_c$ axis are parallel to x axis and y axis of the image coordinate system respectively, and $Z_c$ axis is along the optical axis. On account of lens installation errors, there exists an offset between the principal point $(x_0,y_0)$ and the center of the image O. Hence, image points coordinates after principal point correction are expressed as:

$$\left\{\begin{array}{cc}\hfill x_c& =x_p-x_0=(u_p-u_0)dx,\hfill \\ \hfill y_c& =y_p-y_0=(v_p-v_0)dy.\hfill \end{array}\right.$$

(1)

Here, $(u_p,v_p)$ and $(u_0,v_0)$ stand for pixel coordinates of point p and principal point respectively, and $(dx,dy)$ are pixel separations. Besides radial lens distortion and tangential lens distortion, affine and non-orthogonality deformations also cause image point offset. The distortion is generalized into Equation (2):

$$\left\{\begin{array}{cc}\hfill \Delta x& =x_c{r}^2{k}_1+x_c{r}^4{k}_2+x_c{r}^6{k}_3+(r^2+2x_c^2)p_1+2x_c{y}_c{p}_2+x_c{b}_1+y_c{b}_2,\hfill \\ \hfill \Delta y& =y_c{r}^2{k}_1+y_c{r}^4{k}_2+y_c{r}^6{k}_3+2x_c{y}_c{p}_1+(r^2+2y_c^2)p_2+y_c{b}_1,\hfill \end{array}\right.$$

(2)

where $(\Delta x,\Delta y)$ denote the correction values for errors in the image plane, $x_c=x_p-x_0$ and $y_c=y_p-y_0$ stand for image point coordinates after principal point correction, $r=\sqrt{x_c^2+y_c^2}$ refers to the radical distance from image point to optical axis, $(k_1,k_2,k_3)$ represent radial distortion coefficients (generally considering the first three radial distortion coefficients), $(p_1,p_2)$ stand for tangential distortion coefficients (generally ignoring the third tangential distortion), and $(b_1,b_2)$ refer to affine and non-orthogonality coefficients.

$$\left\{\begin{array}{cc}\hfill x_p& =x_0+\Delta x+c\frac{r_1(X_p-X_0)+r_2(Y_p-Y_0)+r_3(Z_p-Z_0)}{r_7(X_p-X_0)+r_8(Y_p-Y_0)+r_9(Z_p-Z_0)},\hfill \\ \hfill y_p& =y_0+\Delta y+c\frac{r_4(X_p-X_0)+r_5(Y_p-Y_0)+r_6(Z_p-Z_0)}{r_7(X_p-X_0)+r_8(Y_p-Y_0)+r_9(Z_p-Z_0)},\hfill \end{array}\right.$$

(3)

The collinearity equations are given by Equation (3) based on the camera model above, where c represents the principal distance, ${\mathit{X}}_0={[X_0,Y_0,Z_0]}^{\mathrm{T}}$ are coordinates of perspective center in object coordinate system, and $\mathit{R}$ defines rotation of object coordinates into image coordinates by three independent rotation angles $\theta ,\phi ,\kappa$ about axes $X_c,Y_c,Z_c$ in Equation (4). Thus collinearity equations offer functions of six degrees of freedom $(X_0,Y_0,Z_0,\theta ,\phi ,\kappa )$ of the camera.

$$\begin{array}{cc}\hfill \mathit{R}& =\left[\begin{array}{ccc}{r}_1& r_2& r_3\\ r_4& r_5& r_6\\ r_7& r_8& r_9\end{array}\right]\hfill \\ & =\left[\begin{array}{ccc}\cos \phi \cos \kappa & -\cos \phi \sin \kappa & \sin \phi \\ \cos \theta \sin \kappa +\sin \theta \sin \phi \cos \kappa & \cos \theta \cos \kappa -\sin \theta \sin \phi \sin \kappa & -\sin \theta \cos \phi \\ \sin \theta \sin \kappa -\cos \theta \sin \phi \cos \kappa & \sin \theta \cos \kappa +\cos \theta \sin \phi \sin \kappa & \cos \theta \cos \phi \end{array}\right].\hfill \end{array}$$

(4)

3.2. Feature Points Extraction Method

Using active LED markers as feature points, images with high contrast are acquired. The facula of LED marker is shown in Figure 4. Under this condition, the squared centroid method is adopted for sub-pixel image processing, which achieves high extraction accuracy. The squared centroid sets the gray value squared as the weight in the processing window:

$$x_m=\frac{{\displaystyle \sum _{ij}}x{f}^2(x,y)}{{\displaystyle \sum _{ij}}{f}^2(x,y)},y_m=\frac{{\displaystyle \sum _{ij}}y{f}^2(x,y)}{{\displaystyle \sum _{ij}}{f}^2(x,y)}.$$

(5)

Here $(x_m,y_m)$ refer to the coordinates of centroid, $f(x,y)$ is the gray value at the pixel position $(x,y)$. Squared centroid method is computationally fast and easy to implement.

3.3. Space Resection Adjustment Method

In practical measurement, a set of LED markers are deployed in the environment, as shown in Figure 5. The three-dimensional coordinates of each LED marker in the global object coordinate system O-$XYZ$ are obtained. The parameters of interior orientation and spatial relationships between three cameras are also calibrated in advance,

$$\left\{\begin{array}{cc}\hfill {\mathit{X}}_1& ={\mathit{R}}_{21}{\mathit{X}}_2+{\mathit{T}}_{21},\hfill \\ \hfill {\mathit{X}}_3& ={\mathit{R}}_{23}{\mathit{X}}_2+{\mathit{T}}_{23}.\hfill \end{array}\right.$$

(6)

In Equation (6), $({\mathit{R}}_{21},{\mathit{T}}_{21})$ and $({\mathit{R}}_{23},{\mathit{T}}_{23})$ are rotation matrices and translations matrices from coordinates ${\mathit{X}}_2$ in $O_2$-$X_2{Y}_2{Z}_2$ to coordinates ${\mathit{X}}_1$ in $O_1$-$X_1{Y}_1{Z}_1$ and ${\mathit{X}}_3$ in $O_3$-$X_3{Y}_3{Z}_3$ respectively. Meanwhile, we assume the coordinate system of multi-camera rig $O_s$-$X_s{Y}_s{Z}_s$ is identical with the coordinate system $O_2$-$X_2{Y}_2{Z}_2$ of camera-2. Therefore, we can establish the reprojection error equations for each visible LED marker on the basis of collinearity equations:

$$\left\{\begin{array}{cc}\hfill e_{xj}=& x_j-x_{i0}-\Delta x_j-c_i\frac{r_{i1}(X_j-X_{i0})+r_{i2}(Y_j-Y_{i0})+r_{i3}(Z_j-Z_{i0})}{r_{i7}(X_j-X_{i0})+r_{i8}(Y_j-Y_{i0})+r_{i9}(Z_j-Z_{i0})},\hfill \\ \hfill e_{yj}=& y_j-y_{i0}-\Delta y_j-c_i\frac{r_{i4}(X_j-X_{i0})+r_{i5}(Y_j-Y_{i0})+r_{i6}(Z_j-Z_{i0})}{r_{i7}(X_j-X_{i0})+r_{i8}(Y_j-Y_{i0})+r_{i9}(Z_j-Z_{i0})}.\hfill \end{array}\right.$$

(7)

Here j is the serial number of visible LED markers and $i(i=1,2,3)$ denotes the number of camera that observes j-th marker. The six degrees of freedom of camera-2 $({\mathit{R}}_2,{\mathit{X}}_{20})$ with respect to the global object coordinate system can be expressed as follows,

$${\mathit{X}}_2={\mathit{R}}_2(\mathit{X}-{\mathit{X}}_{20}).$$

(8)

Substituting Equation (8) into Equation (6), the following relations are obtained,

$$\left\{\begin{array}{cc}\hfill {\mathit{X}}_1& ={\mathit{R}}_{21}{\mathit{R}}_2(\mathit{X}-{\mathit{X}}_{20})+{\mathit{T}}_{21}={\mathit{R}}_{21}{\mathit{R}}_2(\mathit{X}-({\mathit{X}}_{20}-{\mathit{R}}_2^{-1}{\mathit{R}}_{21}^{-1}{\mathit{T}}_{21})),\hfill \\ \hfill {\mathit{X}}_3& ={\mathit{R}}_{23}{\mathit{R}}_2(\mathit{X}-{\mathit{X}}_{20})+{\mathit{T}}_{23}={\mathit{R}}_{23}{\mathit{R}}_2(\mathit{X}-({\mathit{X}}_{20}-{\mathit{R}}_2^{-1}{\mathit{R}}_{23}^{-1}{\mathit{T}}_{23})).\hfill \end{array}\right.$$

(9)

Therefore, the six degrees of freedom of camera-1 $({\mathit{R}}_1,{\mathit{X}}_{10})$ and camera-3 $({\mathit{R}}_3,{\mathit{X}}_{30})$ can be expressed by $({\mathit{R}}_2,{\mathit{X}}_{20})$,

$$\left\{\begin{array}{cc}\hfill {\mathit{R}}_1& ={\mathit{R}}_{21}{\mathit{R}}_2,\hfill \\ \hfill {\mathit{X}}_{10}& ={\mathit{X}}_{20}-{\mathit{R}}_2^{-1}{\mathit{R}}_{21}^{-1}{\mathit{T}}_{21},\hfill \\ \hfill {\mathit{R}}_3& ={\mathit{R}}_{23}{\mathit{R}}_2,\hfill \\ \hfill {\mathit{X}}_{30}& ={\mathit{X}}_{20}-{\mathit{R}}_2^{-1}{\mathit{R}}_{23}^{-1}{\mathit{T}}_{23}.\hfill \end{array}\right.$$

(10)

In addition, as an orthonormal matrix, rotation matrix ${\mathit{R}}_2$ satisfies following constraint equations:

$$\left\{\begin{array}{cc}\hfill f_1& =r_{21}^2+r_{22}^2+r_{23}^2-1=0,\hfill \\ \hfill f_2& =r_{24}^2+r_{25}^2+r_{26}^2-1=0,\hfill \\ \hfill f_3& =r_{27}^2+r_{28}^2+r_{29}^2-1=0,\hfill \\ \hfill f_4& =r_{21}{r}_{24}+r_{22}{r}_{25}+r_{23}{r}_{26}=0,\hfill \\ \hfill f_5& =r_{21}{r}_{27}+r_{22}{r}_{28}+r_{23}{r}_{29}=0,\hfill \\ \hfill f_6& =r_{24}{r}_{27}+r_{25}{r}_{28}+r_{26}{r}_{29}=0.\hfill \end{array}\right.$$

(11)

Consequently, there are only six unknown parameters, and the solution requires at least three LED markers which do not lie on a common straight line. A non-linear optimization algorithm is proposed to calculate $({\mathit{R}}_2,{\mathit{X}}_{20})$, and reprojection error based object function is established as Equation (12) using Lagrange multiplier method [24].

$$F=\sum _j^n(e_{xj}^2+e_{yj}^2)+\lambda ·\sum _{k=1}^6{f}_k^2=min,$$

(12)

where n stands for the number of visible LED markers, $\lambda$ is the Lagrange multiplier. With fast convergence rate and strong robustness, Levenberg-Marquardt (LM) algorithm [25] is employed for this optimization problem. In order to obtain global optimal solution, the initial value for optimization is calculated using EPnP algorithm [26]. Eventually, we can determine six degrees of freedom $(X_s,Y_s,Z_s,{\theta }_s,{\phi }_s,{\kappa }_s)$ of the multi-camera rig derived from ${\mathit{R}}_2$ and ${\mathit{X}}_{20}$.

3.4. Feature Points Correspondence Method

Because there are only several feature points that are almost exactly the same in one image, it is a huge challenge to match corresponding LED markers under dynamic conditions. NN method is proposed for feature matching by searching for the nearest point couples in two images, and each point couple represents the same LED marker. In order to improve robustness, motion information of the multi-camera rig is combined. The motion state vector of the multi-camera rig at time $t_k$ is ${\mathit{S}}_k=[X_k,Y_k,Z_k,{\nu }_{xk},{\nu }_{yk},{\nu }_{zk},{\theta }_k,{\phi }_k,{\kappa }_k,{\omega }_{xk},{\omega }_{yk},{\omega }_{zk}]$, where $({\nu }_{xk},{\nu }_{yk},{\nu }_{zk})$ and $({\omega }_{xk},{\omega }_{yk},{\omega }_{zk})$ denote velocities and angular velocities respectively. Considering that users’ movements are normally slow, the multi-camera rig is assumed to move with constant velocities in the time $\Delta t$ between two adjacent frames. Hence, the state of the multi-camera rig ${\mathit{S}}_k$ can be predicted by ${\mathit{S}}_{k-1}$ as follows:

$$\left\{\begin{array}{cc}\hfill X_k& =X_{k-1}+{\nu }_{xk-1}\Delta t,\hfill \\ \hfill Y_k& =Y_{k-1}+{\nu }_{yk-1}\Delta t,\hfill \\ \hfill Z_k& =Z_{k-1}+{\nu }_{zk-1}\Delta t,\hfill \\ \hfill {\theta }_k& ={\theta }_{k-1}+{\omega }_{xk-1}\Delta t,\hfill \\ \hfill {\phi }_k& ={\phi }_{k-1}+{\omega }_{yk-1}\Delta t,\hfill \\ \hfill {\kappa }_k& ={\kappa }_{k-1}+{\omega }_{zk-1}\Delta t.\hfill \end{array}\right.$$

(13)

Then the positions and orientations of three cameras can also be predicted. Based on Equation (3), we project LED markers onto image plane and calculate image coordinates of these predicted feature points. Next, we find the nearest point couples between predicted image and real image using NN method, where the distance between two image points is defined as:

$$d=\sqrt{{(x_i-x_j)}^2+{(y_i-y_j)}^2}.$$

(14)

In order to avoid mismatching caused by occlusion and image noise, the ratio of the shortest distance to the second-shortest distance is validated. Furthermore, the reciprocity check is employed to remove outliers. Thus the following steps need to be performed for feature points matching.

• As for a point $P_{r,i}$ on the real image, we calculate the distances from $P_{r,i}$ to all the points on the predicted image and select its nearest neighbor $P_{p,j}$ that has the shortest distance. If the ratio of the shortest distance to the second-shortest distance is less than the threshold $\lambda$, we continue to the next step. If not, we remove the point $P_{r,i}$ as an outlier.
• We calculate the distances from $P_{p,j}$ to all the points on the real image. Then we check whether $P_{r,i}$ has the shortest distance, and whether the ratio of the shortest distance to the second-shortest distance is less than $\lambda$. When both criteria are fulfilled, the nearest point couple ($P_{r,i}$, $P_{p,j}$) are proved to be correct.
• By repeating the above process, we complete the feature points matching (see Figure 6). The value of $\lambda$ is set based on the deployment of LED markers.

3.5. Dynamic Measurement Process

In order to accomplish continuous tracking, the system initialization has to be performed. In the initialization, the multi-camera rig remains stationary and feature points correspondence is completed manually. Once the initial state is determined, 6-DOF of the rig can be calculated real-timely in the measurement field. The complete measurement process is shown in Figure 7.

4. Measurement Error Simulation

On the basis of space resection adjustment method given in the previous section, measurement errors mainly arise from calibration errors of interior orientation parameters, calibration errors of spatial relationship parameters and position errors of LED markers including machining errors and measuring errors. Although measurement accuracy also depends on the focal length, the number and distribution of LED markers [27], they are not discussed in this paper.

By using Monte Carlo simulation technique, the 6-DOF measurement errors are analyzed. The deployment of the multi-camera rig and 15 markers is shown in Figure 8, and in this setup each camera observes 5 non-planar markers. The parameters of the multi-camera rig are set based on the system hardware design described in Section 2. After adding normally distributed noises, the root mean square (RMS) errors between the simulated values of 6-DOF and the true values are calculated. The following simulations are conducted to study the impacts of the above factors on the measurement errors. The sample size is set as ${10}^4$ in each simulation.

Firstly, the impact of the marker position error is studied. The position noises obeying normal distribution are added to each marker in three axes. The standard deviation varies from 0 mm to 0.5 mm. As shown in Figure 9, the RMS error of 6-DOF increases linearly with the marker position error. When a realistic error of 0.2 mm is assumed for the markers, the three-dimensional position of the multi-camera rig is computed to an accuracy of about 0.5 mm.

Since the calibration errors of interior parameters are directly reflected in the errors of image points, we add image point noises that follow normal distribution. For the simulation, the noise is altered in the range of 0.2 pixel, corresponding to 0.75 µm. Figure 10 illustrates a linear relationship between 6-DOF measurement error and image point error. With an image point error of 0.1 pixel or 0.375 µm, the angular error is less than 0.01°.

As part of spatial relationship error, the rotation error between cameras is added to evaluate its influence on 6-DOF measurement. The noise level is changed from 0° to 0.01° and the corresponding measurement error is depicted in Figure 11. There is a clear linear trend for all six degrees of freedom. It can also be observed that the rotation angle about X axis is computed with better robustness than the other two rotation angles.

Finally, a variation of the translation error between cameras is investigated. The noise of relative position is varied within 0.5 mm. As seen from Figure 12, the 6-DOF measurement error indicates again a linear relationship to the translation calibration error [28].

Furthermore, another simulation is carried out to compare the measurement accuracy of a single camera and the multi-camera rig in the same setup above (see Figure 8). With a focal length of 2.4 mm, the single camera covers almost as wide view as the multi-camera rig to observe all 15 markers. Then the normally distributed noises including the marker position noise (0.1 mm), image point noise (0.1 pixel) and spatial relationship noise (only for the multi-camera rig) are added to simulate the 6-DOF measurement error. The spatial relationship noise is composed of the rotation noise (0.005°) and the translation noise (0.1 mm) between cameras, which are typical values for the calibration of non-overlapping cameras. The result in

Table 1. Measurement accuracy comparison between the single camera and the multi-camera rig.

Parameter	The Single Camera	The Multi-Camera Rig
${\theta }_s$ (°)	0.0147	0.0087
${\phi }_s$ (°)	0.0067	0.0084
${\kappa }_s$ (°)	0.0038	0.0065
$X_s$ (mm)	0.715	0.629
$Y_s$ (mm)	1.191	0.754
$Z_s$ (mm)	0.292	0.528

shows that the multi-camera rig gives a higher accuracy than the single camera.

5. Experiment

Before the experiments, the multi-camera rig is fixed to a helmet. Interior orientation parameters and spatial relationship parameters of three cameras are calibrated in a large-scale spatial photogrammetric test field. Then the following experiments are conducted to evaluate the performance of the proposed method.

5.1. Static Measurement Experiment

The static measurement experiment is conducted in a 5 m × 5 m × 3 m measurement field (see Figure 13). Ten LED markers are deployed, and their spatial coordinates are measured using Leica AT901 laser tracker.

5.1.1. Measurement Repeatability

The helmet is randomly placed at ten different positions in the measurement field, and five images are captured at each position. Feature points are all extracted and the pixel coordinate repeatability of each point is shown in Figure 14. From the results, we can observe that the extraction precision of feature points in either axis is better than 0.01 pixel.

Then 6-DOF $(X_0,Y_0,Z_0,\theta ,\phi ,\kappa )$ of the multi-camera rig are calculated, and 6-DOF measurement repeatability at each position is analyzed. As shown in Figure 15, standard deviations of positions along global coordinate axes are less than 0.5 mm and standard deviations of three rotations are better than 0.01°.

5.1.2. Distance Measurement

The helmet is mounted on a motorized translation stage with a long travel of 1000 mm (see Figure 13), and the straightness error of the translation stage is less than 0.02 mm. The translation stage is placed in the measurement field and set to travel 900 mm each time. The positions of the multi-camera rig are obtained before and after translation, so the travel distance D can be calculated by $D=\sqrt{{(X_a-X_b)}^2+{(Y_a-Y_b)}^2+{(Z_a-Z_b)}^2}$. In addition, a SMR is fixed on the helmet to obtain accurate travel distances as reference values using the laser tracker. Nine sets of results are acquired while the translation stage is placed at nine different positions and directions, and the measurement errors of travel distances are shown in

Table 2. Measurement error of travel distance (mm).

Position	Multi-Camera Rig	Laser Tracker	Measurement Error
1	900.567	900.330	0.237
2	900.436	900.242	0.194
3	900.064	900.247	−0.183
4	899.970	900.381	−0.411
5	899.848	900.318	−0.470
6	899.594	900.207	−0.613
7	899.968	900.347	−0.379
8	900.588	900.185	0.403
9	900.863	900.342	0.521

. Using the multi-camera rig, the RMS error of distance measurement is 0.383 mm.

5.2. Dynamic Measurement Experiment

5.2.1. Operating Speed

In order to evaluate the performance of dynamic measurement, the operating speed is tested using the C++ language in Visual Studio 2013 on a laptop with Inter(R) Core(TM) i7-6700HQ CPU at 2.60 GHz and 8 GB RAM. The test is conducted with ten LED markers and the maximum consuming time is shown in Figure 16. The consuming time of single measurement is approximately 33.9 ms, in which the feature extraction takes up 85% of the entire process.

5.2.2. 6-DOF Measurement

As shown in Figure 17, the helmet and the Leica T-Mac are both mounted on the three-axis turntable to assess the dynamic 6-DOF measurement accuracy. Based on the experiment setup, T-Mac has a 0.01° rotational accuracy and about 30 µm positional accuracy. The spatial relationship between these two devices remains constant, no matter how the turntable rotates. Ten LED markers are deployed about five meters in front of the turntable, and their three-dimensional coordinates are measured using Leica AT901 laser tracker. In consequence, positions and orientations of the helmet and the T-Mac are unified in the laser tracker coordinate system O-$XYZ$.

The turntable is set to rotate 20°, 15°, and 10° about its outer axis, middle axis and inner axis respectively at an angular velocity of 5°/s, and then it returns to the starting position. In the feature matching process, NN method is applied with the threshold $\lambda =0.3$. The multi-camera rig and T-Mac are triggered at 20 Hz synchronically, and their motion trajectories are shown in Figure 18.

A test is carried out to validate the proposed feature correspondence method. Here, the sample interval is altered to simulate different angular velocities of the turntable. As for an angular velocity of 10°/s, half of the obtained images are selected with equal interval. Then feature correspondence between adjacent pictures is performed with and without motion prediction respectively. The numbers of image mismatching at different angular velocities are listed in

Table 3. Numbers of mismatching with and without motion prediction

Angular Velocity	Feature Correspondence Without Motion Prediction	Feature Correspondence With Motion Prediction
5°/s	0	0
10°/s	0	0
20°/s	1	1
30°/s	6	0
40°/s	18	3
50°/s	20	7

. The results indicate that using motion prediction helps to identify better correspondences.

Moreover, six degrees of freedom of the helmet in the T-Mac coordinate system $O_T$-$X_T{Y}_T{Z}_T$ at each triggering moment are also acquired (see Figure 19). Based on the 600 sets of data obtained, standard deviations of six degrees of freedom are listed in

Table 4. Standard deviations of dynamic six-degree-of-freedom (6-DOF) measurement

Parameter	Standard Deviation
Angle about $X_T$ axis (°)	0.0137
Angle about $Y_T$ axis (°)	0.0140
Angle about $Z_T$ axis (°)	0.0147
$X_T$ position (mm)	0.829
$Y_T$ position (mm)	0.640
$Z_T$ position (mm)	0.834

. Standard deviations of dynamic measurement are slightly larger than those of static measurement. This problem is probably caused by the time synchronization error for triggering the multi-camera rig and the T-Mac, which needs to be further verified.

6. Conclusions

In this paper, a multi-camera rig with excellent portability and high reliability is presented for dynamic 6-DOF measurement. The multi-camera rig increases the entire field of view significantly, while guaranteeing measurement accuracy by space resection adjustment. The LED markers offer a more flexible and robust solution in complex manufacturing sites. The improved Nearest Neighbor method is employed for feature correspondence under dynamic conditions. Besides, the proposed global tracking method is validated by simulations and experiments, which demonstrate good performance of static and dynamic measurement.

Considering that the proposed feature matching method is suitable for slow-moving conditions, inertial measurement unit (IMU) will be utilized in future research. With high measuring frequency, IMU provides accurate compensations of positions and orientations for vision measurement in a short time. Meanwhile, vision measurement is able to effectively correct the drift error of IMU. Therefore, vision-inertial tracking is a promising method to deal with fast and intricate movements.