A Novel Dynamic Light-Section 3D Reconstruction Method for Wide-Range Sensing

Abstract

Existing galvanometer-based laser-scanning systems are challenging to apply in multi-scale 3D reconstruction because of the difficulty in achieving a balance between a high reconstruction accuracy and a wide reconstruction range. This paper presents a novel method that synchronizes laser scanning by switching the field-of-view (FOV) of a camera using multi-galvanometers. Beyond the advanced hardware setup, we establish a comprehensive geometric model of the system by modeling dynamic camera, dynamic laser, and their combined interaction. Furthermore, since existing calibration methods mainly focus on either dynamic lasers or dynamic cameras and have certain limitations, we propose a novel high-precision and flexible calibration method by constructing an error model and minimizing the objective function. The performance of the proposed method was evaluated by scanning standard components. The results show that the proposed 3D reconstruction system achieves an accuracy of 0.3 mm when the measurement range is extended to 1100 mm × 1300 mm × 650 mm. This demonstrates that for meter-scale reconstruction ranges, a sub-millimeter measurement accuracy is achieved, indicating that the proposed method realizes multi-scale 3D reconstruction and simultaneously allows for high-precision and wide-range 3D reconstruction in industrial applications.

1. Introduction

Light-section vision systems are widely used in many applications for their adaptability, high accuracy, and effective cost [1,2,3], such as rail traffic monitoring [4], medical imaging [5], robotics [6], and industrial production [7]. Such systems typically comprise a camera, laser projector, and mechanical scanning platform. The line laser projects laser stripes onto the surface of the object, whereas the camera captures an image of the object with the laser stripes. The three-dimensional (3D) geometric information of the object is then obtained by triangulation, as extensively reviewed in the literature [8]. The 3D reconstruction of an object can be completed by passing laser stripes or objects through a mechanical scanning platform.

Traditional laser scanners rely primarily on mechanical driver shafts, which are large, complex, and slow [9,10]. To overcome these limitations, various scanning mechanisms have been proposed. For instance, Du [2] designed a system that mounts a line laser on the end of a robotic arm to improve scanning flexibility; however, its scanning accuracy is limited by the precision of the robotic arm. Jiang [11] proposed a system that uses gimbals to drive the laser and camera for scanning; however, the system’s size is significant, and its scanning speed is slow. In recent years, galvanometers have emerged as promising scanning devices because of their small size, fast rotation, and high control accuracy. This galvanometer-based solution provides a better alternative in terms of laser-scanning accuracy and speed [12]. However, existing galvanometer-based laser-scanning systems are primarily designed to perform laser scanning while leaving the camera fixed. The limited FOV of a fixed camera causes a trade-off between the accuracy and the sensing range of the system, which significantly affects its efficiency.

In this study, we propose a novel dynamic light-section 3D reconstruction system that combines a dynamic laser and dynamic camera using multi-galvanometers. Our approach utilizes multiple galvanometers to synchronize laser scanning and the FOV switching of the camera, thereby enabling high-precision and wide-range 3D reconstruction. Calibration is required to achieve this, which includes the system calibration of the galvanometer-based dynamic laser and camera, and their joint calibration. For calibrating galvanometer-based dynamic laser systems, Eisert [13] introduced a geometric model and calibration procedure; however, the model is complicated, and its optimization is difficult, thus leading to a low accuracy. Yu [14] designed a one-mirror galvanometer laser scanner. However, the calibration procedure is complex, and the objective function is difficult to optimize. Similarly, Yang [15] proposed a calibration method based on a precision linear stage. However, this approach relies on a precision instrument and lacks flexibility. For calibrating galvanometer-based dynamic camera systems, Ying et al. [16,17,18] introduced self-calibration methods, which are complex in theory and difficult to implement. Kumar [19] proposed a calibration method based on the look-up table (LUT) using simple linear parameters, which requires complex pre-processing. Junejo et al. [16,17,20,21] proposed feature-based calibration methods, which are time-consuming and have a low accuracy. Han [22] introduced a calibration method for galvanometer-based camera using an end-to-end single-hidden-layer feed forward neural network model, but it is computationally intensive. Boi et al. [23,24] proposed manifold constrained Gaussian process regression methods for galvanometer setup calibration, which rely on data-driven and complex calibration procedures. Hu [25] built a galvanometer mirror-based stereo vision measurement system and established a mirror reflection model, but it still lacks an accurate calibration method.

In conclusion, current light-section 3D reconstruction systems cannot simultaneously have a high accuracy and wide range. Moreover, existing calibration methods only focus on calibrating either dynamic lasers or dynamic cameras and still have some shortcomings, as mentioned above. To address these limitations, this study proposes a novel dynamic 3D reconstruction system that overcomes the trade-off between accuracy and measurement range by synchronizing laser scanning and the FOV switching of a camera based on multiple galvanometers. Additionally, we propose a novel comprehensive calibration solution for the proposed system, encompassing the calibration of the dynamic camera, dynamic laser, and their joint calibration. The contributions of this study can be summarized as follows:

(1)

A novel dynamic light-section 3D reconstruction system is designed based on multiple galvanometers. To the best of our knowledge, this system is the first to synchronize laser scanning and the FOV switching of camera, thus enabling high-precision and wide-range 3D reconstruction simultaneously.

(2)

A novel high-precision and flexible calibration method for the dynamic 3D system is proposed by constructing an error model and objective function based on the combined model of the dynamic camera and dynamic laser. This method is not only applicable to the proposed system but also to other single galvanometer-based dynamic laser or camera systems.

(3)

Experiments were conducted to validate the proposed dynamic 3D reconstruction method and demonstrate its accuracy. To the best of our knowledge, compared to all existing galvanometer-based laser-scanning methods, our approach has the highest measurement range while maintaining the same level of measurement accuracy.

The system design and geometric model are described in Section 2. The proposed calibration method and error compensation methods are described in Section 3. Section 4 presents the validation experiments and results. Finally, Section 5 presents the conclusions.

2. System Design and Geometric Model

2.1. System Design

The dynamic light-section 3D reconstruction system consists of a CMOS camera, a line laser, and two galvanometer mirror systems, as shown in Figure 1. The camera and Galvanometer-1 form a dynamic camera system, whereas the laser and Galvanometer-2 form a dynamic laser system.

Based on the geometric model of the system and pre-calibration, the 3D information of the target can be calculated from the captured laser image and voltage values of the two galvanometers. The working principle is illustrated in Figure 2. A spherical object on a flat plane is employed to demonstrate the process of dynamic 3D reconstruction. First, the system utilizes multi-galvanometer control to scan the target surface. When the system is activated, a line laser projects a laser stripe onto Galvanometer-2, which reflects the stripe onto the surface of the object. By controlling the voltage of Galvanometer-2, the laser stripe can scan the target. Simultaneously, the dynamic camera system captures laser images from different angles by adjusting the voltage of Galvanometer-1. Next, the laser-center-pixel coordinates are obtained using the laser stripe extraction algorithm. The 3D reconstruction of the laser stripe is performed by combining the voltage values of multiple galvanometers, laser pixel coordinates, geometric models of the dynamic 3D reconstruction system, and calibrated parameters. The point clouds of all the laser stripes are converted to the same coordinate frame using the transformation matrix of the dynamic camera. The system performs error correction based on the joint calibration to optimize accuracy. Finally, the system generates a point cloud for the target and completes the dynamic 3D reconstruction. Accurate geometric modeling and calibration methods are essential to ensure the 3D reconstruction accuracy of the system.

2.2. Geometric Model

The geometric model of the system is shown in Figure 2b. When the system is precisely machined, we can assume that for the dynamic camera system, the optical axis of the camera is perpendicular to the rotation axis of Galvanometer-1’s pan mirror and incident at the center of the pan mirror. For the dynamic laser system, the laser plane is perpendicular to the rotation axis of Galvanometer-2’s pan mirror and incident on the center of the mirror. Four coordinate frames are established. $\left\{W\right\}$ is the world coordinate frame defined on the surface of the planar chessboard target, with the origin point O located at the upper-left corner of the chessboard. The X-axis and Y-axis are parallel to the chessboard array, while the Z-axis is perpendicular to the O-XY plane, following the right-hand coordinate system. $\left\{C\right\}$ represents the camera coordinate frame, where the Z-axis corresponds to the camera’s optical axis, and the X-axis and Y-axis are parallel to the image plane, following the right-hand coordinate frame. $\left\{G\right\}$ denotes the coordinate frame of Galvanometers-1, with the Z-axis corresponding to the rotation axis of the pan mirror. The X-axis is aligned with the camera’s optical axis, while the Y-axis aligns with the line connecting the center points of the pan and tilt mirrors. $\left\{V_t\right\}$ is the virtual camera coordinate frame formed by the reflection of $\left\{C\right\}$ through the pan and tilt mirrors of Galvanometers-1. According to the operating principle of a galvanometer, the rotation angle of the pan–tilt mirror is proportional to the voltage. For Galvanometer-1, the voltages of the pan–tilt mirrors are denoted by $U_{1-pan}$ and $U_{1-tilt}$. Therefore, the rotation angle of the pan mirror is ${\theta }_1=k_{1-pan}{U}_{1-pan}$, and the rotation angle of the tilt mirror is ${\theta }_2=k_{1-tilt}{U}_{1-tilt}$. As the pan–tilt mirror rotates, $\left\{V_t\right\}$ reflects the change in $U_{1-pan}$ and $U_{1-tilt}$. When $U_{1-pan}=U_{1-tilt}=0$, the initail virtual camera coordinate frame is denoted by $\left\{V_0\right\}$. The relationship between $\left\{V_t\right\}$ and $\left\{V_0\right\}$ is given by Equation (1).

$$\begin{array}{c}\hfill {}^{V_t}{\mathit{T}}_{V_0}={}^{V_t}{\mathit{T}}_G{}^G{\mathit{T}}_{V_0},\end{array}$$

(1)

where ${}^{V_t}{\mathit{T}}_{V_0}$ is the transformation matrix between $\left\{V_0\right\}$ and $\left\{V_t\right\}$. ${}^{V_t}{\mathit{T}}_G$ denotes the transformation matrix between $\left\{G\right\}$ and $\left\{V_t\right\}$. ${}^G{\mathit{T}}_{V_0}$ denotes the transformation matrix between $\left\{V_0\right\}$ and $\left\{G\right\}$. As shown in the geometric model diagram, $\left\{C\right\}$ is first reflected by a pan mirror and then by a tilt mirror. The geometries of the two reflections are modeled using Equation (3). For $\left\{V_0\right\}$, the rotation angle of the pan–tilt mirror is ${\theta }_1={\theta }_2={45}^{\circ }$. The transformation matrix ${}^{V_t}{\mathit{T}}_{V_0}$ is calculated using Equation (2). Thus, the geometric model of the dynamic camera is established.

$${}^{V_t}{\mathit{T}}_{V_0}={}^{V_t}{\mathit{T}}_G{}^{V_0}{\mathit{T}}_G^{-1}|(U_{1-pan}=U_{1-tilt}=0).$$

(2)

$$\begin{array}{c}\hfill {}^{V_t}{\mathit{T}}_G=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& cos2{\theta }_2& sin2{\theta }_2& d(1-cos2{\theta }_2)\\ 0& sin2{\theta }_2& -cos2{\theta }_2& -dsin2{\theta }_2\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}-cos2{\theta }_1& sin2{\theta }_1& 0& 0\\ sin2{\theta }_1& cos2{\theta }_1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}0& 0& 1& -l\\ 1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 1\end{array}\right]=\\ \hfill \left[\begin{array}{cccc}sin2{\theta }_1& 0& -cos2{\theta }_1& lcos2{\theta }_1\\ cos2{\theta }_{1}cos2{\theta }_2& sin2{\theta }_2& sin2{\theta }_{1}sin2{\theta }_2& -lsin2{\theta }_{1}cos2{\theta }_2+d(1-cos2{\theta }_2)\\ cos2{\theta }_{1}sin2{\theta }_2& -cos2{\theta }_2& sin2{\theta }_{1}sin2{\theta }_2& -lsin2{\theta }_{1}sin2{\theta }_2-dsin2{\theta }_2\\ 0& 0& 0& 1\end{array}\right].\end{array}$$

(3)

The coordinates of the pixel points on the laser stripe are denoted by $(u,v)$. The coordinates of the corresponding 3D points in $\left\{V_t\right\}$ are denoted by $(X_V,Y_V,Z_V)$. Based on the pinhole model of the camera, the mapping relationship between their coordinates can be obtained using Equation (4).

$$Z_V\left[\begin{array}{c}u\hfill \\ v\hfill \\ 1\hfill \end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_V\\ Y_V\\ Z_V\end{array}\right],$$

(4)

where $(u_0,v_0)$ is the principal point of the image, and $f_x$ and $f_y$ are the focal length of the camera. For Galvanometer-2, the rotation angle of the pan mirror is denoted by ${\theta }_3=k_{2-pan}{U}_{2-pan}$, and the rotation angle of the tilt mirror is denoted by ${\theta }_4=k_{2-tilt}{U}_{2-tilt}$. When $U_{2-pan}=U_{2-tilt}=0$, the dynamic laser is in its initial position. The initial laser plane in $\left\{V_0\right\}$ is denoted by ${}^{V_0}plan{e}_0$, and its equation is $A_{0}x+B_{0}y+C_{0}z+D_0=0$. The rotation axis of the dynamic laser in $\left\{V_0\right\}$ is denoted as $\overrightarrow{\mathit{n}}=(n_x,n_y,n_z)$. The plane of the laser after rotation about the rotation axis is denoted as ${}^{V_0}plan{e}_0$, and the equation is $A_{V_0}x+B_{V_0}y+C_{V_0}z+D_{V_0}=0$.

The light path of the dynamic laser is reflected by a mirror and rotated along its axis. The rotation angle of the laser plane is twice that of the mirror plane. Therefore, the equation for the dynamic laser plane after rotation in $\left\{V_0\right\}$ can be solved using Equation (5).

$$\left[\begin{array}{c}{A}_{V_0}\\ B_{V_0}\\ C_{V_0}\end{array}\right]=\mathit{R}\left(\overrightarrow{\mathbf{n}},2{\theta }_4\right)\left[\begin{array}{c}{A}_0\\ B_0\\ C_0\end{array}\right],$$

(5)

where $\mathit{R}$ represents the Rodrigues transformation. Using a point $\mathit{P}=(X_0,Y_0,Z_0)$ on the rotation axis, $D_{V_0}$ can be calculated as $D_{V0}=-A_{V_0}{X}_0-B_{V_0}{Y}_0-C_{V_0}{Z}_0$. Combining this with the transformation matrix ${}^{V_t}{\mathit{T}}_{V_0}$ in Equation (2), the equation for the dynamic laser plane in $\left\{V_t\right\}$ can be calculated as

$${}^{V_t}plane={}^{V_t}{\mathit{T}}_{V_0}{}^{V_0}plane.$$

(6)

The equation for ${}^{V_t}plane$ is denoted as $A_{V}x+B_{V}y+C_{V}z+D_V=0$. By extracting the pixel points $(u,v)$ from the laser stripe, the corresponding 3D point can be calculated as ${}^{V_t}\mathit{P}=(X_V,Y_V,Z_V)$. Therefore, the relationship between ${}^{V_t}\mathit{P}$ and the change in the galvanometer mirror angles can be expressed by Equation (7).

$$\left\{\begin{array}{c}{Z}_V=D_V/[A_V\times (u-u_0)/f_x+B_V\times (v-v_0)/f_y+C_V]\hfill \\ X_V=(u-u_0)/f_x\times Z_V\hfill \\ Y_V=(v-v_0)/f_y\times Z_V\hfill \\ \left\{A_V,B_V,C_V,D_V\right\}=\mathit{F}\left({\theta }_1,{\theta }_2,{\theta }_4;A_0,B_0,C_0,D_0,l,d,\overrightarrow{\mathbf{n}},\mathbf{P}\right).\hfill \end{array}\right.$$

(7)

$\mathit{F}$ represents a parameterized mapping function, where ${\theta }_1,{\theta }_2,{\theta }_4$ are variables and other parameters are constants. Because $\left\{V_t\right\}$ changes constantly with the scanning angle, it is necessary to convert all ${}^{V_t}\mathit{P}$ into a coordinate frame that is fixed with $\left\{W\right\}$. For ease of calculation, we choose $\left\{V_0\right\}$ and obtain ${}^{V_0}\mathit{P}=(X_{V_0},Y_{V_0},Z_{V_0})={}^{V_0}{\mathit{T}}_{V_t}{}^{V_t}\mathit{P}$.

Thus, we establish the relationship between $(u,v)$ and $(X_{V_0},Y_{V_0},Z_{V_0})$ to formulate the 3D reconstruction. In the geometric model of the 3D dynamic system, Equation (7) shows that $f_x,f_y,u_0,v_0$ can be obtained by calibrating the camera. $A_0,B_0,C_0,D_0$ can be obtained from the laser plane calibration. $l,d,\overrightarrow{\mathit{n}}=(n_x,n_y,n_z),\mathit{P}=(X_0,Y_0,Z_0)$ are unknown, and a calibration algorithm must be designed to obtain the unknowns.

3. Calibration Method

The proposed system calibration method is divided into three parts: the dynamic camera calibration, the dynamic laser calibration, and the joint calibration of the dynamic camera and laser for error correction.

3.1. Dynamic Camera Calibration

Firstly, we perform intrinsic parameter calibration using Zhang’s method [26] for distortion correction and obtain the camera parameters $f_x,f_y,u_0,v_0$. The dynamic camera geometric model described in Section 2 is used to determine the constraint relationship between $\left\{V_t\right\}$ and $\left\{G\right\}$, as shown in Equation (3); thus, we can obtain the parameters l and d. The proposed dynamic camera calibration method uses a large calibration board as shown in Figure 2b. The calibration board measures 740 × 740 mm and comprises a total of 35 × 35 circular markers. These markers are constructed from 25 individual 7 × 7 sub-patterns. Each sub-pattern features a central larger circular marker with a diameter of 15 mm, while the remaining smaller circular markers have a diameter of 10 mm, with a center-to-center spacing of 20 mm. The purpose of the larger circular markers is to establish the relationships between calibration points across different FOVs. The calibration board is scanned by varying the galvanometer voltage to obtain numerous images at different rotation angles. Each image corresponds to a virtual coordinate frame. The number of images is denoted as n. The transformation matrix between $\left\{V_t\right\}$ and $\left\{W\right\}$ is obtained through Zhang’s extrinsic parameter calibration method [26], denoted as ${}^{V_0}{\mathit{T}}_W,{}^{V_1}{\mathit{T}}_W,{}^{V_2}{\mathit{T}}_W,\dots ,{}^{V_n}{\mathit{T}}_W$. As the relative positions between the calibration points in these images are known, the transformation matrix between the virtual coordinate frames is calculated as ${}^{V_1}{\mathit{T}}_{V_0},{}^{V_2}{\mathit{T}}_{V_0},\dots ,{}^{V_n}{\mathit{T}}_{V_0}$. These values are taken as observations. Multiple sets of observations are used to solve for the parameters to be calibrated. The initial pan–tilt angles of Galvanometer-1 are denoted by ${\theta }_1^{\left(0\right)}$ and ${\theta }_2^{\left(0\right)}$. They are the corresponding angles of $\left\{V_0\right\}$ and the first calibration image. The pan–tilt angles of Galvanometer-1 corresponding to $\left\{V_n\right\}$ and nth calibration images are denoted as ${\theta }_1^{\left(n\right)}$ and ${\theta }_2^{\left(n\right)}$. Therefore, $\left\{V_0\right\}$ and $\left\{V_n\right\}$ are defined as follows:

$$\begin{array}{c}{}^{V_0}{\mathit{T}}_G={}^{V_t}{\mathit{T}}_G{|}_{{\theta }_1={\theta }_1^{\left(0\right)},{\theta }_2={\theta }_2^{\left(0\right)}}=k_{ij}({\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d),\\ {}^{V_n}{\mathit{T}}_G={}^{V_t}{\mathit{T}}_G{|}_{{\theta }_1={\theta }_1^{\left(n\right)},{\theta }_2={\theta }_2^{\left(n\right)}}=g_{ij}({\theta }_1^{\left(n\right)},{\theta }_2^{\left(n\right)},l,d),\end{array}$$

(8)

where $k_{ij}$ and $g_{ij}$ represent the function of ${}^{V_0}{\mathit{T}}_G$ and ${}^{V_n}{\mathit{T}}_G$. l and d are parameters to be calibrated using Equation (7). The transformation matrix between $\left\{V_0\right\}$ and $\left\{V_n\right\}$ is a 4 × 4 matrix, which can be expressed as follows:

$${}^{V_n}{\mathbf{T}}_{V_0}\left(n\right)=\left[\begin{array}{cccc}{a}_{11}^n& a_{12}^n& a_{13}^n& a_{14}^n\\ a_{21}^n& a_{22}^n& a_{23}^n& a_{24}^n\\ a_{31}^n& a_{32}^n& a_{33}^n& a_{34}^n\\ 0& 0& 0& 1\end{array}\right]=a_{ij}^n.$$

(9)

Simultaneously, ${}^{V_n}{\mathit{T}}_G$ can be calculated using ${}^{V_0}{\mathit{T}}_G$ from Equation (8) and ${}^{V_n}{\mathit{T}}_{V_0}$. For ease of representation, this is denoted as $h_{ij}$.

$$\begin{array}{cc}\hfill & {}^{V_n}{\mathit{T}}_G={}^{V_n}{\mathit{T}}_{V_0}\left(n\right)·{}^{V_0}{\mathit{T}}_G=\hfill \\ \hfill & \sum _{i=1}^4\sum _{j=1}^4{a}_{ij}^n{k}_{ij}\left({\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=h_{ij}\left(a_{ij}^n,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)\hfill \end{array}$$

(10)

Equation (10) is the result of $\left\{V_t\right\}$ obtained from multiple observations. Equation (8) shows the results calculated using the geometric model of the dynamic camera. For all the measured coordinate frames ($\left\{V_0\right\},\left\{V_1\right\},\left\{V_2\right\},\dots ,\left\{V_n\right\}$), the sum of the errors between the theoretical and measured values must be minimized. Therefore, the objective function is formulated using Equation (11).

$$\begin{array}{cc}\hfill & l^{*},d^{*}=\underset{l,d}{argmin}\sum _{t=1}^n\sum _{i=1}^4\sum _{j=1}^4\hfill \\ \hfill & {\left[h_{ij}\left(a_{ij}^n,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)-g_{ij}\left({\theta }_1^{\left(n\right)},{\theta }_2^{\left(n\right)},l,d\right)\right]}^2.\hfill \end{array}$$

(11)

In Equation (3), the parameters l and d exist only in a translation vector. Based on the objective function, Equation (12) can be obtained. Finally, the parameters l and d are obtained by solving Equation (12) using the least-square method.

$$\left\{\begin{array}{c}{g}_{14}\left({\theta }_1^{\left(1\right)},{\theta }_2^{\left(1\right)},l,d\right)-h_{14}\left(a_{ij}^1,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0\hfill \\ g_{24}\left({\theta }_1^{\left(1\right)},{\theta }_2^{\left(1\right)},l,d\right)-h_{24}\left(a_{ij}^1,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0\hfill \\ g_{34}\left({\theta }_1^{\left(1\right)},{\theta }_2^{\left(1\right)},l,d\right)-h_{34}\left(a_{ij}^1,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0\hfill \\ \dots \hfill \\ g_{14}\left({\theta }_1^{\left(n\right)},{\theta }_2^{\left(n\right)},l,d\right)-h_{14}\left(a_{ij}^n,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0\hfill \\ g_{24}\left({\theta }_1^{\left(n\right)},{\theta }_2^{\left(n\right)},l,d\right)-h_{24}\left(a_{ij}^n,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0\hfill \\ g_{34}\left({\theta }_1^{\left(n\right)},{\theta }_2^{\left(n\right)},l,d\right)-h_{34}\left(a_{ij}^n,{\theta }_1^{\left(0\right)},{\theta }_2^{\left(0\right)},l,d\right)=0.\hfill \end{array}\right.$$

(12)

3.2. Dynamic Laser Calibration

Based on the geometric modeling of the dynamic laser presented in Section 2, the equation of ${}^{V_0}plan{e}_0$ and the rotation axis of the dynamic laser $\overrightarrow{\mathit{n}}$ must be calibrated when Galvanometer-1 is at the initial position. The calibration of ${}^{V_0}plan{e}_0$ is conducted using the methodology outlined in [27]. This process involves the acquisition of laser images at various positions using a checkerboard calibration plate. Through this approach, the laser plane is accurately defined by fitting multiple laser lines. For the extraction of the laser center, the technique described in [28] is employed. This algorithm, based on the Hessian matrix, can achieve sub-pixel precision extraction and a processing speed of 1350 frames per second. Following these procedures, we successfully derive the equation $A_{0}x+B_{0}y+C_{0}z+D_0=0$.

The Galvanometer-2 voltages $U_{2-tilt}=U_1,U_2,\dots ,U_m$ are used to move the laser and obtain multiple laser planes. Next, the respective equations are calibrated in the same manner as ${}^{V_0}plan{e}_0$ and denoted as ${}^{V_0}plan{e}_1,{}^{V_0}plan{e}_2,\dots ,{}^{V_0}plan{e}_m$. The unit normal vectors of these planes are calculated as ${\overrightarrow{\mathit{n}}}_0(n_{x_0},n_{y_0},n_{z_0})$, ${\overrightarrow{\mathit{n}}}_1(n_{x_1},n_{y_1},n_{z_1})$, ${\overrightarrow{\mathit{n}}}_2(n_{x_2},n_{y_2},n_{z_2}),\dots ,$${\overrightarrow{\mathit{n}}}_m(n_{x_m},n_{y_m},n_{z_m})$. In the absence of errors, the laser planes intersect along the same straight line. This line is the laser rotation axis $\overrightarrow{\mathit{n}}$ and is also the rotation axis of the tilt mirror in Galvanometer-2. For a normal vector in any laser plane, we obtain $\overrightarrow{\mathit{n}}·{\overrightarrow{\mathit{n}}}_i=0$$(i=0,1,2,\dots ,m)$. However, $\overrightarrow{\mathit{n}}·{\overrightarrow{\mathit{n}}}_i$ is not exactly equal to zero because of various errors. Therefore, for all laser planes, the objective function is formulated as shown in Equation (13).

$$\begin{array}{c}\hfill {\overrightarrow{\mathbf{n}}}^{*}=\underset{n_x,n_y,n_z}{argmin}\left(\sum _{i=0}^m{\left(n_x{n}_{x_i}+n_y{n}_{y_i}+n_z{n}_{z_i}\right)}^2\right).\end{array}$$

(13)

The direction vector $\overrightarrow{\mathit{n}}$ of the rotation axis is obtained by minimizing the objective function. $\mathit{P}=(X_0,Y_0,Z_0)$ is a point on the rotation axis in all laser planes and can be obtained using the least-square method. All parameters in Equation (7) are obtained by calibrating the camera, laser plane, dynamic camera, and dynamic laser. Thus, we complete the calibration of the proposed dynamic 3D system.

3.3. Joint Calibration for Error Correction

For a well-calibrated dynamic light-section 3D reconstruction system, there are two sources of error, dynamic camera and dynamic laser, as listed in

Table 1. Error Source and Analysis.

	Error	Source and Analysis
1	Rotation angle of galvanometer mirror $(\theta 1,\theta 2)$	Non-linear deviations exist between voltage and rotation angle of Galvanometer-1.
2	Dynamic camera geometric model	The spectacular reflection geometric model has deviations with mechanical structure.
3	Calibration of parameter $(l,d)$	This error has been optimized by the proposed error model and objective function in this paper.
4	Camera intrinsic parameters calibration	This non-linear error is optimized using Zhang’s [26] calibration method.
5	Rotation angle of galvanometer mirror $(\theta 3,\theta 4)$	Non-linear deviations exist between voltage and rotation angle of Galvanometer-2.
6	Dynamic laser geometric model	This error depends on the accuracy of the laser’s mechanical installation.
7	Calibration of laser rotation axis	This error is optimized by the proposed objective function.
8	Laser center curve extraction	Center extraction algorithm is based on the Hessian Matrix and ensures a high extraction accuracy.

.

This study proposes an error-correction method based on the joint calibration of a dynamic camera and dynamic laser. After the calibration is completed, error correction is performed based on the 3D reconstructed results. Theoretically, when Galvanometer-1 is scanning and Galvanometer-2 is fixed, the reconstructed laser point cloud coincides perfectly. However, as explained in the error source analysis, there is some deviation between the multiple laser point clouds owing to these errors. We correct these errors using point-cloud registration based on the Iterative Closest Point (ICP) algorithm [29] to obtain the accurate transformation matrix.

The calibration process is designed based on the following principle. If Galvanometer-2 remains stationary, the laser will be out of the FOV after Galvanometer-1 has scanned a certain range. Therefore, multiple calibration positions must be set in advance to maintain the laser in the FOV. These are set in advance as $p_1,p_2,\dots ,p_n$, and the corresponding voltages of Galvanometer-2 at these positions are $(s_1,t_1),(s_2,t_2),\dots ,(s_n,t_n)$, respectively. The laser plane equations $plan{e}_{p_1},plan{e}_{p_2},\dots ,plan{e}_{p_n}$ at these positions are calibrated using the laser plane calibration method described in Part B. The voltages of Galvanometer-1 are denoted by ($s_1^{\prime }$, $t_1^{\prime }$), ($s_2^{\prime }$, $t_2^{\prime }$)$,\dots ,$($s_m^{\prime }$, $t_m^{\prime }$), respectively. The error-correction flow is presented in Algorithm 1.

Algorithm 1 Joint Calibration for Error Correction

1: Input: $f_x,f_y,u_0,v_0$; $l,d$; $A_1,B_1,C_1,D_1,\dots ,A_n,B_n,C_n,$ $D_n$; $(s_1,t_1),(s_2,t_2),\dots ,(s_n,t_n)$.   2: Output: ${}^{V_0}{\mathit{T}}_{V_1},{}^{V_0}{\mathit{T}}_{V_2},\dots ,{}^{V_0}{\mathit{T}}_{V_m}$.   3: Initialize: i← 1, $(s,t)$←$(s_1,t_1)$, $(s^{\prime },t^{\prime })$←$f(s,t)$, du ← 0.1, ${}^{V_1}plane$←$plan{e}_{p_1}$, $\mathit{M}$← Unit matrix.   4: Using Equation (7) to obtain the laser point cloud ${}^{V_0}{\mathit{P}}_1$.   5: while $i<m$ do   6:      Laser Image capture and center curve extraction;   7:      ${}^{V_t}plane=g(s,t,s^{\prime },t^{\prime })$;   8:      Using Equations (1)–(3) to calculate ${}^V{\mathit{T}}_{V_t}$;   9:      ${}^{V}plane$←${}^V{\mathit{T}}_{V_t}{}^{V_t}plane$; 10:      Using Equation (7) to obtain the point cloud ${}^{V_0}{\mathit{P}}_i$. 11:      ${}^V{\mathit{T}}_{V_t}$← Transformation matrix between ${}^{V_0}{\mathit{P}}_1$ and ${}^{V_0}{\mathit{P}}_i$; 12:      ${}^{V_0}{\mathit{T}}_{V_t}$←${\mathit{M}}^V{\mathit{T}}_{V_t}$; 13:      if $i\%50==0$ then 14:          $(s,t)$←$(s_{(i/50)},t_{(i/50)})$, ${}^{V_t}plane{\leftarrow }^{V}plane$; 15:          Using Step. (8)–Step. (11) to obtain point cloud $\mathit{P}$. 16:          $\mathit{M}$← Transformation matrix between $\mathit{P}$ and ${}^{V_0}{\mathit{P}}_i$; 17:      end if 18:      $i\leftarrow i+1$; 19:      $(s^{\prime },t^{\prime })$$\leftarrow (s^{\prime }+du\times i\%200,t^{\prime }+1\times int(i/200))$; 20: end while

4. Experiment

The proposed dynamic 3D reconstruction system was built as shown in Figure 3. The camera model was MV-CA004-10UC, with a pixel size of 6.9 µm × 6.9 µm, resolution of 720 pixels × 540 pixels, and frame rate of 500 fps. The exposure time of the camera to capture the dark image of the laser was 500 µs. The laser model was LXL65050-16, and the laser wavelength was 650 nm. The models of both Galvanometer-1 and Galvanometer-2 were TSH8310. The galvanometer was used to scan a range of $\pm {20}^{\circ }$ using a control voltage range from $-10$ V to +10 V. The maximum scan frequency was 1 kHz, with an angular resolution of 0.0008; thus, the system had the potential for a high accuracy and resolution.

4.1. Calibration Accuracy Verification

4.1.1. Dynamic Camera Calibration Accuracy

Twenty-five images of the calibration board were collected when $U_{1-pan}=U_{1-tilt}=0$. The camera was calibrated using Zhang’s [26] calibration method in OpenCV. After calibration, the intrinsic parameters were $f_x=$ 7801.38, $f_y=$ 7798.24, $u_0=$ 359.51, and $v_0=$ 269.54. The focal length was 53.83 mm. According to the calibration method for the dynamic cameras presented in Section 3, the system parameters were solved as l = 83.45 mm and d = 22.14 mm. Based on these calibration results, the geometric model proposed in Section 2 can be used to calculate the theoretical transformation matrix for the pan–tilt mirror of Galvanometer-1 at different angles.

The transfer matrices corresponding to these angles are directly measured using a calibration board. The matrix 2-norm is calculated according to Equation (14) to compare the theoretical matrices $\mathit{A}$ and measured transfer matrices $\mathit{B}$ for the calibration accuracy verification. The pan–tilt voltages of Galvanometer-1 are varied from $-10$ V to +10 V at intervals of 4 V. Thirty-six positions are measured. The error between the theoretical and measured transfer matrices is obtained, and the error curves are shown in Figure 4a. The results show that the RMSE (Root Mean Square Error) is 1.231 mm between the theoretical and measured values.

$$E(\mathit{A},\mathit{B})=\sqrt{\sum _{i=1}^4\sum _{j=1}^4{\left({\mathit{A}}_{ij}-{\mathit{B}}_{ij}\right)}^2}$$

(14)

This confirms the accuracy of the dynamic camera calibration. The observed errors originate from the geometric model and the calibration process, as explained in the error analysis section. It is important to note that the measured values obtained for the virtual camera using the calibration board may also exhibit slight deviations. Consequently, these findings serve as a validation of the accuracy of the dynamic camera calibration; however, these cannot be solely relied upon to assess the accuracy of the calibration. A more detailed accuracy verification can be conducted based on the outcomes of the 3D reconstruction analysis.

4.1.2. Dynamic Laser Calibration Accuracy

With Galvanometer-1 fixed, the calibration board was positioned within the FOV of the virtual camera. The tilt mirror of Galvanometer-2 was rotated 30 times with a step size of $0.1$ V, allowing the system to scan the calibration board, whose position was randomly changed 5 times (ensuring clear imaging in the virtual camera); the same 30 scans were repeated for each position. The laser rotation axis is solved as $(\overrightarrow{\mathbf{n}},\mathbf{P})=([0.99,0.02,-0.0004],[-18310.30,-195.93,257.97])$. Figure 4b visually represents the laser plane and the rotation axis. Notably, the calibrated rotation axis align with the intersection of the laser planes, providing evidence for the accuracy of the dynamic laser calibration. A detailed accuracy assessment is subsequently performed by analyzing the results of the 3D reconstruction.

4.1.3. Joint Calibration Accuracy

A calibration sphere was selected as the 3D reconstruction target for error correction. Galvanometer-2 was controlled to project the laser stripe onto the sphere, while Galvanometer-1 was fixed. The virtual camera, controlled by Galvanometer-1, captured images of the laser stripe from different views. The 3D reconstruction of these laser stripe images was performed based on the calibration results and geometric models of the 3D dynamic system. The reconstructed point clouds, which are indicated as white, are shown in Figure 2a, ’Error Correction’. Notably, white point clouds exhibit non-overlapping regions owing to errors. The correction method described in Algorithm 1 is employed to register these white point clouds. The registration results are shown as colored point clouds in Figure 2a. The distance between the point clouds before and after the correction is calculated to evaluate the error. The calculation formula is as follows:

$$d=\frac{1}{|P_s|}\sum _{i=1}^{\left|P_s\right|}{∥p_t^i-p_s^i∥}^2,$$

(15)

Here, $p_s$ represents the point cloud of the laser stripe captured in the first virtual camera view, and $p_t$ represents the point cloud of the laser stripe captured from another view. $|P_s|$ denotes the number of points in $p_s$. The error between $p_t$ and $p_s$ is determined by performing a nearest-neighbor search, denoted as Error1. After the point-cloud registration, the error between $p_t$ and $p_s$ is calculated as Error2. In addition, the error before correction is computed as Error3 using the matched points from the point-cloud registration result. The error curves are shown in Figure 4c. The RMSE of Error1 and Error3 before correction are calculated as 4.928 and 5.475 mm, respectively. However, after error correction, the RMSE of Error2 is significantly reduced to 0.197 mm. These results evidently indicate a substantial improvement in the accuracy following the error-correction process.

4.2. 3D Reconstruction Accuracy Verification

4.2.1. Standard Blocks Reconstruction Test

A standard stair block was employed to test the stability of the system and analyze its reconstruction accuracy at different angles. The stair block had a distance of 30 mm between its two planes, with machining errors within 1 µm. The 3D dynamic system proposed in the paper was used to reconstruct a stair block. Scanning was performed by synchronously controlling the tilt mirrors of both Galvanometer-1 and Galvanometer-2, rotating each by 0.1°. Once the scanning and reconstruction processes were completed, a point cloud of the stair block was generated. Two planes (Plane-1 and Plane-2) of the stairs were fitted, and the distance between them was calculated. Point clouds belonging to Plane-2 were used to fit a plane equation using the least-square method. Next, 500 points belonging to Plane-1 were randomly selected, and the average distance between these points and Plane-2 was calculated as the distance of the fitted plane. The difference between the calculated and actual distances is considered as the error, which serves as a measure of the reconstruction accuracy achieved by the system.

The reconstruction distance is 650 mm. The FOV for a single virtual camera is 120 mm × 120 mm, while the dynamic camera’s FOV expands to 1300 mm × 1300 mm (including a 10% overlap area for better stitching), thus enlarging the camera’s imaging range by a factor of 117.4. The scanning range of the dynamic laser is 1100 mm × 1640 mm. The measurement range of the system is determined by the overlapping FOV of the dynamic camera and dynamic laser, which measures 1100 mm × 1300 mm. Thirty different positions are selected to analyze the reconstruction accuracy at different angles. The dynamic camera and laser simultaneously scan the target from these positions to complete the 3D reconstruction process. Figure 5a shows examples of reconstructions obtained from four different positions, providing a visual representation of the reconstructed 3D models. The thickness error, which is related to the rotation angles of Galvanometer-1 and Galvanometer-2, is analyzed, as shown in Figure 5b. It is evident from the graph that the error in 3D reconstruction increases as the rotation angles of the galvanometers deviate from their initial positions (calibration position). This is because the camera’s focal length is adapted to the calibration position, and imaging areas far from the calibration position may become blurred due to defocusing, thereby affecting accuracy. Furthermore, as analyzed in Table 1, errors due to various reasons accumulate more as the distance from the calibration position increases. The RMSE for these thirty positions is calculated as 0.165 mm. These values provide evidence of the high precision achieved by the proposed system for 3D reconstruction.

4.2.2. Accuracy and Reconstruction Range Comparison with Existing Methods

To compare the performance of the proposed method with that of existing methods [12,14,15,30,31,32,33], we conducted comparative experiments using the standard component scanning method. The accuracy of the dynamic light-section 3D system depends primarily on the working distance. To perform a fair comparison, we repeated the standard component scanning procedure at various reconstruction distances, namely, 100, 200, 350, 400, and 1000 mm, which are consistent with the working distances employed in existing methods. As analyzed in the previous experiment, the system’s reconstruction accuracy is related to the scanning position. In order to reduce errors other than calibration errors, we placed the target at the center of the scanning area.

The 3D reconstruction accuracy, the measurement ranges of the traditional methods, and the magnification factor of the proposed method’s reconstruction range compared to traditional methods are presented in

Table 2. Accuracy and reconstruction range comparison of traditional and proposed methods.

Working Distance/mm	Traditional Methods			Proposed Method
	Name Year	Accuracy/mm	Range/mm	Accuracy/mm	Range/mm	Factor of Expanded Range
100	NOM-LSS 2017 [16]	0.01	10 × 10	0.01	130 × 200	260
250	3DM-LS 2018 [17]	0.1	80 × 80
	EAC-LSS 2020 [14]	0.061	80 × 80	0.057	350 × 500	27.3
350	IS-LSS 2020 [30]	0.08	200 × 200	0.08	500 × 700	8.75
400	FFV-LSS 2007 [32]	0.222	150 × 200
	FLR-LSS 2022 [31]	0.1	150 × 200	0.092	650 × 800	17.3
1000	U3D-LSS 2016 [33]	0.382	200 × 300	0.314	1600 × 2000	53.3

. From the obtained results, it can be concluded that the proposed method exhibits smaller errors and larger measurement ranges than the existing methods at the corresponding working distances. This demonstrates the superior performance of our method in terms of accuracy and range compared with existing methods.

4.2.3. Large Object Scanning Test

The large high-precision machined flat plate and sphere were also used to test the 3D reconstruction accuracy. The size of the plate was 740 mm × 740 mm, and the sphere had a diameter of 350 mm. Similar to Experiment B $\left(1\right)$, the system scanned the target and obtained its point clouds with the scanning distance set at 650 mm, and the 3D measurement range was 1100 mm × 1300 mm. The position and angle of the target were arbitrarily changed within the scanning range, and the reconstruction was repeated three times for each target. For the plate target’s point cloud, a plane equation was fitted using the RANSAC algorithm. The distances between all the points and the fitted plane were calculated, and the average of these distances was considered the error of the dynamic 3D system reconstruction. For the sphere target’s point cloud, a sphere was fitted based on the RANSAC algorithm, and the diameter was calculated. The difference between the calculated diameter and the sphere’s actual diameter was taken as the error of the dynamic 3D reconstruction. The RMSE values for the three measurements were 0.281 mm for the plate and 0.226 mm for the sphere, respectively.

This experimental result demonstrates that the system achieves sub-millimeter reconstruction accuracy within a meter-scale reconstruction range, indicating that the system enables high-precision multi-scale 3D reconstruction across a wide-range reconstruction area. Moreover, a limitation of this system is that the dynamic camera extends the FOV using a galvanometer. When the extension angle is large, there can be some defocus, resulting in blurred images and subsequently affecting the accuracy of the 3D reconstruction. To address this issue, image deblurring algorithms can be used to improve imaging quality, thereby enhancing the reconstruction accuracy at the edges of the FOV.

5. Conclusions

A dynamic light-section 3D reconstruction system is proposed in this paper, which overcomes the trade-off between accuracy and measurement range by using multiple galvanometers. A geometric model of the system was established, and a flexible and accurate calibration method was developed. The experimental results demonstrate that the proposed system performs well in terms of measurements, indicating its potential for industrial applications where high-precision and wide-range 3D reconstruction is required. Furthermore, the proposed method can be used in conjunction with an active tracking system for 3D reconstruction of moving targets. This will be introduced in our subsequent work.