A Coarse-to-Fine Registration Approach for Point Cloud Data with Bipartite Graph Structure

Abstract

Alignment is a critical aspect of point cloud data (PCD) processing, and we propose a coarse-to-fine registration method based on bipartite graph matching in this paper. After data pre-processing, the registration progress can be detailed as follows: Firstly, a top-tail (TT) strategy is designed to normalize and estimate the scale factor of two given PCD sets, which can combine with the coarse alignment process flexibly. Secondly, we utilize the 3D scale-invariant feature transform (3D SIFT) method to extract point features and adopt fast point feature histograms (FPFH) to describe corresponding feature points simultaneously. Thirdly, we construct a similarity weight matrix of the source and target point data sets with bipartite graph structure. Moreover, the similarity weight threshold is used to reject some bipartite graph matching error-point pairs, which determines the dependencies of two data sets and completes the coarse alignment process. Finally, we introduce the trimmed iterative closest point (TrICP) algorithm to perform fine registration. A series of extensive experiments have been conducted to validate that, compared with other algorithms based on ICP and several representative coarse-to-fine alignment methods, the registration accuracy and efficiency of our method are more stable and robust in various scenes and are especially more applicable with scale factors.

1. Introduction

With the rapid development of optical measurement technology, three-dimensional (3D) models of real-world objects with different views have been employed to collect data for many application scenarios [1,2,3,4,5]. As the fundamental technique in 3D data processes, the registration method is mainly responsible for determining the correspondence and transformation of multiple-view 3D data, which facilitates the implementation of many reverse engineering applications, such as multi-platform data alignment for airborne and terrestrial laser scanning [6], foot alignment for reconstruction in the footwear consumer field [7], remote sensing image registration in military and civilian fields [8], skull registration for craniofacial reconstruction in the medical field [9], cross-source point cloud registration in large-scale scanned data for street views outdoors [10], 3D face registration for face recognition [11], etc. During the past few decades, the high-performance registration method has become an increasingly important research topic. However, the existing registration algorithms still have many defects in terms of efficiency and accuracy in practice. According to the transformation relationship of different-view 3D data, registration methods can be divided into rigid registration algorithms and non-rigid registration algorithms. In this paper, we mainly focus on rigid alignment methods with some extensibility, and all processed 3D data is PCD.

As the best-known registration algorithm, ICP conducts an iterative process to align the relatively similar 3D point clouds well with constraint conditions of uniform density, high overlap rate and good initial position, and it is prone to trap into local optimal problems [12,13]. Multiple ICP variants have also been proposed to improve performance. Trimmed ICP (TrICP) utilizes the Least-Trimmed Squares approach to improve robustness in low data overlaps [14]. Generalized ICP(GICP) designs a structure by minimizing the plane-to-plane distance to enhance the resistance to noise [15]. Globally Optimal ICP (Go-ICP) provides a globally optimal Euclidean solution under $L_2$ error with a branch-and-bound scheme to avoid local optimization [16]. Multi-resolution ICP (MRICP) incorporates a hierarchical octree for data multi-resolution spatial partitioning to speed up computation and enhance robustness [17]. Markov chain Monte Carlo simulated annealing (MCMC-SA) ICP combined an MCMC method and an improved SA method to accelerate convergence with the global minimum [18]. Although ICP-based algorithms achieve high registration accuracy, the iterative computational approach and good initial position condition restriction limit their applications in practical 3D data processing.

Probability distribution alignment (PDA) methods regard the registration problem as a probability density estimation problem and have good robustness in noise and different data overlap rates. The Normal Distributions Transform (NDT) utilizes a Gaussian probability distribution to represent each point and completes the alignment by maximizing the sum of the Gaussian probability distributions of transformed unregistered PCD [19]. Coherent Point Drift (CPD) fits the Gaussian mixture model (GMM) centroids of target PCD to source PCD through estimating the maximum likelihood [20]. Student’s t latent mixture model (TLMM) constructs a hierarchical Bayesian network to algin different PCD based on student-t mixture model (SMM), which effectively improves the robustness [21]. Among these PDA algorithms, some have lower accuracy with fast registration speed, such as NDT, and conversely, high-performance methods (CPD and TLMM are also applicable for non-rigid alignment) utilize EM iterative algorithms to solve the optimization of the objective function, which cannot avoid large computational and local optimization problems.

In recent years, the PCD registration research based on deep learning framework has gradually attracted the attention of scholars. The Deep Closest Point (DCP) predicts a rigid transformation for two PCD from a deep learning perspective [22]. A Deep Gaussian Mixture Registration (DeepGMR) model designs a neural network to search GMM parameters and computation blocks to obtain optimal transformation [23]. As a data-driven model, 3DMatch learns local volumetric patch descriptors for establishing 3D data correspondences during the rough registration phase [24]. The Robust Point Matching (RPM)-Net confirms point correspondences with differentiable Sinkhorn layers and estimates optimal annealing parameters with the second network [25]. These algorithms have good performance in alignment robustness, accuracy and large data. However, the majority of deep learning methods suffer from heavy memory resources, which limits the generality of application.

The coarse-to-fine alignment algorithms are the most widely used non-rigid registration methods. At first, common coarse alignment algorithms, such as Random Sample Consensus (RANSAC) [26], 4-Points Congruent Sets (4PCS) [27], Principal Component Analysis (PCA) [28], etc., perform fast rough position transformation estimation by randomly selecting some points with distance constraints or downscaled processing. The fine alignment algorithms, such as ICP and its variants, conduct further transformation computation of two PCD on the basis of previous initial transformation correspondence, which greatly reduces the iterative computation. However, the results of these coarse registration methods may be unstable and have a large error, which sometimes imposes a greater burden for the fine alignment process. Then, the coarse registration methods utilize similarity measures of local feature points and descriptors to determine the initial transformation correspondence. SAmple Consensus Initial Alignment (SAC-IA) utilizes Fast Point Feature Histograms (FPFH) local geometry descriptors to match points in different point clouds [29]. Keypoint-based 4PCS (K4PCS) adopts 3D Difference-of-Gaussians or 3D Harris corner detector to extract key points and then performs coarse alignment instead of raw PCD to improve computation efficiency and accuracy [30]. The enhanced performance of coarse alignment algorithms contributes to the application of coarse-to-fine registration methods. Liu et al. [31] utilized FPFH features to find corresponding point pairs, RANSAC to eliminate initial incorrect correspondence and NDT to refine the transformation. Li et al. [32] proposed a graph-enhanced sample consensus (GESAC) to estimate rough transformation with robust shape-annealing, which improved RANSAC-based fine registration accuracy. Although these methods show good alignment performance, the nature of local feature point extraction is prone to interference from noise and inhomogeneous density. Shi et al. [33] adopted a PDA model of NDT to compute the initial transformation matrix, where ICP performs the fine alignment, which improves the robustness of the alignment algorithm to noise. Currently, the top state-of-art coarse-to-fine registration methods summarize the results of continuous optimization of rough alignment and ICP. Wu et al. [34] designed a coarse-to-fine registration algorithm based on FPFH eigenvalue and volume constraints. This algorithm can reduce mismatch rate and improves the point cloud registration precision with ICP. Lu et al. [35] proposed a point cloud registration algorithm combined with improved K4PCS and ICP to improve the speed and accuracy. Wu et al. [36] presented a triangular model registration algorithm based on differential topological singularity points (DTSP)-ICP. This algorithm is able to obtain the mesh model nature of differential topological structure and avoid local errors based on Euclidean distance. Shi et al. [33] utilized SAC-IA to conduct rough alignment and perform fine registration with NDT and ICP, respectively. According to their research, SAC + ICP has better performance. Yao et al. [37] proposed an improved iterative closest point (ICP) algorithm based on the curvature feature similarity. PCA was used to confirm a better initial value, and the similarity of curvature features was used to determine the nearest points for fine registration. Kamencay et al. [38] presented an improved feature-based registration algorithm to reduce time consumption and improve ICP convergence speed, with the combination of 3D scale-invariant feature transform (SIFT) key-point extraction and K-nearest neighbor (KNN) clusters. Xu et al. [39] designed a fast point cloud registration algorithm combined with intrinsic shape signatures (ISS) detector, 3D shape context (3DSC) descriptor, RANSAC and an improved ICP method. The ISS key points described by 3DSC were used for rough alignment with RANSAC, and ICP conducted accurate registration. Wen et al. [9] proposed a hierarchical coarse-to-fine alignment method for skull registration. The improved spin image and k-means algorithm rough stage were used in the rough stage. The advanced ICP algorithm improved performance in point sampling and mismatched point elimination. Wang et al. [40] introduced a coarse-to-fine trimmed strategy with Generalized Iterative Closest Point (GICP) algorithm. This algorithm implemented an inner iteration process and outer strategy to refine the match results. Through various improvements and optimizations, the robustness, accuracy and speed of the coarse-to-fine algorithms have been improved. The coarse-to-fine alignment methods based on local feature points and descriptors are still very competitive. However, these methods do not take global characteristics into consideration, which still easily increases the mismatching rate. This paper is dedicated to proposing a rough alignment method that combines local features with global characteristics assignment and adopts TrICP to complete the subsequent fine alignment process.

Moreover, the general rigid registration methods will fail in point sets with scale factors, while the scale factor is easily produced by scanning data with different resolutions or multi-perspective distance. Several pioneering works have been proposed to extend rigid registration algorithms to the isotropic scale case. Zha et al. [41] computed the scale with extended signature images (ESIs) correspondence and applied it to ICP. Du et al. [42] integrated a bounded scale into ICP. Ying et al. [43] incorporated a scale factor into ICP and iteratively searched for the optimal solution to register data sets with isotropic stretches. Du et al. [44] provided a new isotropic scaling ICP algorithm with the corner point constraint adding to the objective function. Huang et al. [10] presented a coarse-to-fine algorithm to register two cross-source PCD and the estimated scale value with the same GMM mode during the fine registration phase. Wang et al. [40] estimated the isotropic scale factor by solving a point-to-point distance optimization function. Yang et al. [45] proposed a new objective function to compute similarity and affine transformation parameters (including a scale factor) based on a bidirectional kernel mean p-power error (KMPE) loss. In these methods, the scale factor is estimated within the iterative solving process of the objective function, which inevitably leads to complex computational problems. Huang et al. [46] implemented an independent scale normalization process for cross-source PCD registration. Motivated by this method, we have designed a pluggable top-tail (TT) strategy with few computations to extend the proposed coarse-to-fine registration algorithm in isotropic scale scenes.

Table 1. The comparison of different methods for extending rigid registration algorithms to isotropic scale cases.

Method	Computation	Pluggable	Extensibility
Scale factor estimation from ESI [41]	large	no	enable
ICP with bounded scale [42]	large	no	enable
Scale-ICP proposed in [43]	large	no	enable
Scale-ICP proposed in [44]	large	no	enable
Scale estimation with GMM [10]	large	no	enable
Scale factor estimation [40]	large	no	enable
Scale factor computation [35]	large	no	enable
Scale normalization process [46]	normal	yes	enable
TT strategy in this paper	small	yes	enable

compares the computation amount, flexibility and extensibility of the above-mentioned methods.

To the best of our knowledge, the proposed method is an effective solution in two-PCD alignment with poor initial positions and an isotropic scale factor solution. The key to this achievement is the combination of scale factor estimation, local feature points extraction and description, global point pairs correspondence determination and fine registration.

Thus, the main contributions of this paper are as follows:

• TT strategy. Top-tail strategy is an independent and pluggable component, which is designed to be easily and flexibly combined with a variety of existing coarse alignment methods, rather than involved in complex optimum computation. This strategy consists of a normalization operation and a scaling factor estimation process. The normalization operation occurs before the coarse alignment process, and the scaling factor estimation occurs after the coarse registration. The normalization operation adjusts the two data sets to the same scale coordinate system. The coarse alignment results of the normalized original and target datasets contain a rotation matrix, according to which the normalized source object is corrected. Then, a boundary most-value ratio operation is utilized to estimate the scaling factor value based on corrected data sets. The rotated, scale-adjusted source PCD and target PCD are used as input objects for further fine registration.
• Bipartite graph matching scheme. Based on local features and descriptors of key points, bipartite graph matching is adopted to determine the initial correspondence from a global perspective. The bipartite graph structure is utilized to store feature descriptor similarity and a key-points index. Moreover, the weight between bipartite graph nodes should be recorded as the ratio of feature descriptor similarity above the pre-set threshold value. Through the Hungarian algorithm, the maximum set of matching feature points forms the initial correspondence pairs.

The rest of this paper is organized as follows: We present the preliminaries throughout this paper in Section 2. Section 3 details the specific implementation process of the registration approach proposed in this paper. The experimental performance evaluation is conducted in Section 4. Conclusions and discussions are carried out in Section 5.

2. Preliminaries

In this section, we review some of the relevant background knowledge and related knowledge that will be used in this paper.

2.1. 3D SIFT

The 3D SIFT describes local features with the ability to resist the influence of viewing angle changes, noise interference, brightness changes, affine transformations and rotation transformations, which are explored in the difference of Gaussian (DOG) pyramid [47]. At first, a Gaussian pyramid is built through a voxelized point cloud with scale $\epsilon ,$ and the scale space $L\left(\mathrm{X},\epsilon \right)$ can be obtained from the convolution of Gaussian blur kernel $G\left(\mathrm{X},\epsilon \right)$ with 3D image, as described in (1).

$$\begin{array}{c}L\left(\mathrm{X},\epsilon \right)=G\left(\mathrm{X},\epsilon \right)*Q\left(\mathrm{X}\right)\\ G\left(\mathrm{X},\epsilon \right)=e^{-\frac{{\left|\left|\mathrm{X}\right|\right|}^2}{2{\epsilon }^2}}{\left(\sqrt{2\pi }\epsilon \right)}^{-3}\\ DOG\left({\mathrm{X}}_i,{\epsilon }_{j+1}\right)=L\left({\mathrm{X}}_i,{\epsilon }_{j+1}\right)-L\left({\mathrm{X}}_i,{\epsilon }_j\right)\end{array}$$

(1)

where $\mathrm{X}$ denotes 3D coordinates $\left(x,y,z\right)$.

Then, a series of octaves with different scales should be generated. For every scale $\epsilon$, two adjacent scales can be described as (2).

$${\epsilon }_{i+1}=k{\mathsf{\epsilon }}_i,{\epsilon }_0=k^{-1}{\mathsf{\epsilon }}_{min},(i=0,1,\dots ,\xi +2)$$

(2)

To detect local extreme values in the DOG pyramid, nearest k-neighborhood points are compared with the query point. The points with minima or maxima DOG values are regarded as 3D SIFT key points.

2.2. FPFH Descriptor Estimation

The FPFH descriptor is an improved version of the PFH descriptor, which is computed as a histogram of geometric properties at a point $P$ [29]. Within the range of middle point $P_C$ with radius $r$, the computation is based on the estimated angle and positional relationship between normal vectors $n_s$ and $n_t$ for the nearest k neighbors point pair $P_s$ and $P_t$. Figure 1 presents the relationship of two points in UVW local coordinate system. The $U$,$V$,$W$ and parameters ($\alpha ,\beta ,\theta$) are calculated as (3):

.

$$\{\begin{array}{c}U=n_s \\ V=U\times \frac{\left(P_t-P_s\right)}{\Vert P_t-P_s{\Vert }_2} \\ W=U\times V \\ \alpha =V n_s \\ \beta =U\frac{\left(P_t-P_s\right)}{\Vert P_t-P_s{\Vert }_2} \\ \theta =\mathit{arctan}\left(W{n}_s,U{n}_t\right) \end{array},$$

(3)

where $\alpha$, $\beta$ and $\theta$ of k-neighborhood point pairs constitute triplets.

Each triplet feature value need be separated into some subdivisions, of which the number of occurrences should be recorded as the histogram description. Compared with PFH, the FPFH descriptor reduces the computation complexity from O(nk²) to O(nk): (1) FPFH increases neighborhood selection range from r to 2r, which also includes the neighborhood of all points in the neighborhood; (2) FPFH only computes the relationships of point and its k-neighborhood to form the simplified point feature histograms (SPFH). The neighboring values of SPFH are utilized to weight the final histogram, as computed in (4):

$$FPFH\left(P\right)=SPFH\left(P\right)+\frac{1}{k}{{\displaystyle \sum }}_{i=1}^k\frac{1}{Wigh{t}_i}SPFH\left(P_i\right)$$

(4)

2.3. Bipartite Graph Matching

A weighted bipartite graph structure $G=\left\{N{{\displaystyle \cup }}^{}V,E,W\right\}$ contains two types of vertices ($N$—the set of nodes ($\{n_1,n_2,\dots ,n_m\}$), $V$—the set of nodes ($\{v_1,v_2,\dots ,v_m\}$)), edges $E\in \left\{N\times V\right\}$ and $W$—the weight matrix ($\{w_{11},w_{12},\dots ,w_{mm}\}$). For $e_{ij}\in E$, each edge carries a weight $w_{ij}$ that represents the connection strength between $n_i$ and $v_i$, where i and j denote the i-th and j-th vertices in $N$ and $V$. If edge subset $M\subseteq E$ and no two edges in $M$ share a common vertex incident to them, the $M$ should be regarded as a match. Moreover, if $\left|M\right|=\left|N\right|=\left|V\right|$, the $M$ is a perfect match. The Kuhn−Munkres (KM) theorem is used to find a perfect matching of weighted complete bipartite graph [48]. The KM algorithm transforms the weights and sets an initial benchmark at each vertex of the X-set and Y-set. Then, the greedy algorithm preferentially selects the edge with the largest weight for matching. When there is another one where the largest edges of the two X vertices have the same Y vertex, the selected complete benchmark needs to be adjusted to form a new matching subgraph with largest weight. Finally, when an equal subgraph is found, the corresponding maximum weight matching is found.

2.4. TrICP Algorithm

Based on the least-squares (LS) optimal matching idea, the ICP algorithm puts the source cloud points $P=\left\{p_j\right|j=1,\dots ,n\}$ into the frame of the target point set $Q=\left\{q_i\right|i=1,\dots ,m\}$ to obtain consistent point cloud. As followed in (5), to repeat the calculation of optimal rigid body transformation until the convergence criterion is satisfied or the preset number of iterations is reached, the ICP algorithm can obtain a good estimate of the rotation $R$ and the translation $T$.

$$\begin{gathered} F\left(R,T\right)=\frac{1}{m}{\displaystyle \sum }_{i=1}^m{\left|\left|q_i-R^k{p}_j^{k-1}-T^k\right|\right|}^2, \left(1\le j\le n\right) \\ M{E}_k=\frac{1}{m}{\displaystyle \sum }_{i=1}^m\left|\left|q_i^k-p_i^{k-1}\right|\right| \end{gathered}$$

(5)

where $k$ is the iteration number, ME denotes the mean error after the k-th iteration computation, and the value of $\left|\left| M{E}_k-M{E}_{k-1}\right|\right|$ represents the preset convergence value.

As an optimization variant of ICP, TrICP utilizes the overlap ratio for points matching and Least-Trimmed Squares (LTS) to enhance the robustness for noise [14]. The pre-set overlap rate between the source point cloud $P$ and the target point cloud $Q$ is $\gamma$. The corresponding points number $n_c$ can be described as $n_c=$ $\gamma n$. TrICP reserves $n_c$ pairs of points that satisfy the overlap ratio after distance ordering by LTS and searches the transformation matrix by Singular Value Decomposition (SVD) method, as shown in (6).

$$\begin{gathered} d_i^2\left(R,T\right)=\Vert argmin\Vert q-p_i\left(R,T\right)\Vert -p_i{\left(R,T\right)\Vert }^2, \\ {d}_1^2\le d_2^2\le d_3^2\le \dots \le d_{n_c}^2 \\ Su{m}_{d_i}={\displaystyle \sum }_{i=1}^{n_c}{d}_{i:n_c}^2 \\ MSE=Su{m}_{d_i}/n_c \end{gathered}$$

(6)

where $d_i^2$ represents squared distance value of the i-th point pair, $Su{m}_{d_i}$ is the sum of $n_c$ $d_i^2$ value, and $MSE$ is the termination condition of the iterative computation.

3. The Proposed Coarse-to-Fine Registration Method Overview

In this section, we detail the coarse-to-fine registration algorithm based on bipartite graph perfect matching for scaled 3D data. The design scheme of proposed registration algorithm described in Figure 2 contains four phases: data pre-processing, TT strategy, coarse alignment and fine registration. The specific steps of these phases include filter process, point cloud normalization process, key points extraction, FPFH descriptor presentation, bipartite graph construction, correspondence point pair confirmation, initial rotation matrix estimation, scale factor estimation and fine registration.

3.1. Data Pre-Processing

Data pre-processing of PCD registration workflows mainly eliminates noise, which is essential for subsequent alignment processing. In this paper, the raw sampled 3D face data is scanned from depth camera RealSense with different views, which cannot be processed directly due to inevitable noise and outliers caused by a device’s inherent noise, object surface characteristics and object reflection [49]. Based on our previous study [7] and research [50], a hierarchical filter specifically designed for depth image point cloud is presented. Since the 3D model in the public library, such as the bunny rabbit, dragon and scanned room from an open-source library used in Section 4, does not involve the noise problem of sampling equipment, the ability of this filter should degenerate to a Gaussian filter. The solution of noise filtering in the acquisition process of non-depth camera equipment is not within the scope of this paper.

• Range filter. In the multi-view scene, the noise is mainly generated by the background because each scanning device will capture the opposite one or bracket. In this level, the filter performs a wide range and coarse-grained filtering by removing invalid points with a depth value of 0.
• Gray image segmentation filter. For large-scale outlier noise, to compute the threshold of target and background in an IR image obtained directly from the camera, the outliers are eliminated with a mathematical morphological opening operation in binary image converted from IR image. Regard the new binary image as an index map to filter the 3D point cloud.
• Gaussian filter. The Gaussian filter is utilized to remove small-scale noise points (when the density is less than the threshold value) located inside or at the boundary of point cloud sets. The $k$ neighborhood points around each point cloud are searched in turn, and the average Euclidean distance of the sampled point to its $k$ neighborhood points can be calculated. The distances of all points in the point cloud should form a Gaussian distribution, and the noise reduction can be achieved by providing the mean and variance value.

3.2. Top-Tail Strategy

To solve the problem of PCD alignment for the scale variation, the TT strategy, including normalization computation and scale estimation, has been designed to flexibly nest with multiple coarse alignment algorithms to adjust the initial position and scale factors of unregistered point cloud objects.

• Normalization Computation. As scale factors should improve the difficulty for the point cloud registration process, we firstly normalize the PCD for the subsequent coarse alignment to be proceed smoothly. The scale-adjusted problem can be described as (5) by extending (7):

  $$F\left(S,R,T\right)=\frac{1}{m}{\displaystyle \sum }_{i=1}^m{\left|\left|q_i-SR{p}_j-T\right|\right|}^2, \left(1\le j\le n\right)$$

  (7)

  where $S$ is the scale factor.
  Inspired by [46], the normalization process computes the maximum component of different dimensions and then calculates the proportion in different dimensions for each point, as described in (8).

  $$C{P}_{nor}\left(x,y,z\right)=\frac{CP\left(x_i,y_i,z_i\right)-CP{\left(x,y,z\right)}_{min}}{CP{\left(x,y,z\right)}_{max}-CP{\left(x,y,z\right)}_{min}}$$

  (8)

  where $CP\left(x,y,z\right)$ denotes the coordinate values of points in the point cloud, $i$ is the i-th index of points, label $nor$ denotes the normalized value of points, label $max$ is the maximum value of points, and label $min$ is the minimum value of points.
  Through the normalization process, the effect of translation and scale can be eliminated, and (7) can be simplified to (9).

  $$F\left(R\right)=\frac{1}{m}{\displaystyle \sum }_{i=1}^m{\left|\left|q_i-R{p}_j\right|\right|}^2, \left(1\le j\le n\right)$$

  (9)
• Scale estimation. To remove scale variation, the source point cloud P needs to be rotated with R (computed from coarse alignment) to obtain the PR set. Then, the maximum distance of PR points and Q points should be calculated, respectively. The scale can be computed by comparing these two maximum distances as (10).

  $$scale=\frac{\left|\left|Q_{max}-Q_{min}\right|\right|}{\left|\left|P{R}_{max}-P{R}_{min}\right|\right|}$$

  (10)
  Through this method, we cannot deal with the scale problem completely. However, the adjusted PCD is sufficient for the fine registration since most of the scale difference is eliminated.

3.3. The Coarse Registration

In the coarse alignment phase, the most important solution is to find the correspondent key point set. Instead of regarding points with smallest distance between the FPFH local feature descriptors as a matching set, the KM method is utilized to confirm the point set correspondence globally, which is used for initial rotation matrix estimation.

• Feature points extraction and description. In order to improve processing efficiency caused by a large amount of 3D data, the key points extracted by the 3D SIFT method and described with FPFH are used to simply represent the two pre-processed normalized PCD sets as $KN{P}^{\prime }$ and $KN{Q}^{\prime }$, respectively. From a local perspective, these feature points have the ability of being scale-invariant and rotation-invariant.
• Point pairs matching. The feature points extraction and description only take the local relationships of points into consideration. This step desires to incorporate global factors to determine the initial point pair matching. The graph is an ideal data structure for representing the global relationships of types of vertices. Among many graph structures [51,52], the bipartite graph structure is the most similar to the PCD structure, the simplest representation of the relationships between points and has the least computation amount. Moreover, inspired by the solution of the M-M assignment problem [53], we convert the point cloud feature matching process to handle the perfect matching based on the bipartite graph structure. The points in the source point cloud set $KN{P}^{\prime }$ and the target point set $KN{Q}^{\prime }$ denote the nodes of a bipartite graph structure $G$, as shown in Figure 3.

Figure 3. The bipartite graph structure representation of the $KN{P}^{\prime }$ and $KN{Q}^{\prime }$.

Figure 3. The bipartite graph structure representation of the $KN{P}^{\prime }$ and $KN{Q}^{\prime }$.

In this phase, the KM algorithm is implemented with the point cloud bipartite graph structure $G$. The number of the point cloud set $KN{P}^{\prime }$ and $KN{Q}^{\prime }$ is recorded as $n$ and $m$, respectively. In this bipartite graph structure $G$, a $\max \left(m,n\right)*\max \left(m,n\right)$ adjacent matrix should be considered to represent edges of $G$, where the value of the i-th row and j-th column represent the similarity weight component $\frac{similarity{\left(FPFH\right)}_{ij}}{{{\displaystyle \sum }}_{j=1}^{n}similarity{\left(FPFH\right)}_{ij}}$ of local feature descriptors. Here, the main objective is to find an assignment of point cloud set $KN{P}^{\prime }$ and $KN{Q}^{\prime }$, with maximum total similarity weight. During the perfect matching process, the KM algorithm is used to find the maximum matching weight in the bipartite graph with the following steps: (1) select $w_{ij}$ as the maximum weight of edges, which is connected to $kn{p}_i^{\prime }$ and $kn{q}_j^{\prime }$; (2) choose a weighted match to find a subgraph with weight coverage; (3) set a coverage until a good match is obtained for the subgraph equation. The pseudo-code of the point pairs matching computation is shown in Algorithm 1.

Algorithm 1. Point Pairs Matching Algorithm
1	input: pre-processed source point cloud $KN{P}^{\prime }$, pre-processed target point cloud $KN{Q}^{\prime }$,     FPFH descriptor for $KN{P}^{\prime }$ and FPFH descriptor for $KN{Q}^{\prime }$
2	output: initial point pairs set
3	begin
4	$N \leftarrow KN{P}^{\prime }$
5	$V \leftarrow KN{Q}^{\prime }$
6	create $\max \left(m,n\right)*\max \left(m,n\right)$ adjacent matrix and padding with zero
7	for each $e_i\in N$ and $e_j\in V$ do
8	adjacent matrix value $w_{ij}\leftarrow$ similarity weight proportion of $KN{P}^{\prime }$ and         $KN{Q}^{\prime }$ based on FPFH descriptor
9	append edge $\left(e_i,e_j\right)$
10	end for
11	generate bipartite graph structure $G=\left\{N{{\displaystyle \cup }}^{}V,E,W\right\}$ for $KN{P}^{\prime }$ and $KN{Q}^{\prime }$
12	initialize the matching subgraph as empty
13	for each $e_i\in N$ and $e_j\in V$ do
14	find max $w_{ij}$ for labeling $e_i$
15	find 0 for labeling $e_j$
16	end for
17	while complete match of equal subgraphs $\ne$ exist
18	find an augmentation path
19	while augmentation path $\ne$ success
20	modify labeling value and find new find an augmentation path
21	end while
22	add the augmented path to the matching subgraph
23	end while
24	output matching as initial point pairs set
25	end

• Mismatch elimination. The optimal matching result of weighted bipartite graph responds to point correspondence matching solution between $KN{P}^{\prime }$ and $KN{Q}^{\prime }$, but with some mismatching problems. In order to eliminate mismatched point pairs, two strategies about local descriptors’ similarity weight are mainly adopted and performed. At first, for each row i in the adjacent matrix, the corresponding maximum similarity location column j_max and the sum of similarity value of $kn{q}_{1,\dots ,j\_max,\dots n}^{\prime }$ with $kn{p}_i^{\prime }$ will be recorded. Moreover, only the ratio $\mathit{similarity}(FPFH{)}_{ij\_max}$ to ${\sum }_{j=1}^n \mathit{similarity}(FPFH{)}_{ij}$ at location j is recorded, and all other columns are filled in 0. By this method, the point pair with the largest similarity is retained and other similarity point pair combinations are excluded, so the execution efficiency and accuracy of the algorithm are improved. Second, a lower threshold will be set for the maximum similarity component ration, which is denoted as $threshol{d}_{low}$. When the optimum perfect maximum point pair is obtained, the $w_{ij}$ of each point pair set needs to be compared with the $threshol{d}_{low}$ value. If the $w_{ij}$ value is lower than $threshol{d}_{low}$, the correspondent point pair $\left(kn{p}_i^{\prime },kn{q}_j^{\prime }\right)$ should be abandoned to maintain the point pairs accuracy.
• Rotation estimation. The acquisition of a corresponding point pair set is the most critical step in rough registration process, which aims to calculate the rotation matrix between the point pair sets. For $KN{P}^{\prime }$ and $KN{Q}^{\prime }$, the rotation relationship is described in Equation (9). The singular value decomposition (SVD) algorithm is one of the most reliable orthogonal matrix decomposition methods, where $U$ and $Z$ represent two mutually orthogonal matrices and $A$ represents an angular matrix. Referenced in [54], the SVD method is applied to solve the rotation matrix R, as shown (11).

  $$H=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left(kn{p}_i-\overline{p}\right){\left(kn{q}_i-\overline{q}\right)}^T,$$

  (11)

  where $\overline{p}=\frac{1}{n}{\displaystyle \sum }_{i=1}^{n}kn{p}_i$ represents the center of mass $KN{P}^{\prime }$, and $\overline{q}=\frac{1}{n}{\displaystyle \sum }_{i=1}^{n}kn{q}_i$ denotes the center mass of $KN{Q}^{\prime }$.
  The SVD of matrix $H$ can be denoted as $H=UD{Z}^T$, where $D$ is $diag\left(d_i\right)$ and $d_1\ge d_2\ge d_3\ge 0$. When $\det \left(U\right)\det \left(V\right) \ge 0$, the $A$ should equal to $I_3$, otherwise $A$ is $diag\left(1,1,-1\right)$. If $rank\left(H\ge 2\right)$, the rotation matrix $R$ can be obtained by repeating $UA{Z}^T$ computation for several iterations. Moreover, the iterative process should end when the result of two adjacent iterations $\frac{\left|f\left(R_k\right)\right|}{n}$ is less than pre-set threshold, where $k$ is the iteration number.

3.4. The Fine Registration

The transformation relationship between two PCD contains translation matrix T, rotation matrix R and scale factor S. As the initial rotation matrix $R$ has been estimated from coarse registration method and the S can be obtained from TT strategy, the further fine registration is conducted with TrICP to update $R$ and compute $T$.

We list the pseudo-code of the proposed coarse-to-fine registration method in Algorithm 2.

Algorithm 2. The Coarse-to-Fine Registration Algorithm
1	input: source point cloud $P$, target point cloud $Q$, SVD iteration number    $iter\_nu{m}_{SVD}$, and TrICP overlap ratio $rati{o}_{TrICP}$
2	output: final transformation matrix
3	begin
4	pre-process $P$ and $Q$ to obtain $P^{\prime }$ and $Q^{\prime }$
5	normalize $P^{\prime }$ and $Q^{\prime }$ to obtain $N{P}^{\prime }$ and $N{Q}^{\prime }$
6	extract feature points from $N{P}^{\prime }$ as $KN{P}^{\prime }$ and obtain FPFH descriptor
	extract feature points from $N{Q}^{\prime }$ as $KN{Q}^{\prime }$ and obtain FPFH descriptor
7	create bipartite graph structure for $KN{P}^{\prime }\mathrm{and}KN{Q}^{\prime }$
8	explore perfect maximum match between $KN{P}^{\prime }$ and $KN{Q}^{\prime }$ as initial     correspondence points set
9	eliminate the mismatching point pairs
10	calculate $KN{P}^{\prime }$ center mass as $\overline{p}$ and $KN{Q}^{\prime }$ center mass as $\overline{q}$
11	compute matrix $H$ as $\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left(kn{p}_i-\overline{p}\right){\left(kn{q}_i-\overline{q}\right)}^T$
12	for iteration $i=1:iter\_nu{m}_{SVD}$do
13	if $rank\left(H\right)\ge 2$ then
14	while $\frac{\left|f\left(R_k\right)\right|}{n}>threshol{d}_{pre\_set}$
15	update $R=UA{Z}^T$
16	else
17	abort
18	end if
19	end for
20	rotate $KN{P}^{\prime }$ with initial rotation matrix $R$ to obtain $KN{P}^{\prime }R$ point set
	rotate $P^{\prime }$ with initial rotation matrix $R$ to obtain $P^{\prime }R$ point set
21	estimate scale factor $S$ with $\frac{\left|\left|KN{Q^{\prime }}_{max}-KN{Q^{\prime }}_{min}\right|\right|}{\left|\left|KN{P}^{\prime }{R}_{max}-KN{P}^{\prime }{R}_{min}\right|\right|}$
	adjust $P^{\prime }R$ as $S{P}^{\prime }R$ with $S$
22	search the nearest point from $Q^{\prime }$ in $S{P}^{\prime }R$ and calculate the distance squared $D^2$
23	sort $D^2$ from smallest to largest and calculate the sum of the first $rati{o}_{TrICP}*Q^{\prime }$
24	compute and output final rotation matrix $R$ and translation matrix $T$
25	end

4. Implementation and Performance Evaluation

In this section, we implement the proposed coarse-to-fine registration algorithm successfully with C++ language and point cloud library (PCL), which runs in ThinkPad X1 with Intel i7-10710 CPU and 16G memory. To comprehensively validate the performance of proposed algorithm, three groups of representative point cloud models have been selected, such as bunny rabbit, dragon from Stanford 3D model public library and room from PCL library. As shown in Figure 4, self-sampling raw 3D data, collected from two identical depth cameras in the same environment with a 45-degree difference, has also been considered to evaluate this algorithm. All testing PCD need pre-processing to eliminate noise interference with the hierarchical filter method proposed in this paper.

Table 2. The details of test data sets.

Name	PCD Presentation	Size	Description
bun045		40,097 points	bunny rabbit with 45 degrees rotation and obvious features
bun090		30,379 points	bunny rabbit with 90 degrees rotation and obvious features
dragon000		41,841 points	original dragon
dragon048		22,092 points	dragon with 48 degrees rotation and missing points
room000		112,586 points	original room with indistinct features
room040		112,624 points	room with 40 degrees rotation and indistinct features
face000		62,695 points	real face scanned data with normal features
face045		58,882 points	real face scanned data with 45 degrees rotation and normal features

comprehensively summarizes the details of the test data sets.

The first group consists of bun045 and bun090 as testing objects with low overlap (poor initial position). Even in some experimental parts, bun090 is scaled to form a testing data set with a higher difficulty coefficient. Dragon000 and scaled dragon048 form the second data set to simulate testing objects with missing data and scaling factors. The third data set, containing room000 and room040, is used to validate registration performance without obvious features. The real scanning data of face000 and face045 constitute the fourth data set, used to measure the practicality of the alignment methods.

The performance validation experiment was conducted in three rounds. In the first phase, we mainly compared the registration accuracy of ICP and TrICP with or without the coarse alignment method and TT strategy proposed in this paper. In second stage, four representative coarse-to-fine registration algorithms were selected and compared with our proposed method in alignment accuracy and time consumption. For the last round, we combined the proposed TT strategy with the coarse alignment process in the above four selected algorithms and compared the performance of four combined registration methods with our proposed method in multiple scaling factors.

For the whole experiment, to reduce the computation time, we processed all testing PCD with voxel filter down sampling method VoxelGrid from PCL open-source library. The VoxelGrid divides the point cloud into voxel size value cubes and regards the center of gravity in each voxel as a point, which can greatly reduce the number of original PCD. The voxel size can directly affect the efficiency of the down sampling result, and only an appropriate voxel size value can better preserve the local morphological features. After testing, we find that the down sampling effect is optimal when the voxel size is set to 0.002: the number of bun045 is reduced to 6813, the number of bun090 is reduced to 6041, the number of dragon000 is reduced to 7155, the number of dragon048 is reduced to 4529, the number of room000 is reduced to 5387, the number of room040 is reduced to 7590, the number of face000 is reduced to 8508, the number of face045 is reduced to 8667, and all the down sampling data sets do not affect the subsequent processing. When the voxel size is set as less than 0.002, the retained number of points is larger. On the contrary, when A is set as larger than 0.002, the local morphological structure of some testing data sets (such as room and scanned face) could be damaged, and the feature point extraction results are much affected. Considering the above factors, the voxel size value is uniformly set to 0.002.

Time consumption and root mean square error (RMSE) are two important criterions for the registration algorithm performance evaluation. The time consumption is the indicator to measure the algorithm efficiency, and the RMSE, as described in (12), is used to quantify the registration accuracy.

$$RMSE=\sqrt{\frac{1}{mt}{\displaystyle \sum }_{i=1}^{mt}\left|\left|R{p}_i+T-q_j\right|\right|}$$

(12)

where $mt$ denotes the matching points number and $(p_i$,$q_j$) is the matching point pair. The range of alignment accuracy obtained by the same registration algorithm in different PCD sets is not uniform, while different registration algorithms have the same range of alignment accuracy on the same PCD set. Since it is impossible to label a specific accuracy value for a particular alignment algorithm, we identify a registration algorithm with a higher accuracy by comparing the alignment RMSE of different algorithms on the same PCD set in the subsequent experiment.

4.1. The Availability Evaluation of Proposed Coarse Registration Method with TT Strategy

In the experiment, bun045-bun090, dragon000-dragon048 magnified 2 times and face000-face045 are selected to be the testing data set, as shown in Figure 5. We conduct tests on two ICP algorithms (ICP and TrICP) with different parameter-setting conditions, and the registration results are shown in Figure 6. The iteration number parameter can reflect ICP performance, and fewer iterations indicate a lower time consumption for processing the same data. Therefore, we set the iteration number parameter as 5, 10, 15, 30 and compare the corresponding alignment results. For the TrICP, a higher overlap ratio means to search for more points with a larger computation effort. We set the TrICP ratio to 0.65, 0.75, 0.85, 0.95 and compare the corresponding registration results. In Figure 7, we exhibit the registration results of the ICP and TrICP with the proposed coarse registration process and TT strategy (regarded as coarse-to-ICP and coarse-to-TrICP) under the same parameter-setting conditions of Figure 6. The registration errors are analyzed in

Table 3. The RMSE of ICP, coarse-to-ICP, TrICP and coarse-to-TrICP registration results.

Algorithm		Rabbit	Dragon	Face
ICP	5 iterations	0.0152612	0.0218693	0.0200825
	10 iterations	0.015228	0.0202673	0.0151241
	15 iterations	0.0152134	0.0199412	0.0137541
	30 iterations	0.015211	0.0187953	0.0132282
coarse-to-ICP	5 iterations	0.0135064	0.014808	0.0162803
	10 iterations	0.0134225	0.013352	0.0133654
	15 iterations	0.0134095	0.0132034	0. 0132222
	30 iterations	0.0134096	0.013181	0.0132146
TrICP	0.65 ratio	0.0261943	0.0322299	0.0349535
	0.75 ratio	0.0215244	0.0319884	0.0337509
	0.85 ratio	0.0160249	0.020968	0.0305069
	0.95 ratio	0.0155848	0.020663	0.0188006
coarse-to-TrICP	0.65 ratio	0.0126224	0.01676	0.0279336
	0.75 ratio	0.0124424	0.0162853	0.0202301
	0.85 ratio	0.0121801	0.0137099	0.0140438
	0.95 ratio	0.0118857	0.0132985	0.0133452

and Figure 8 and Figure 9. In visual presentation, the source point cloud data is marked as red and the target point cloud set as blue. Bun045, dragon000 and face000 are labeled in red, while bun090, dragon048 magnified 2 times and face045 are colored in blue.

The registration visual results of ICP and TrICP are presented in Figure 6a,b, respectively. We can obviously find that both of the algorithms have a bad registration effect in the bunny rabbit and scaled dragon, which indicates that ICP and TrICP have a bad ability to process PCD sets with a poor initial position and scale variations. Figure 7 illustrates the registration results of coarse-to-ICP and coarse-to-TrICP for bunny rabbit, scaled dragon and scanned face, which shows that the two algorithms can perform well in all three data sets. By comparing these visual results, our coarse registration method with TT strategy conducts very well, which significantly enhances comprehensive availability of original ICP and TrICP.

RMSE values with different iterations of ICP and ratio setting conditions of TrICP are recorded in Table 3. For the bunny rabbit and face testing sets without scaling factors, the coarse alignment process combined with TT strategy should degenerate to the coarse alignment method. As shown in the table, ICP and TrICP have higher RMSE values than coarse-to-ICP and coarse-to-TrICP, with the same bunny rabbit, face and parameter setting conditions, which validates the availability of the proposed coarse alignment method. For the scaled dragon data set, coarse-to-ICP and coarse-to-TrICP obviously have much lower RMSE values, which further demonstrates the effectiveness of the coarse alignment method combined with TT strategy.

In Figure 8, we compare the registration RMSE of ICP and coarse-to-ICP under different iterations and plot the results in a line chart. The line chart can be interpreted as: the x-axis shows different iterations; the y-axis represents registration error; the blue diamond connected by the dotted line indicates the RMSE trend of ICP; and the orange circle connected by the solid line shows the RMSE trend of coarse-to-ICP. In contrast to ICP, the coarse-to-ICP achieves convergence more efficiently (decrease from about 30 iterations to about 10 iterations) for a scanned face with good quality. Moreover, coarse-to-ICP can effectively avoid registration failure for poor initial position and scale variations data sets and can achieve convergence at about 15 iterations. We can conclude that the proposed coarse alignment method combined with TT strategy improves the performance of ICP.

Figure 9 displays the RMSE comparison of TrICP and coarse-to-TrICP with different ratio setting conditions. In this line chart, the x-axis represents multiple ratios, the y-axis shows RMSE value, the blue diamond connected by the dotted line presents RMSE variations of TrICP, and the orange circle connected by the solid line indicates RMSE of coarse-to-TrICP. By comparing the two RMSE curves, it can be analyzed that the proposed coarse registration method with TT strategy not only effectively compensates for the defects of TrICP in dealing with poor initial position and scale variations problems, but also speeds up algorithm convergence at a lower ratio setting. According to the observation of experimental results, when the ratio is set to 0.85, coarse-to-TrICP shows good performance. Moreover, comparing the RMSE of coarse-to-ICP and coarse-to-TrICP, the latter can achieve a lower RMSE, which indicates a more comprehensive and stable registration ability. Therefore, the TrICP algorithm used in subsequent experiments is set with a 0.85 ratio.

4.2. The Performance Evaluation of Multiple Coarse-to-Fine Registration Methods

In this part of the experiment, we mainly compare the performance of the complete coarse-to-fine registration algorithm proposed in this paper (combined with TT strategy) with four coarse-to-fine registration algorithms, such as NDT + ICP [33] (as the representative of probability model + ICP), ISS-3DSC-RANSAC + ICP [39] (as the representative of feature points with descriptors + coarse process + ICP), SAC-IA + ICP [33] (as the representative of coarse process + ICP) and K4PCS + ICP [35] (as the representative of feature points + coarse process + ICP).

The face000-face045, room000-room040, bun045-bun090, dragon000-dragon048 magnified two times and bun045-bun090 magnified 0.5 times are selected as the validation objects for this round of experiments. The visual registration results of different algorithms are exhibited in Figure 10. In these figures, the original source datasets are shown in green, the original target dataset is shown in blue and the registered datasets are in red. The total registration time, coarse phase time, fine process time and RMSE of each algorithm are recorded and analyzed in Figure 11.

For Figure 10a,b, the visual alignment results of face and room data sets apparently show that all five coarse-to-fine registration algorithms can effectively process PCD with obscure or general features. However, in Figure 10c, the visual results indicate that the alignment effect of a bunny rabbit with four compared coarse-to-fine registration algorithms, especially the NDT + ICP algorithm, is worse than the method proposed in this paper. This proves that our proposed algorithm is more robust in processing low overlap PCD sets. Moreover, Figure 10d,e reveals that four common coarse-to-fine algorithms fail in scaling alignment, except for the proposed method.

The numerical records and analysis of five coarse-to-fine algorithms about alignment time and accuracy for different datasets are presented in Figure 11: the horizontal coordinates are grouped by dataset name, where each group covers five registration algorithms (four compared algorithms and our proposed algorithm); the main vertical coordinate indicates the time in milliseconds, where the total registration time in blue bars, the coarse registration time in orange bars and the fine registration time in gray bars follow this axis; the secondary vertical coordinate shows the RMSE, where the registration errors of the five algorithms in all data models, represented by yellow boxes connected by yellow solid lines, follow this axis. As shown in Figure 11, the average registration time of NDT + ICP, ISS-3DSC-RANSAC + ICP, SAC-IA + ICP, K4PCS + ICP and our method for all data sets is 96.1208 ms, 17.2456 ms, 168.6878 ms, 10.377 ms and 76.081 ms, respectively. Analyzing from the perspective of algorithm execution efficiency, we can conclude that K4PCS + ICP is the highest and SAC-IA + ICP is the lowest, and our algorithm performs generally. For the non-scale point cloud object face and room, all five algorithms have similar small registration error, which indicates that these methods have close accuracy for aligning PCD with small position deviations or few features. However, our algorithm has the smallest 0.0117964 RMSE of bunny rabbit with poor initial position, which shows the proposed method has superior performance in processing low overlap rate PCD sets. Moreover, for data sets with scale variations, the RMSE of the four coarse-to-fine alignment algorithms is much higher compared to our proposed algorithm, which implies that these four algorithms fail to register scaling data sets and our proposed methods have more generality.

4.3. The Extensibility Evaluation of Multiple Registration Methods with TT Strategy

In order to test the performance of the coarse process in different coarse-to-fine algorithms combined with the TT strategy proposed in this paper, we scale the bunny rabbit and dragon data sets with multi factors (0.5, 1.5, 2) to simulate isotropic scale cases and form testing data sets: dragon000-dragon048 magnified 0.5 times, dragon000-dragon048 magnified 1.5 times, dragon000-dragon048 magnified 2 times, bun045-bun090 magnified 0.5 times, bun045-bun090 magnified 1.5 times and bun000-bun090 magnified 2 times. The experiment is conducted with the same coarse-to-fine registration algorithms mentioned in Section 4.2. The four course methods of NDT, ISS-3DSC-RANSAC, SAC-IA and K4PCS combine with the TT strategy, respectively, to modify NDT + ICP as NDT + ICP-TT, ISS-3DSC-RANSAC + ICP as ISS-3DSC-RANSAC + ICP-TT, SAC-IA + ICP as SAC-IA + ICP-TT, and K4PCS + ICP as K4PCS + ICP-TT. Figure 12 and Figure 13 respectively record the visual registration results of dragon and bunny rabbit with different scaling variations. In Figure 14, we compare scale estimation value with the real scale factor. Moreover, the registration errors are recorded and analyzed in

Table 4. The RMSE results of different coarse-to-fine registration algorithms with TT strategy under multiple scale factors.

Algorithm		Dragon (RMSE)	Rabbit (RMSE)
NDT + ICP-TT	0.5 scale	0.00331975	0.00613571
	1.5 scale	0.0108856	0.0169368
	2 scale	0.013279	0.0225726
ISS-3DCS-RANSAC + ICP-TT	0.5 scale	0.00332043	0.00564314
	1.5 scale	0.0109569	0.0169366
	2 scale	0.013332	0.0225729
SAC-IA + ICP-TT	0.5 scale	0.00331957	0.0056327
	1.5 scale	0.0108835	0.0169363
	2 scale	0.013251	0.0225733
K4PCS + ICP-TT	0.5 scale	0.00331974	0.00564319
	1.5 scale	0.0108855	0.0169364
	2 scale	0.013312	0.0225728
Our method	0.5 scale	0.00313056	0.00554097
	1.5 scale	0.00946669	0.0162888
	2 scale	0.013191	0.0202102

and Figure 15.

In Figure 12 and Figure 13, the original source data are colored in green, the scaled target object is labeled in blue and registered scaled-adjusted target object is shown in red. From the visual registration results in these figures, we find that the NDT + ICP-TT is less stable (large deviations in bun045 and bun090 magnified by 0.5), the performance of ISS-3DCS-RANSAC + ICP-TT, SAC-IA + ICP-TT and K4PCS + ICP-TT are close to each other, and our method is better. Our proposed TT strategy can extend the four coarse-to-fine registration algorithms to scale scenes effectively.

Figure 14 records scale factor estimation value of dragon and bunny rabbit data sets with different scale factors. The x-axis represents the scale factor category, the y-axis represents the scale estimation value, the blue bar indicates the scale factor estimation value of the dragon data set, the orange bar indicates the scale factor estimation value of the bunny rabbit object, and the red triangle shows the real scale factor value. For scale factor 0.5, the estimation value for dragon and bunny rabbit is 0.520670 (+4.13% error) and 0.488558 (−2.23% error). For scale factor 1.5, the estimation value for dragon and bunny rabbit is 1.61036 (+7.36% error) and 1.46719 (−2.19% error). For scale factor 2, the estimation value of dragon and bunny rabbit is 2.08243 (+4.12% error) and 1.95423 (−2.29% error). The average error of our scale-estimation value for dragon and bunny rabbit is −2.24% and +5.2%.

The registration results for RMSE of different coarse-to-fine algorithms with TT strategy are recorded in Table 4. As shown in the table, our method obtains the smallest errors of six scaled data sets, which indicates that our method’s performance is more stable and consistent than the other four coarse-to-fine registration algorithms with TT strategy. In Figure 15, we exploit boxplots to illustrate the RMSE for all data in Table 3: the x-axis represents five scale-adjusted coarse-to-fine registration algorithms; the y-axis represents the RMSE of each registration method; the central solid mark shows the median; and the bottom and top edges of the box are the 25th and 75th percentiles, respectively. Analyzing Table 3 and Figure 15, we explore that, except for NDT + ICP-TT which has a large RMSE in registering bun045 and bun090 magnified by 0.5, NDT + ICP-TT, ISS-3DSC-RANSAC + ICP-TT, SAC-IA + ICP-TT and K4PCS + ICP-TT have relatively similar errors, but they higher than our method for other data sets. Moreover, the comparison in Figure 10c and Figure 11, Table 4 and Figure 15 reveals that (1) NDT + ICP obviously fails to match bun045 and bun090, while NDT + ICP-TT has good registration results in bun045-bun090 magnified by 1.5 times and bun045-bun090 magnified by 2; (2) the registration RMSE of ISS-3DSC-RANSAC + ICP, SAC-IA + ICP and K4PCS + ICP is smaller than that of ISS-3DSC-RANSAC + ICP-TT, SAC-IA + ICP-TT and K4PCS + ICP-TT for the same PCD set, which is unavoidably caused by the scaling estimation error.

From the analysis above, we can conclude that our proposed TT strategy can be easily combined with current coarse-to-fine registration algorithms to adjust the source and target data sets to the same scale coordinate system, which effectively extend the applicability of alignment methods to scaling scenes. With a comprehensive consideration of the experimental results of Section 4.2 and Section 4.3, the efficiency of the coarse-to-fine registration algorithm (including the TT strategy) proposed in this paper is not optimal, but the excellent stable accuracy indicates this method has good extensibility and applicability whether including scaling factors or not.

5. Conclusions and Future Work

In this paper, we have proposed a coarse-to-fine registration algorithm with local feature extraction, feature description and bipartite graph global matching to confirm the initial correspondence, as well as TrICP to refine the transformation relationship. A weight threshold rejection method has been adopted to eliminate the initial correspondence error for further TrICP processing. Moreover, the TT strategy is designed to extend this alignment method to scaling cases, such as 3D object acquisition with different devices, by combing rough registration flexibly. The experiment is conducted in three phases and the results show that: 1. Compared with ICP on different iterations and TrICP on different overlap ratio settings for multiple PCD testing sets, the proposed coarse registration method combined with the TT strategy can accelerate the registration convergence speed, improve the registration accuracy and compensate the availability in scaling scenes effectively; 2. Compared with NDT + ICP, ISS-3DCS-RANSAC + ICP, SAC-IA + ICP and K4PCS + ICP algorithms, the RMSE and time consumption indicate that the proposed coarse-to-fine registration method has good efficiency, obviously higher accuracy and is more extensive; 3. Through comparing the RMSE of multiple coarse alignment methods combined with the proposed TT strategy, the corresponding original coarse-to-fine registration algorithms can be effectively improved in stability, accuracy and availability range. Although it is difficult to be widely applied due to the usage conditions, the ICP algorithm has been able to be used in some 3D data processing scenes, such as facial recognition, general consumer use, etc., with initial position, overlap rate and missing data conditions meeting the requirements. The comparison registration algorithms and the proposed algorithm in this paper are all based on the improvement of ICP. Specifically, the proposed registration method has good performance in aligning PCD sets with lower overlap, missing points, obscure features and scaling factors, which provides a better guarantee for extending the 3D data processing scenarios that have been applied. Future work should also address working with non-uniform density, improving the accuracy of TT strategy both in uniform and non-uniform density and optimizing the whole registration efficiency.