Next Article in Journal
Data Immunity in Near Field Radio Frequency Communication Systems—NFC as an Aspect of Electromagnetic Information Security
Previous Article in Journal
An Automated Geographical Information System-Based Spatial Machine Learning Method for Leak Detection in Water Distribution Networks (WDNs) Using Monitoring Sensors
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Application of Micro-Plane Projection Moving Least Squares and Joint Iterative Closest Point Algorithms in Spacecraft Pose Estimation

1
National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China
2
University of Chinese Academy of Sciences, Beijing 100049, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(13), 5855; https://doi.org/10.3390/app14135855
Submission received: 23 April 2024 / Revised: 20 June 2024 / Accepted: 25 June 2024 / Published: 4 July 2024

Abstract

:
Accurately determining the attitude of non-cooperative spacecraft in on-orbit servicing (OOS) has posed a challenge in recent years. In point cloud-based spatial non-cooperative target attitude estimation schemes, high-precision point clouds, which are more robust to noise, can offer more accurate data input for three-dimensional registration. To enhance registration accuracy, we propose a noise filtering method based on moving least squares microplane projection (mpp-MLS). This method retains salient target feature points while eliminating redundant points, thereby enhancing registration accuracy. Higher accuracy in point clouds enables a more precise estimation of spatial target attitudes. For coarse registration, we employed the Random Sampling Consistency (RANSAC) algorithm to enhance accuracy and alleviate the adverse effects of point cloud mismatches. For fine registration, the J-ICP algorithm was utilized to estimate pose transformations and minimize spacecraft cumulative pose estimation errors during movement transformations. Semi-physical experimental results indicate that the proposed attitude parameter measurement method outperformed the classic ICP registration method. It yielded maximum translation and rotation errors of less than 1.57 mm and 0.071°, respectively, and reduced maximum translation and rotation errors by 56% and 65%, respectively, thereby significantly enhancing the attitude estimation accuracy of non-cooperative targets.

1. Introduction

Orbital technologies can provide essential on-orbit maintenance services for spacecrafts, including satellites and space stations, that have experienced malfunctions, depleted fuel, or require module replacements [1]. These spacecrafts are classified as cooperative or non-cooperative based on pre-installed identifiable markings [2]. Cooperative spacecrafts are equipped with laser corner reflectors or optical markers that can actively communicate with maintenance spacecraft. Non-cooperative spacecrafts or satellite debris in need of on-orbit servicing lack such communication capabilities, and their structure, dimensions, and kinematics may be partially or completely unknown [3]. Accurate measurements of the relative positions and attitudes of these uncooperative spacecrafts are crucial for the success of space missions [4,5].
Currently, the sensors commonly used for target vision measurement in on-orbit missions include vision cameras [6], infrared cameras [7], LiDAR [8,9], and time-of-flight (ToF) cameras [10,11]. In space environments, vision cameras may encounter issues like overexposure blurring due to varying lighting conditions, while active sensors [12] can maintain stable operation. Active sensors provide 3D point cloud data with spatial coordinates and intensity information, offering unique advantages for spatial target attitude estimation. Therefore, this paper selects active sensors as the hardware device for acquiring target 3D point cloud data.
Current position estimation schemes utilizing 3D point cloud data are typically categorized into template matching based schemes and point cloud feature matching-based schemes. Template-based matching involves estimating position by matching a known target model, which is basically difficult to obtain in advance in practical applications. Opromolla et al. [13] developed an online 3D template matching method and proposed a model-based attitude estimation approach [14]. Zhao et al. [15] introduced a lidar-based fusion depth map and point cloud for attitude tracking, utilizing the Iterative Closest Point (ICP) algorithm to align the simplified sparse point cloud with the known target model point cloud to determine relative attitude. In simulation experiments, translation error and rotation error were no more than 1 cm and 0.5°, respectively. Liu et al. [16] used point cloud data generated by the Flash Lidar sensor with a target model acquired in advance to estimate the relative attitude for a close-range satellite using dense point cloud data directly, although noise and artifacts were not considered in the experiment. Concerning the feature matching approach, effectively performing noise reduction on acquired target point cloud data is crucial for accurate matching. Martínez et al. [17] proposed a ToF-based attitude estimation scheme, detecting changes in the target’s attitude at different time points by identifying spacecraft components like nozzles and fairings. However, only model data were used for validation in the experiments, leaving the performance in noisy environments unknown. Wang et al. [18] utilized the ICP method for aligning two frames of a point cloud and introduced a point median filtering technique to eliminate poorly corresponding point pairs. However, this approach may not be suitable for tumbling targets or complex geometric targets with high rotational speeds due to the simplistic target geometry considered. Hu et al. [19] leveraged the initial positional attitude values and key point information from stereo vision, obtained initial attitude data through joint calibration, and used the ICP algorithm for inter-frame alignment to determine the final attitude of the target. Zhou et al. [20] introduced a feature point selection method based on Neighborhood Feature Variance (NFV) to remove redundant points and retain more significant points for fine alignment, thereby enhancing the accuracy of attitude estimation for spatial targets. A limitation of this method is that the accuracy of key point extraction is constrained by the determination of the radius of the NFV neighborhood. It is evident that effectively filtering out redundant point noise in the pose estimation process significantly influences the accuracy of pose estimation.
In order to enhance the accuracy of point cloud alignment and minimize computational effort, this study introduces a novel point cloud-based approach for estimating the attitude of non-cooperative targets in close-range operations. Existing methods for point cloud alignment and attitude estimation often rely on model-based approaches or specific target components. For example, traditional methods like the ICP [21] algorithm and its variants such as Generalized Iterative Closest Point (GICP) [22] are widely used but have notable limitations. One significant drawback of the ICP algorithm is its sensitivity to initial alignment and noise, which can lead to suboptimal results, especially with high noise levels and outliers. Variants like GICP attempt to address some of these issues but still struggle with complex geometries and high-speed rotational targets. Additionally, methods relying on typical target components (e.g., nozzles and fairings) or pre-acquired models can be limited by their dependence on accurate prior knowledge, which may not always be available or reliable in all scenarios.
To overcome these limitations, our approach employs a moving least squares (MLS)-based microplanar projection technique. This technique transforms the original point cloud into a series of smooth manifold surfaces, effectively filtering out redundant and noisy points while preserving the geometric characteristics of the point cloud surface. By doing so, it enhances the accuracy of point cloud alignment and reduces computational effort. Furthermore, we introduce the Joint Iterative Closest Point (J-ICP) algorithm, which mitigates cumulative errors and achieves robust alignment of point clouds from satellite targets, addressing the issues of noise sensitivity and initial alignment dependency found in traditional methods.
The proposed method not only delivers significantly superior estimation results compared to traditional alignment algorithms but also operates efficiently without relying on extensive prior models or specific target features. This makes it particularly suitable for close-range operations involving non-cooperative targets, where quick and accurate attitude estimation is critical.
In summary, our main contributions are:
Introduction of the microplanar projection-based moving least squares (mpp-MLS) preprocessing method: This method enhances point cloud alignment accuracy by removing redundant and noisy points while preserving the geometric characteristics of the point cloud surface.
Development of an improved J-ICP algorithm: This algorithm precisely aligns point clouds by considering small angle transformations over multiple time points, thus avoiding local optimization issues commonly encountered in single-frame data alignment.
Validation through semi-physical experiments: The proposed method has been validated through semi-physical experiments, demonstrating significant reductions in maximum translational and rotational errors compared to classical methods, with only a slight increase in computational overhead. These results highlight the method’s potential for practical applications in space mission control.
The subsequent sections of this paper are structured as follows: Section 2 elaborates on the handling of redundant points and the alignment method. Section 3 presents the validation of the proposed method through semi-physical experiments conducted on simulated spatial targets. The results of these experiments showcase the method’s effectiveness and robustness. Finally, Section 4 delves into the implications of the findings and outlines potential future research directions.

2. Methods

This section outlines a non-cooperative target position estimation method utilizing point cloud feature point matching. The system, illustrated in Figure 1, is segmented into three key components for processing point cloud data collected at various time points: preprocessing, coarse matching, and fine matching. To enhance point cloud alignment accuracy and reduce computational load, initial on-site point cloud data containing the target are preprocessed to eliminate redundant and noisy surfaces by transforming the original point cloud into a series of smoother point clouds. Subsequently, rough alignment of point clouds between frames is conducted, establishing point relationships through calculation of Fast Point Feature Histograms (FPFH) [23] features to derive preliminary matching outcomes. Finally, fine alignment is achieved using the J-ICP algorithm to refine the estimated attitude parameters of the target.

2.1. Unordered Point Cloud Normal Vector Estimation

Current 3D sensing technology utilizes laser emissions to measure the time it takes for the emitted light to reflect from a target surface to the sensor, enabling the determination of point locations on the surface. However, the resulting point cloud data are often disordered and contain varying levels of noise, both of which can directly impact the accuracy of estimating normal vectors for the point cloud. Normal vectors are crucial as they describe the orientation of an object’s surface in three-dimensional space, providing essential information for understanding the object’s geometry. These normal vectors play a key role in localized surface information for spatial target position estimation, aiding in tasks such as surface fitting, feature extraction, and model matching.
When dealing with disordered point cloud data and unknown actual normal vectors, it is typical to assess the consistency between the estimated normal vectors of a point and those of its neighboring points. It is desirable for neighboring points to exhibit similar normal orientations in regions that are geometrically similar. The angular difference in the estimated normal vectors of neighboring points Δ θ i j can be calculated as Equation (1).
Δ θ i j = cos 1 n i n j n i n j
If the distribution of normal angle differences in the input point cloud is concentrated, it suggests that the normal of the point cloud closely aligns with those of its neighboring points. This indicates that the local surface of the point cloud experiences minimal directional changes, resulting in accurate and consistent normal estimation within that region. Conversely, a significant difference in normal directions signifies inaccurate normal estimation.

2.2. Fitting Micro-Planar Projections Using Moving Least Squares Process Based on Moving Least Squares Method

In order to enhance the registration accuracy of point cloud data, it is crucial to address the challenges posed by uneven sampling and the presence of noisy redundant points affecting normal vector estimation. Preprocessing the input point cloud set is essential not only to alleviate computational burden on the on-board computer but also to enhance alignment efficiency and accuracy by eliminating unnecessary details. While existing downsampling techniques like random, distance, and uniform methods exist, they primarily rely on a simplistic global selection approach that overlooks the local geometric characteristics of the point cloud. Building upon previous research by Marc et al. [24], we introduce a microplanar projection-based moving least squares (mpp-MLS) downsampling method for selecting projected points. This method aims to enhance alignment accuracy and efficiency by fitting a microplane to a local region of the target point cloud and effectively removing redundant points through projection.
First, from a piece of the point cloud with measurement noise, select the local region point set P i 3 ,   i 1 ,   .   .   .   , N centered on the index point r . We estimate a local plane by fitting these points with a cubic polynomial H = x | < n , x > D = 0 , x 3 , n 3 ,   n = 1 . The goal of the fit is to minimize the points P i to the plane H by reducing the sum of the weighted distances from the P i to the center point r of the plane, with the weight determined by the distance to the center point. Then the plane H can be obtained by locally minimizing Equation (2),
i = 1 N n , p i D 2 θ p i q
where θ is a smooth monotonically decreasing, constant positive function, and q is the projection of r on the H , r moves to the plane H along the direction of the plane normal vector n , which denotes that q is the projection of r on the H . Here, we could define q = r + t n where t , Equation (3) is redefined as follows:
i = 1 N n , p i r t n 2 θ p i r t n
where the smaller r is, the closer the local tangent plane H is to r , and q is the origin of the standard orthogonal coordinate system on the local tangent plane H .
Then, the surface in r domain is fitted by binary polynomial (as illustrated in Figure 2) to make q i be the projection of p i on H , and f i is the distance between p i and H . We can obtain the approximate coefficients of the polynomial g by minimizing the Equation (4).
i = 1 N g x i , y i f i 2 θ p i q
where ( x i , y i ) is the local coordinate representation of q i in H . The Gaussian function proposed in [25]:
θ ( d ) = e d 2 h 2
is used as the weight function, and the constant parameter h determines the selection range of the neighborhood point set. The features with dimensions smaller than h are smoothed out. The larger the h is, the smoother the projected point cloud will be.
In this study, we utilized satellite model point cloud data collected by a ToF camera and applied the mpp-MLS algorithm for processing the input point cloud. As shown in Figure 3, the initial point cloud is depicted in blue, while the point cloud processed by the algorithm is shown in red. It is evident that the red point cloud preserves the geometric features of the satellite surface while eliminating redundant and noisy points. By observing the XZ planar view, we further verified that the mpp-MLS algorithm accurately represents the planar features of the satellite point cloud. These findings indicate that mpp-MLS not only eliminates redundant noise points in the point set, but also more precisely captures the curvature and geometric features of the point set’s surface, thereby enhancing alignment accuracy.
The point cloud data acquired from 3D sensors typically lack normal vector information. In such cases, a parameter is utilized to estimate the approximate normal vector of the input point cloud. The mpp-MLS processing technique is applied to the input point cloud, which maintains its structural integrity and effectively smooths the surface while preserving key features. Figure 4 illustrates this process, where (a) shows the normal vector estimation directly on the original point cloud with noticeable dispersion in normal vector directions. In contrast, (b) depicts the normal vector estimation results after applying the mpp-MLS method, showcasing highly consistent normal vector directions in the point cloud’s planar neighborhood.
The normal vector consistency of the neighboring points in the two point clouds is calculated separately. The results are illustrated in Figure 5. Approximately 80% of the normal vector angle difference values estimated from the original point cloud fall within the range from 20° to 160°, whereas around 80% of the normal vector angle difference values estimated from the point cloud after mpp-MLS processing are distributed between 0° and 20°. This indicates that the algorithm proposed in this study effectively enhances the uniformity of the normal vector direction in the disordered point cloud, thereby improving registration accuracy.

2.3. Point Cloud Alignment and Parameter Optimization Point Cloud Pose Estimation and Optimization

The point cloud-based position estimation process is essentially a point cloud alignment operation. As the source point cloud P s o u r c e = p i and the target point cloud Q target = { q i } are given, the relative transformation relation T = ( R , t ) is calculated through the common feature part between them [26,27]. Given the resilient nature of local features within point clouds when faced with occlusion and noise, this study utilized the local features of a non-cooperative target’s point cloud to predict the bit position. The process flow is illustrated in Figure 1. Initially, the local features of two preprocessed point clouds were compared to establish correspondence and obtain a preliminary registration result. Following this, the transformation matrix was refined based on the coordinate positions of the points to achieve precise attitude adjustment.

2.3.1. Coarse Registration

Feature Description: In this study, we utilized the Fast Point Feature Histograms (FPFH) [28] descriptor to extract local features from a 3D point cloud. This method involves constructing a multidimensional point cloud by analyzing the spatial variances between the target point and its neighboring points, including the angular disparities among the normal vectors of the cloud. The resulting histogram accurately characterizes the geometric attributes of the point within its vicinity.
The FPFH descriptor captures the surface characteristics of a point cloud by considering the spatial and geometric relationships of each point and its neighborhood. The accuracy of the FPFH feature space heavily relies on the quality of normal vector estimates for points in the neighborhood. To enhance these estimates, this study employed the mpp-MLS algorithm, previously discussed, to handle point clouds with non-cooperative targets. The refined normal vector estimation achieved through the mpp-MLS algorithm enhances the precision and performance of FPFH descriptors in local feature extraction.
Feature Matching: The feature matching process between the source and target point clouds involves calculating the translation and rotation parameters. This is achieved by establishing one-to-one matching pairs through the calculation of point cloud FPFH features. Each point in the target point cloud is traversed to find the nearest-neighbor similar matching points in the source point cloud using the KD-tree search algorithm. By obtaining these matching pairs, we can determine the correspondence between the two sets of point clouds and subsequently calculate the necessary translation and rotation parameters.
During the feature matching process, a small number of mismatched pairs can have a significant impact on subsequent pose estimation. To mitigate this, the RANSAC algorithm [29] was employed in this study to enhance the robustness of the initial pose estimation. The algorithm operates by treating the dataset as comprising ‘inlier points’ (correctly matched pairs) and ‘outlier points’ (mismatched pairs or noise), and iteratively optimizes the model through random sampling and computation. This process aims to acquire higher accuracy results of the final pose estimation by effectively handling outliers and optimizing model parameters.

2.3.2. Fine Registration by J-ICP

After the coarse alignment, the two point clouds have been roughly aligned, but it is not sufficient to meet the accuracy requirements. Therefore, fine registration after coarse registration is a necessary step in the process of acquiring precise registration results of the given point cloud set.
The ICP algorithm [21] has become a classic algorithm in point cloud fine alignment due to its wide range of applications. This algorithm is based on matching point pairs to align surfaces. It involves finding nearest-neighbor point pairs between a source and target point cloud, calculating transformation parameters and an objective function, applying the transformation to a subset of the source cloud, and iteratively optimizing the objective function to achieve convergence and obtain the optimal transformation matrix. In essence, the ICP algorithm is employed to determine the transformation matrix between two frames by minimizing a specific objective function:
After the coarse alignment, the two sets of point clouds have been roughly overlapped, but they do not meet the required accuracy. Therefore, fine alignment is necessary to improve the accuracy of the point cloud alignment results. The ICP algorithm [21] is a well-known method for accurate point cloud alignment, which involves surface fitting based on aligning pairs of points. This algorithm identifies nearest-neighbor point pairs between the target and source point, calculates transformation parameters and an objective function, and then operates on a subset of the source point cloud. Through continuous iterations, the algorithm optimizes the objective function to achieve convergence conditions and determine the optimal transformation matrix. In essence, the ICP algorithm is utilized to estimate the transformation matrix between two frames in order to minimize the objective function defined in Equation (6).
min R , t E ( R , t ) = N i = 1 R p i + t p c l o s e s t   2 2
p c l o s e s t   = arg min q j Q p i q i 2 2
where p i represents a point in the point cloud set of the current frame, while q i is the corresponding point in the next frame, and Q denotes the set of all points in the point cloud of the next frame. The relationship between them is expressed in Equation (7).
When aligning point cloud data acquired during small angle tumbling maneuver for targets with symmetry features, it has been observed that the traditional ICP algorithm often becomes stuck in local optimal solutions, resulting in a notable increase in alignment error. To address this issue, this paper introduces a novel approach called Joint Iterative Closest Point (J-ICP), which aims to minimize the distance between point clouds from multiple frames by taking into account the point cloud data obtained at various time points during small angle transformations. Assuming that the point clouds P 1 , P 2 , , P n are acquired separately at t 1 , t 2 , , t n time, the goal of J-ICP is to find a set of transformations T 1 , T 2 , , T n 1 to align each point cloud P i with reference P r and minimize the total alignment error of multiple frames by iterating. The target function can be expressed as:
E T 1 , T 2 , , T n 1 = i = 1 n 1 p P i T i ( p ) N N P r T i ( p ) 2
where N N P r ( p ) denotes that the point p is the nearest neighbor in the reference point cloud P r , and · 2 denotes the squared Euclidean distance. The transformation T i is a rigid-body transformation with rotational and translational components that could align each of the P i with the P r .
For the transformation matrix under different frames obtained by the iterative nearest point algorithm T j i = R t 0 1 , which consists of a rotation matrix R and translation matrix t . The subscript letters indicate the transformation direction of this matrix, and the T j i subscript letter indicates the direction of transformation of the matrix, which is denoted as p i frame to p j frame to frame transformation, assuming that the current acquisition of three frames of the target point cloud data   P i , P j , P k That is, the joint alignment optimization formula is expressed as:
Δ T k j = T k i T j i + T k j 2
where   Δ T k j denotes the transformation matrix obtained by joint alignment of three frames from   j frame to   k frame. As the calculation process shown in Figure 6, we can find that this method can effectively avoid the local optimization problem caused by the symmetry feature of single-frame data alignment for it relies on the interrelationship of cross-frame data to guide the alignment process.
From the perspective of computational complexity, the proposed J-ICP algorithm has a complexity of O ( k n log n ) , where k is the number of iterations and n is the number of points in the point cloud. This complexity encompasses nearest neighbor search and transformation matrix computation, which is comparable to the complexity of the ICP algorithm. In contrast, the GICP algorithm, which incorporates point covariance information, has a computational complexity of O ( k n log n + k n ) . Although the theoretical complexity of J-ICP is similar to that of ICP, J-ICP reduces cumulative error and enhances registration accuracy by combining multiple ICP processes. Consequently, J-ICP maintains computational efficiency while typically achieving superior registration accuracy.

3. Semi-Physical Experiment and Analysis

In order to verify the effectiveness of the estimation algorithm proposed above, we conducted ground semi-physical experiments.

3.1. Experimental Environment Setup

The satellite model was considered a non-cooperative target, with its motion simulated by controlling the slide’s translation and the turntable’s rotation. The slide’s position control error was kept below 0.1 mm, while the rotary table’s angle control error was maintained under 0.01°, as illustrated in Figure 7.
The servicing spacecraft’s vision system was equipped with a ToF camera, whose specifications are detailed in Table 1. The point cloud of the target mockup was captured by the ToF camera at a frequency of one frame per second during the experiment.
To validate the robustness of the algorithm, we compared the method proposed in this paper with classical methods (ICP, GICP). All the algorithms mentioned above were implemented on a PC (I7-2.8 GHz, I7-7700 with 8 GB RAM) using Visual Studio 2019.

3.2. Results of Semi-Physical Experiments

In this study, we captured the point cloud data of a satellite model and the surrounding background using a ToF camera in a laboratory setting. Without proper preprocessing, the resulting attitude estimation would only be suitable for a dynamic target in a static environment and may not be applicable to real-world space environments.
We calculated the relative pose of the target following the process in Figure 1 The three-axis translation is represented by the three components of the translation vector t , and the three-axis rotational Euler angles need to be decomposed by the rotation matrix R . In this study, we assumed that the rotation angles α , β , and γ around the XYZ axes represented the roll, pitch, and yaw in the Euler angles, respectively.
Figure 8 illustrates the impact of the mpp-MLS algorithm discussed in Section 2.2. The point cloud data before optimization contained substantial noise due to the system noise, ambient light noise, and multi-path reflection noise inherent to the ToF camera. When displaying multiple frames of point cloud data, the XZ view showed the data overlapping, which complicated the accurate calculation of the attitude information for each frame. However, after applying the mpp-MLS algorithm, the XZ view clearly revealed significant angular rotation displacement in the point cloud data acquired at different frame rates. This processing greatly aided in the subsequent precise calculation of attitude parameters.

3.2.1. Coarse Registration after mpp-MLS Processing

Point cloud coarse alignment aims to approximately align two frames of point clouds that are initially far apart, with the objective of establishing a reliable initial estimate for subsequent fine alignment. This study explored the impact of resolution and noise criteria on point cloud alignment algorithms and employed the Random Sample Consistency (RANSAC) alignment algorithm to enhance the precision of coarse alignment.
In order to assess the robustness of the mpp-MLS algorithm under varying levels of noise, this study employed the RANSAC algorithm to conduct a comparative experiment on coarse alignment with noise both before and after processing, as illustrated in Figure 9.
In order to verify the robustness of the features extracted by the algorithm to the point cloud noise, in the experiments of this paper, we superimposed the Gaussian noise with the mean value μ = 0 and standard deviation σ = 0.01 ,   0.02 ,   0.03 ,   0.04 ,   0.05 ,   0.07 in the x , y , z directions of the original point cloud of the satellite to simulate the measurement noise of the 3D sensor. Coarse alignment was achieved by changing the noise standard of the point cloud and using the RANSAC algorithm. The obtained rotation error and translation error are shown in Figure 10. The results show that the alignment results using the point cloud processed by the algorithm proposed in this paper were better than those using the point cloud without processing, both in terms of point cloud resolution and point cloud noise criterion. Moreover, the error stability was also better than before in the case of low noise.

3.2.2. Fine Registration by J-ICP

To evaluate the accuracy of point cloud registration and relative pose measurement of the proposed algorithm, the transformation matrix T p r e d of the source point cloud P was computed using the registration algorithm for a given target point cloud Q . The actual transformation matrix between the two point cloud sets is denoted as T g t . The transformation error Δ T from Q to P is defined as in Equation (10).
Δ T = T p r e d   T g t 1 = Δ R Δ t 0 1
where Δ R is the 3 × 3 rotation residual matrix, and Δ t is the 3 × 1 translation residual vector. Ideally, if T g t is entirely accurate, Δ t should be the identity matrix. To represent the rotation error of the algorithm, we converted Δ R into Euler angles according to the ZYX rotation sequence. The three elements of Δ T were taken as the translation error.
In order to enhance the accuracy of position estimation results, this study utilized the J-ICP algorithm in conjunction with coarse alignment results to achieve precise target positioning.
Position estimation of a non-cooperative target was conducted under translational and rotational motions, with motion constraints set at a moving distance of 570 mm and a total change angle of 57°. The experimental results, depicted in Figure 11, illustrate the motion estimation curves and errors for the target with unknown motion. For spatial motion, the average estimation errors in the XYZ-axis direction for translation were 1.1 mm, 1.5 mm, and 0.25 mm, respectively. Additionally, the average errors for Eulerian roll, pitch, and yaw in the XYZ-axis were 0.045°, 0.019°, and 0.071° respectively.
As illustrated in Figure 12, we compared the fine alignment results obtained by applying three fine alignment methods (J-ICP, ICP, GICP) after coarse alignment. After the satellite model underwent translation and rotation by the slide rail and turntable, point cloud data were registered frame by frame. The XYZ axis translation error measured utilizing the GICP algorithm was (1.36, 2.46, 0.35) mm, while the XYZ axis Euler angle error was (0.112, 0.056, 0.059)°. However, by applying the method proposed above, those could be reduced to (1.07, 1.57, 0.25) mm and (0.071, 0.046, 0.019)° respectively. By implementing a joint registration algorithm, this research successfully eliminated cumulative errors and achieved improved accuracy in pose estimation parameters.
By comparing the performance of the proposed J-ICP algorithm with classical algorithms ICP and GICP in terms of mean translation error and mean rotation error, as shown in Table 2, we found that our algorithm significantly outperformed the classical algorithms. Specifically, compared to the classical ICP algorithm, the average translation error was reduced by approximately 48%, 56%, and 54%, while the average rotation error was reduced by approximately 56%, 48%, and 65%, respectively. These results clearly demonstrate the superiority of the J-ICP algorithm in pose estimation, not only enhancing the accuracy of attitude measurement but also significantly reducing system errors. This establishes a solid theoretical and experimental foundation for the practical application of pose estimation for non-cooperative spacecraft.
In addition, we compared the computational effort of different algorithms and used computation time as an evaluation metric. In order to ensure that no other irrelevant variables were introduced in the process of fine alignment, all methods underwent a standardized preprocessing step on the initial input point cloud before analysis. As depicted in Table 3, the total time taken by the methods presented in this paper for a single-frame point cloud estimation was approximately 98 ms, with a tracking frequency of up to 10 Hz.
In summary, compared with the classical method, this paper achieved a 56% reduction in maximum translational error in position estimation and a 65% reduction in maximum rotational error, with an 18.1% increase in computational consumption. These results suggests that the proposed method has the huge application potential in practical control.

4. Discussion

The proposed algorithm was validated using a mockup point cloud, which introduces several notable differences compared to an actual spacecraft’s point cloud data. These differences are essential to consider for the practical application of the algorithm.
One significant aspect is the resolution of the point cloud data. The ToF camera used in our laboratory had a resolution of 640 × 480 pixels, whereas LiDAR systems typically used for actual spacecraft produce higher-quality data. While this study focused on algorithm validation, future research must explore how variations in imaging resolution impact the algorithm’s performance. Another critical difference lies in the noise characteristics. Laboratory-acquired mockup data may include system noise from the camera, ambient light noise, and multipath reflection noise, which are relatively stable and controllable. In contrast, point cloud data from an actual on-orbit spacecraft would be subject to more diverse and complex noise sources, including space radiation, temperature fluctuations, and other environmental factors. Addressing these complex noise characteristics is crucial for ensuring the robustness of the proposed algorithm in real-world conditions. The surface reflectivity properties of the mockup and actual spacecraft also differ significantly. The materials and structures of the satellite model used in our experiments are simplified compared to those of real spacecrafts, which have more complex materials and reflectivity characteristics. These differences can affect the quality and accuracy of the point cloud data, necessitating future studies to investigate how varied reflectivity properties influence the performance of pose estimation algorithms. Additionally, the geometric complexity of the model used in the experiments presents another difference. The 1:35 scale Beidou satellite model lacks the detailed geometric intricacies of an actual spacecraft due to its smaller size. Real spacecrafts would produce point cloud data with richer geometric texture information. Therefore, future work should focus on validating the algorithm using actual spacecraft models to capture these complex details.
To enhance the applicability of the proposed algorithm to actual spacecraft, several measures can be considered. These include advanced data preprocessing and filtering techniques, such as Gaussian filtering, median filtering, and neighborhood-based statistical filtering, to reduce sensor noise and improve data reliability. Employing dual cameras to capture point cloud data and applying interpolation or downsampling techniques can help match the resolution of LiDAR scan data. Optimizing algorithm parameters and models based on actual spacecraft data, and validating the algorithm’s performance in real-world environments through simulations or practical tests, will ensure its robustness and applicability under various conditions.
The results from our experiments indicate that the proposed method significantly improves the accuracy of point cloud alignment and robustness against noise and outliers. However, the computational efficiency of our approach can still be further enhanced. In the context of accelerating algorithmic performance, integrating machine learning methods like PARSAC [30] can provide substantial benefits. PARSAC addresses the computational bottlenecks found in traditional methods such as CONSAC by predicting sample weights for all model instances simultaneously. Adopting a similar strategy, future work could focus on integrating a neural network to predict sample weights for the coarse registration process in point cloud alignment. By predicting both sample and inlier weights, the alignment process could be handled in parallel, significantly accelerating the computation. This would particularly benefit scenarios where rapid and accurate alignment is crucial, such as in dynamic space missions involving non-cooperative targets.
In the next step, we will focus on the following aspects: (1) leveraging machine learning techniques such as PARSAC for coarse registration can potentially reduce the computational overhead and improve the efficiency of point cloud alignment; (2) conducting pose estimation experiments on non-geometric targets in complex scenes to test the universality of this method; (3) introducing visible or near-infrared cameras to further enhance the target surface feature extraction, eliminate redundant features, and improve the accuracy of the attitude estimation; (4) deploying this paper’s method on an embedded platform for real-time performance verification.

5. Conclusions

This study introduces a novel technique for relative attitude estimation of non-cooperative targets using point cloud data. The proposed method enhances inter-frame alignment accuracy by selecting local regions of the point cloud for planar fitting, projecting noisy points at various distance scales, and implementing noise suppression techniques, including a noise filtering method based on moving least squares microplane projection (mpp-MLS) that preserves local features.
A key advancement presented in this research is the enhancement of the ICP alignment algorithm, referred to as the J-ICP algorithm. The J-ICP algorithm incorporates small-angle transformed poses between neighboring frames, effectively mitigating error growth in position estimation and reducing computational time. This approach significantly improves the accuracy and efficiency of the pose estimation process.
A semi-physical experimental platform was established to validate the methodology proposed in this paper. The experimental findings demonstrate that during coarse alignment, the method exhibited strong robustness towards noise interference encountered during point cloud acquisition. Furthermore, in fine alignment, the J-ICP method outperformed the traditional ICP approach by minimizing translation and Euler angle errors. Specifically, the cumulative translation error remained below 1.57 mm, while the cumulative Euler angle error stayed within 0.071 degrees, achieving a reduction of maximum translation and rotation errors by 56% and 65%, respectively. The total elapsed time for the process was approximately 98 ms, which enabled a pose tracking frequency up to 10 Hz.

Author Contributions

Conceptualization, Y.L. and J.Y.; methodology, Y.L. and Z.S.; validation, Y.L., Z.S. and Y.H.; formal analysis, J.Y.; investigation, Y.L.; resources, Y.W.; data curation, Y.L., Z.S., and J.Y.; writing—original draft preparation, Y.L.; writing—review and editing, Z.S. and Y.H.; supervision, F.Z.; funding acquisition, Y.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Key Research and Development Program of China, grant numbers: 2023YFC2604900 (sub-project grant number: 2023YFC2604904); the National Key Research and Development Program of China, grant number: 2023YFF0719800 (sub-project grant number: 2023YFF0719801).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Chen, L.; Huang, P.F.; Cai, J.; Meng, Z.J.; Liu, Z.X. A non-cooperative target grasping position prediction model for tethered space robot. Aerosp. Sci. Technol. 2016, 58, 571–581. [Google Scholar] [CrossRef]
  2. Opromolla, R.; Fasano, G.; Rufino, G.; Grassi, M. A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations. Prog. Aeosp. Sci. 2017, 93, 53–72. [Google Scholar] [CrossRef]
  3. Wang, S.J.; Cao, Y.X.; Zheng, X.; Zhang, T. A learning system for motion planning of free-float dual-arm space manipulator towards non-cooperative object. Aerosp. Sci. Technol. 2022, 131, 11. [Google Scholar] [CrossRef]
  4. Pesce, V.; Haydar, M.F.; Lavagna, M.; Lovera, M. Comparison of filtering techniques for relative attitude estimation of uncooperative space objects. Aerosp. Sci. Technol. 2019, 84, 318–328. [Google Scholar] [CrossRef]
  5. Deng, Z.; Li, A. A large depth-of-field virtual measurement network for non-cooperative 6DOF pose estimation in occlusion scenes. Measurement 2023, 218, 113111. [Google Scholar] [CrossRef]
  6. Volpe, R.; Palmerini, G.B.; Sabatini, M. A passive camera based determination of a non-cooperative and unknown satellite’s pose and shape. Acta Astronaut. 2018, 151, 805–817. [Google Scholar] [CrossRef]
  7. Cassinis, L.P.; Fonod, R.; Gill, E. Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft. Prog. Aeosp. Sci. 2019, 110, 14. [Google Scholar] [CrossRef]
  8. Perfetto, D.M.; Opromolla, R.; Grassi, M.; Schmitt, C. LIDAR-based model reconstruction for spacecraft pose determination. In Proceedings of the IEEE International Workshop on Metrology for AeroSpace (MetroAeroSpace), Torino, Italy, 19–21 June 2019; pp. 1–6. [Google Scholar]
  9. Zhu, W.S.; She, Y.C.; Hu, J.Q.; Wang, B.C.; Mu, J.Z.; Li, S. A hybrid relative navigation algorithm for a large-scale free tumbling non-cooperative target. Acta Astronaut. 2022, 194, 114–125. [Google Scholar] [CrossRef]
  10. May, S.; Dröschel, D.; Holz, D.; Wiesen, C.; Fuchs, S. 3D Pose Estimation and Mapping with Time-of-Flight Cameras. Int. Conf. Intell. Robot. Syst. (IROS) 2008. [Google Scholar] [CrossRef]
  11. Zhu, W.; Mu, J.; Shao, C.; Hu, J.; Wang, B.; Wen, Z.; Han, F.; Li, S. System design for pose determination of spacecraft using time-of-flight sensors. Space Sci. Technol. 2022, 2022, 9763198. [Google Scholar] [CrossRef]
  12. Tzschichholz, T.; Boge, T.; Schilling, K. Relative pose estimation of satellites using PMD-/CCD-sensor data fusion. Acta Astronaut. 2015, 109, 25–33. [Google Scholar] [CrossRef]
  13. Opromolla, R.; Fasano, G.; Rufino, G.; Grassi, M. A Model-Based 3D Template Matching Technique for Pose Acquisition of an Uncooperative Space Object. Sensors 2015, 15, 6360–6382. [Google Scholar] [CrossRef] [PubMed]
  14. Opromolla, R.; Fasano, G.; Rufino, G.; Grassi, M. Pose Estimation for Spacecraft Relative Navigation Using Model-Based Algorithms. IEEE Trans. Aerosp. Electron. Syst. 2017, 53, 431–447. [Google Scholar] [CrossRef]
  15. Zhao, G.P.; Xu, S.X.; Bo, Y.M. LiDAR-Based Non-Cooperative Tumbling Spacecraft Pose Tracking by Fusing Depth Maps and Point Clouds. Sensors 2018, 18, 3432. [Google Scholar] [CrossRef]
  16. Liu, L.J.; Zhao, G.P.; Bo, Y.M. Point Cloud Based Relative Pose Estimation of a Satellite in Close Range. Sensors 2016, 16, 824. [Google Scholar] [CrossRef] [PubMed]
  17. Martínez, H.G.; Giorgi, G.; Eissfeller, B. Pose estimation and tracking of non-cooperative rocket bodies using Time-of-Flight cameras. Acta Astronaut. 2017, 139, 165–175. [Google Scholar] [CrossRef]
  18. Wang, Q.S.; Lei, T.; Liu, X.F.; Cai, G.P.; Yang, Y.F.; Jiang, L.H.; Yu, Z.W. Pose Estimation of Non-Cooperative Target Coated With MLI. IEEE Access 2019, 7, 153958–153968. [Google Scholar] [CrossRef]
  19. Hu, L.; Sun, D.Q.; Duan, H.X.; Shu, A.; Zhou, S.S.; Pei, H.D. Non-Cooperative Spacecraft Pose Measurement with Binocular Camera and ToF Camera Collaboration. Appl. Sci. 2023, 13, 1420. [Google Scholar] [CrossRef]
  20. Zhou, Y.R.; Li, X.L.; Hu, H.X.; Su, L.X.; Du, H.; Fu, W.M.; Xu, L.J. Neighbor feature variance (NFV) based feature point selection method for three dimensional (3D) registration of space target. Measurement 2023, 222, 11. [Google Scholar] [CrossRef]
  21. Besl, P.J.; McKay, N.D. A Method for registration of 3-D shapes. In Proceedings of the Sensor Fusion Conference: Control Paradigms and Data Structures, Boston, MA, USA, 12–15 November 1991; pp. 586–606. [Google Scholar]
  22. Segal, A.V.; Haehnel, D.; Thrun, S. Generalized-ICP. In Robotics: Science and Systems V; Trinkle, J., Matsuoka, Y., Castellanos, J.A., Eds.; The MIT Press: Cambridge, MA, USA, 2010. [Google Scholar]
  23. Li, R.H.; Yang, B.; Lu, Q. Rough Registration Method for Point Cloud of Spatial Non-cooperative Target by Improving FPFH. In Proceedings of the 6th Symposium on Novel Optoelectronic Detection Technology and Applications, Beijing, China, 3–5 December 2019. [Google Scholar]
  24. Alexa, M.; Behr, J.; Cohen-Or, D.; Fleishman, S.; Levin, D.; Silva, C.T. Computing and rendering point set surfaces. IEEE Trans. Vis. Comput. Graph. 2003, 9, 3–15. [Google Scholar] [CrossRef]
  25. Levin, D. Mesh-independent surface interpolation. In Geometric Modeling for Scientific Visualization; Springer: Berlin/Heidelberg, Germany, 2004; pp. 37–49. [Google Scholar]
  26. Wong, J.M.; Kee, V.; Le, T.; Wagner, S.; Mariottini, G.L.; Schneider, A.; Hamilton, L.; Chipalkatty, R.; Hebert, M.; Johnson, D.M.S.; et al. SegICP: Integrated Deep Semantic Segmentation and Pose Estimation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)/Workshop on Machine Learning Methods for High-Level Cognitive Capabilities in Robotics, Vancouver, BC, Canada, 24–28 September 2017; pp. 5784–5789. [Google Scholar]
  27. Chen, J.; Wu, X.J.; Wang, M.Y.; Li, X.F. 3D shape modeling using a self-developed hand-held 3D laser scanner and an efficient HT-ICP point cloud registration algorithm. Opt. Laser Technol. 2013, 45, 414–423. [Google Scholar] [CrossRef]
  28. Rusu, R.B.; Blodow, N.; Beetz, M. Fast Point Feature Histograms (FPFH) for 3D Registration. In Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan, 12–17 May 2009; pp. 1848–1853. [Google Scholar]
  29. Le, V.H.; Vu, H.; Nguyen, T.T.; Le, T.L.; Tran, T.H. Acquiring qualified samples for RANSAC using geometrical constraints. Pattern Recognit. Lett. 2018, 102, 58–66. [Google Scholar] [CrossRef]
  30. Kluger, F.; Rosenhahn, B. PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample Consensus. arXiv 2024, arXiv:2401.14919. [Google Scholar] [CrossRef]
Figure 1. Flowchart of the pose estimation.
Figure 1. Flowchart of the pose estimation.
Applsci 14 05855 g001
Figure 2. Schematic of discrete group point projection in the node neighborhood.
Figure 2. Schematic of discrete group point projection in the node neighborhood.
Applsci 14 05855 g002
Figure 3. Spatial target model point cloud resampling results; blue is the initial point cloud, red is the resampled point cloud. (a) XY view; (b) XZ view.
Figure 3. Spatial target model point cloud resampling results; blue is the initial point cloud, red is the resampled point cloud. (a) XY view; (b) XZ view.
Applsci 14 05855 g003
Figure 4. Unordered point cloud normal vector estimation results. (a) Estimation of normal vectors of the original point cloud; (b) point cloud normal vector estimation results after mpp-MLS processing.
Figure 4. Unordered point cloud normal vector estimation results. (a) Estimation of normal vectors of the original point cloud; (b) point cloud normal vector estimation results after mpp-MLS processing.
Applsci 14 05855 g004
Figure 5. Point cloud phase consistency detection.
Figure 5. Point cloud phase consistency detection.
Applsci 14 05855 g005
Figure 6. Schematic of joint alignment.
Figure 6. Schematic of joint alignment.
Applsci 14 05855 g006
Figure 7. Model of the target satellite, slide, rotor, and camera system in the laboratory environment.
Figure 7. Model of the target satellite, slide, rotor, and camera system in the laboratory environment.
Applsci 14 05855 g007
Figure 8. Point cloud before and after optimization using the mpp-MLS algorithm.
Figure 8. Point cloud before and after optimization using the mpp-MLS algorithm.
Applsci 14 05855 g008
Figure 9. Coarse registration results (model point cloud in blue, field point cloud after alignment in red). (a) Coarse registration without mpp-MLS processing; (b) coarse registration after mpp-MLS processing.
Figure 9. Coarse registration results (model point cloud in blue, field point cloud after alignment in red). (a) Coarse registration without mpp-MLS processing; (b) coarse registration after mpp-MLS processing.
Applsci 14 05855 g009
Figure 10. Alignment errors of the RANSAC method for different noise criteria. (a) Translation error; (b) rotation error.
Figure 10. Alignment errors of the RANSAC method for different noise criteria. (a) Translation error; (b) rotation error.
Applsci 14 05855 g010
Figure 11. Pose-estimation results of our method for the translation experiment. (a) Translation of the XYZ axis; (b) rotation of the XYZ axis; (c) position errors of the XYZ axis; (d) rotation error of the XYZ axis.
Figure 11. Pose-estimation results of our method for the translation experiment. (a) Translation of the XYZ axis; (b) rotation of the XYZ axis; (c) position errors of the XYZ axis; (d) rotation error of the XYZ axis.
Applsci 14 05855 g011
Figure 12. Comparison of errors of fine alignment results. (a) X-axis translation; (b) Y-axis translation; (c) Z-axis translation; (d) X-axis Euler angle; (e) Y-axis Euler angle; (f) X-axis Euler angle.
Figure 12. Comparison of errors of fine alignment results. (a) X-axis translation; (b) Y-axis translation; (c) Z-axis translation; (d) X-axis Euler angle; (e) Y-axis Euler angle; (f) X-axis Euler angle.
Applsci 14 05855 g012
Table 1. Technical specifications of the ToF cameras.
Table 1. Technical specifications of the ToF cameras.
ParameterValue
Resolution640 × 480 px, 0.3 MP
Pixel Size10.0 µm (H) × 10.0 µm (V)
Illumination4 × VCSEL laser diodes, Class1, @ 850 nm
Lens Field of View69° × 51° (nominal)
Table 2. Comparison of pose parameter errors among the ICP, J-ICP, and GICP algorithms.
Table 2. Comparison of pose parameter errors among the ICP, J-ICP, and GICP algorithms.
ICPGICPJ-ICP
Mean Translation Error—X (mm)2.06
(±1.50)
1.36
(±0.94)
1.07
(±0.81)
Mean Translation Error—Y (mm)3.59
(±3.02)
2.46
(±2.01)
1.57
(±1.14)
Mean Translation Error—Z (mm)0.54
(±0.42)
0.35
(±0.27)
0.25
(±0.20)
Mean Rotation Error—pitch (°)0.162
(±0.135)
0.112
(±0.89)
0.071
(±0.052)
Mean Rotation Error—yaw (°)0.089
(±0.065)
0.056
(±0.041)
0.046
(±0.036)
Mean Rotation Error—roll (°)0.055
(±0.041)
0.059
(±0.043)
0.019
(±0.013)
Table 3. Comparison of the amount of computation for different methods.
Table 3. Comparison of the amount of computation for different methods.
Time Consumption (ms)Method
ICPGICPJ-ICP
Mean839298
Std. Dev465
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, Y.; Han, Y.; Yao, J.; Wang, Y.; Zheng, F.; Sun, Z. Application of Micro-Plane Projection Moving Least Squares and Joint Iterative Closest Point Algorithms in Spacecraft Pose Estimation. Appl. Sci. 2024, 14, 5855. https://doi.org/10.3390/app14135855

AMA Style

Li Y, Han Y, Yao J, Wang Y, Zheng F, Sun Z. Application of Micro-Plane Projection Moving Least Squares and Joint Iterative Closest Point Algorithms in Spacecraft Pose Estimation. Applied Sciences. 2024; 14(13):5855. https://doi.org/10.3390/app14135855

Chicago/Turabian Style

Li, Youzhi, Yuan Han, Jiaqi Yao, Yanqiu Wang, Fu Zheng, and Zhibin Sun. 2024. "Application of Micro-Plane Projection Moving Least Squares and Joint Iterative Closest Point Algorithms in Spacecraft Pose Estimation" Applied Sciences 14, no. 13: 5855. https://doi.org/10.3390/app14135855

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop